Auditing sitemap.xml: urlset vs index, URL counts, and duplicate loc checks

A sitemap is an XML file that lists candidate URLs for crawlers (and, for large sites, child sitemap files). Search engines treat it as a hint, not a guarantee of indexing—but a broken, empty, or duplicated sitemap still wastes crawl budget and confuses tooling. Auditing structure before deploy is cheaper than debugging “why isn’t this page in Search Console?” later.

This guide covers urlset vs sitemapindex, URL / loc counts, lastmod, and duplicate or empty loc checks. Paste and analyze locally so staging hosts and unreleased paths never need to leave your machine—see Why “local only” matters for developer tools. Pair sitemaps with robots.txt Sitemap tips: robots.txt points crawlers at the file; the sitemap lists what you want discovered.

Two document kinds: urlset vs sitemapindex

The sitemaps.org protocol defines two root shapes. Mixing them up is the most common structural mistake.

Root	Child entries	Each entry’s loc points to
urlset	<url>	A page URL you want listed
sitemapindex	<sitemap>	Another sitemap XML URL (often a urlset)

urlset (page list)

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/</loc>
    <lastmod>2026-09-01</lastmod>
  </url>
  <url>
    <loc>https://example.com/pricing</loc>
  </url>
</urlset>

Optional siblings under <url> include lastmod, changefreq, and priority. Engines weight these lightly compared to a correct absolute loc; do not invent daily changefreq values to “rank higher.”

sitemapindex (sitemap of sitemaps)

When a site exceeds practical size limits (often discussed around 50,000 URLs or ~50 MB uncompressed per file), split into multiple urlsets and publish an index:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://example.com/sitemaps/pages.xml</loc>
    <lastmod>2026-09-01</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://example.com/sitemaps/blog.xml</loc>
  </sitemap>
</sitemapindex>

Reference the index (or each child) from Sitemap: in robots.txt and in Search Console. Do not put page URLs inside a sitemapindex, or child sitemap URLs inside a urlset—crawlers expect the matching shape.

Namespaces may appear as the default xmlns or as prefixed tags (sm:urlset). Local auditors that match on local name (urlset / sitemapindex, url / sitemap, loc) handle both.

What counts actually mean

When you paste a file into an auditor, useful live stats usually include:

Stat	Meaning
Document kind	Root is urlset, sitemapindex, or something else (unknown / incomplete stats)
URL entries	Number of <url> elements in a urlset
Sitemap entries	Number of <sitemap> elements in an index
loc count	Non-empty <loc> values under those entries
lastmod count	How many <lastmod> elements appear anywhere
Empty locs	<loc> present but blank or whitespace-only
Duplicate locs	Same trimmed loc string appears more than once

Entry count and loc count should usually match. A gap often means a missing or empty loc under an entry. An unknown root (for example a bare <urls> typo or HTML mistaken for XML) means you should not trust counts until the document is a real sitemap.

lastmod is optional. Having many lastmod values does not prove freshness to crawlers; inconsistent or future-dated lastmods can look worse than omitting them. Prefer honest dates when you include them.

Duplicate and empty loc checks

Empty loc: Invalid for consumers. Fix generators that emit <loc></loc> or whitespace-only text. Every entry needs one absolute HTTPS URL (scheme + host + path).

Duplicate loc: The same URL listed twice wastes space and can confuse “URL count” dashboards. Duplicates are often case-sensitive on the exact string—https://example.com/A and https://example.com/a may both appear if your site is case-sensitive on the path. Trim whitespace before comparing. Common causes:

• CMS merges that concatenate two feeds without deduping
• Trailing-slash vs no-slash variants listed as separate locs (normalize with redirects and slug rules first)
• Staging URLs accidentally mixed into a production export

Duplicates inside one file are the easy catch. Cross-file duplicates across several urlsets under one index need a build-step or script; still start by cleaning each file so individual counts are trustworthy.

Pretty-print vs minify (and what “valid” means here)

Like other browser XML tools, a sitemap formatter typically:

1. Parses with the browser’s XML parser (well-formed check—not full XSD schema validation).
2. Analyzes root kind, entries, locs, and warnings.
3. Serializes with pretty-print (2- or 4-space indent) or minify for smaller downloads.

That is the same “parse, then transform” pattern as validating and formatting XML in the browser. Pretty-print may normalize attribute quoting or empty tags; the logical tree should match what the parser understood. For reviews and diffs, pretty-print first; for production CDN delivery, minify is fine if your CDN already gzips XML.

Very large sitemaps can stress tab memory. Prefer splitting into an index of smaller urlsets rather than pasting a multi‑megabyte blob into a text box.

A safe local audit workflow

1. Export or build sitemap.xml (or each child under an index) from your CMS / static generator.
2. Paste into a local auditor; confirm kind is urlset or sitemapindex as intended.
3. Check entry vs loc counts, empty locs, and duplicate locs; fix the generator if warnings appear.
4. Pretty-print for human review or minify for ship size; copy or download.
5. Point robots.txt Sitemap: at the absolute HTTPS URL of the index (or single urlset)—see Writing robots.txt.
6. Submit or refresh in Search Console / Bing Webmaster after deploy; confirm listed URLs are publicly reachable and not Disallowed.

Sitemaps advertise candidates; they do not override robots.txt Disallow, login walls, or noindex. Keep those layers consistent.

Try it locally in your browser

Use the Sitemap XML formatter to:

• Paste a urlset or sitemapindex (default namespace or prefixed tags).
• Review live URL / sitemap counts, lastmod totals, empty loc warnings, and duplicate loc URLs.
• Pretty-print or minify with 2- or 4-space indent, then copy or download sitemap.xml.

DOMParser, analysis, and serialization run only in this tab—your sitemap is not uploaded to LocalTools.

Related reading

• Writing robots.txt: User-agent groups, Allow/Disallow, and Sitemap tips — how Sitemap: lines complement the XML file.
• Validating and formatting XML in the browser — well-formedness vs schema validation and pretty-print caveats.
• Slug rules for URLs and filenames — stable public paths that belong in loc.
• Why “local only” matters for developer tools — why draft SEO XML belongs in-tab, not on a random paste site.