Analysis Published 5 November 2026 10 min read

How Googlebot Decides Which Sitemap to Crawl First, and Where Crawl Budget Comes In

There is no first sitemap. Googlebot rereads sitemap files on its own schedule, puts their URLs in the same queue as everything it found through links, and orders that queue by demand and by what your server can take. One field in the file moves a URL up, and only while it tells the truth.

Roman Makuev
Last updated

Googlebot doesn’t decide which sitemap to crawl first, because it doesn’t crawl sitemaps as a queue. A website with multiple sitemaps gets no crawl priority for the one listed first, and other search engines work the same way. An XML sitemap file is a URL like any other: Google fetches it on its own schedule, takes the URLs out of it, and puts them into the same crawl queue as every URL it found through links. Googlebot prioritizes that queue URL by URL, by crawl demand, meaning how popular a URL is and how stale Google’s copy has become, and capped by the crawl capacity your server has shown it can take. The order of files in a sitemap index or in robots.txt plays no part, and of everything inside the file only lastmod can move a URL forward, and only while it has been accurate.

Below: what Googlebot does with a sitemap file, what changed when Google rewrote its crawl budget documentation on 22 July 2026, what makes lastmod count, and how to see in your own logs which sitemap Googlebot actually reads first.

What Googlebot does with a sitemap file

A sitemap is a discovery source. Google finds it in one of two ways, a Sitemap: line in robots.txt or a submission in Search Console, and from then on it rereads the file when its systems decide to. The Sitemaps report in Search Console shows the result per file: when Google last read it, whether it could, and how many URLs it discovered from it. Google’s sitemap documentation calls a submission a hint that doesn’t guarantee the file gets downloaded or used, and the one way to push a change, the sitemap ping endpoint, was announced as going away in June 2023 because Google found pings “not very useful.”

Once the file is read, its URLs stop being “sitemap URLs.” Each one is a candidate in the crawl queue next to the URLs found through internal links and external links, and from that point it is scheduled on the same terms. A URL the site lists in its sitemap and never links to, an orphan page, arrives with less to recommend it. Popularity is one of the two inputs to crawl demand in Google’s documentation, and a URL nothing links to has no popularity Google can see, which is why I read a sitemap-only URL as a discovery problem before I read it as a crawl budget one.

What in a sitemap Google uses for crawl order

Most of what people adjust in a sitemap to change crawl order has no effect on it.

ElementUsed for crawl orderWhat it means in practice
<loc>Discovery onlyTells Google the URL exists; says nothing about when to fetch it
<lastmod>Yes, when consistently accurateThe one field that can bring a recrawl forward
<priority>NoIgnored by Google; 1.0 on every URL changes nothing
<changefreq>NoIgnored by Google; “daily” doesn’t make a daily crawl
Order of URLs in a fileNoThe file is read whole and its URLs are scheduled one by one
Order of sitemaps in an indexNoGoogle documents no ordering; the first file listed gets no head start
Order of Sitemap: lines in robots.txtNoEach line is a discovery source; robots.txt can be cached for up to 24 hours, so a new line isn’t seen at once
The sitemap’s file name or URLNoChanging it daily to force a fresh read is, in John Mueller’s words from September 2026, “a bad idea”

The limits are the protocol’s: 50,000 URLs or 50 MB uncompressed per file, and an index file to group more. Splitting a sitemap is useful, though for diagnosis rather than priority, which the section on splitting below comes back to.

Crawl budget after Google’s July 2026 rewrite

Crawl budget is the set of URLs Googlebot can and wants to crawl on a website, and it matters for SEO because a page that isn’t crawled isn’t indexed and can’t rank. Google rewrote its “Optimize your crawl budget” documentation on 22 July 2026, and most of what is written about sitemaps and crawl budget predates it. The model is the same two parts. Crawl demand is how much Google wants to crawl a site’s URLs, from how many it knows about, how popular they are and how stale its copies are. The crawl capacity limit, also called hostload, is how much crawling the server can take without slowing down. What changed is how the capacity half is described.

  • Everyone starts at the same place. “Every site starts with the same default, conservative crawl capacity limit,” and Google’s systems raise it over time where there is demand and the server stays healthy.
  • Capacity is measured in connection time. The limit now covers the total time the server spends holding connections open for Google, so slow responses use it up faster than fast ones, independent of how many URLs get fetched.
  • The limit is shared. “The crawl capacity limit is shared across all crawlers,” and high demand from one can reduce what’s available to the others. AdsBot after a campaign launch or the Shopping crawler after a feed change draws from the same pool as Googlebot.
  • Errors and throttling are named. Server errors (5xx) and rate-limiting signals such as HTTP 429 lower the limit; a firewall rate-limiting Googlebot is spending the crawl budget it is meant to protect.
  • 304 is recommended. The page now advises supporting HTTP caching with 304 Not Modified, so a recrawl of an unchanged page costs a header and no body.

For the sitemap question this means the order in which Google gets to the URLs in your sitemaps is set mostly outside the sitemap: by how fast and how steadily the server answers, by what other Google crawlers are taking from the same allowance, and by demand for each URL. The sitemap decides which URLs are candidates and, through lastmod, which of them Google has a reason to revisit.

lastmod: the one field that moves a URL up the queue

Google uses lastmod if it is consistently and verifiably accurate, which in practice means Google compares the date with what it sees when it fetches the page. In June 2024 Google added to its sitemap documentation what counts: the value should reflect “the last significant update to the page,” and “an update to the main content, the structured data, or links on the page is generally considered significant, however an update to the copyright date is not.”

The common way to lose it is the deploy timestamp. A CMS or a build pipeline that writes the generation time into every lastmod tells Google every URL changed on every release, Google fetches a sample, finds the pages the same, and the field stops counting. The fix is to write the date of the last content change from the database, never the date the file was built, and to accept that an honest sitemap shows most URLs with old dates. Old dates are what accuracy looks like on a site where most pages don’t change.

The one deliberate exception to “only live, indexable URLs in a sitemap” uses the same field. When a batch of URLs has been removed or redirected and you want Google to notice sooner, John Mueller’s advice since 2019 has been a temporary sitemap that lists those URLs with a lastmod set to the date of the change, kept until the Pages report shows them processed and then deleted. It works through lastmod, so it only works on a site whose dates Google still believes.

How to see which sitemap Googlebot reads first on your site

The literal question has an answer, and it’s in your logs rather than in Google’s documentation. Google publishes no rule for how often it rereads each sitemap file, so I read what it actually does, in four steps.

  1. Sitemap fetches from verified Googlebot. Filter 30 days of server log files to requests for your sitemap files from Googlebot, verified by reverse DNS or against Google’s published IP ranges, because anything can claim the user agent. For each file, count the fetches and note the dates. This is the order and frequency Google reads your sitemaps in, and it is usually uneven: the index and the files whose contents change get read more often than files that never do.
  2. Match it against Search Console. The Sitemaps report’s Last read column should agree with the newest fetch in the logs for each file. A file the logs show fetched daily and the report shows read weeks ago is a file Google fetched and couldn’t parse or didn’t use; open its status.
  3. Time from sitemap to fetch, per file. For URLs added to each sitemap in the period, find the first Googlebot request for each URL in the logs and the days between the URL appearing and being fetched. This is the number that answers what people mean by “which sitemap gets crawled first”: the sitemap whose new URLs get fetched within a day is being trusted, and one whose new URLs wait two weeks is not, or its URLs have no links pointing at them.
  4. Crawl stats for the context. Search Console’s Crawl stats report splits requests by purpose, discovery against refresh, and by file type, where sitemap fetches sit under XML. A discovery share that collapses after a release, or response times climbing in the same weeks, explains a slower step 3 better than anything in the sitemap does.

On a small site the numbers are small and still readable: a few dozen sitemap fetches a month and a handful of new URLs are enough to see whether new pages get fetched in a day or a week.

Splitting sitemaps for diagnosis, not for priority

Splitting one sitemap into several doesn’t change crawl order, and it is still worth doing, because Search Console’s Pages report can be filtered by sitemap. With a split by template, “Discovered – currently not indexed” stops being a site-wide number and becomes a number per section.

SitemapWhat’s in itWhat the Pages report filtered to it tells you
Commercial templatesCategory, product, service pagesWhether the pages that earn money are the ones waiting
EditorialBlog, guides, newsHow fast new posts are picked up, separate from the catalogue
Recently changedURLs whose content changed in the last 30 days, generated dailyWhether changes get recrawled, and in the logs, whether this file is the one read most often
Stable pagesAbout, legal, evergreen pagesA baseline; anything excluded here is a template fault
TemporaryRemoved or redirected URLs with the date of the changeWhen Google has processed the change, so the file can be deleted

A recently changed file is my own practice, and Google doesn’t document whether a file whose contents keep changing gets reread more often than a static one. Step 1 of the log method is how to find out on your own site, which beats either assumption.

What actually moves a URL up the crawl queue

The levers sit on the two sides of the budget model. On demand: link your important pages, new and changed ones first, from pages with a high crawl frequency, the homepage, the category page, the latest posts, so the URL arrives with a reason to fetch it, which is a site structure decision more than a sitemap one, and keep lastmod honest so a change is believed. On capacity: fast, steady responses, 304 on unchanged pages, and no 5xx or 429 during traffic peaks. Since Google retired the crawl rate limiter in Search Console in January 2024, the server’s responses are the only throttle there is, and Mueller’s advice for crawling that is too heavy is a 503 or 429 for a while, not hourly edits to robots.txt.

And on waste: every crawl spent on faceted URLs, parameters, duplicate content, redirect chains and soft 404s is wasted crawl budget, spent before the URLs in your sitemap get their turn. Those are covered one by one in our guide to crawlability problems, and robots.txt is the right tool for some of them, with the catch that a URL blocked there can’t be read for its noindex or canonical either.

Does your site have a crawl budget problem at all

Google’s documentation scopes crawl budget to two kinds of site: large websites, a million or more unique pages with content that changes moderately often, and medium or larger ones, 10,000 or more pages with very rapidly changing content, plus any site with a large share of URLs in “Discovered – currently not indexed.” Below that, Google crawls efficiently enough that sitemap order is moot twice over. A 300-page site with new pages that take weeks to be crawled has a discovery or quality problem, and the fix is links to those pages and a reason to index them.

Sitemap crawl order FAQ

Does the order of sitemaps in robots.txt matter?

No. Each Sitemap: line is a separate discovery source. What does matter is timing: Google can cache robots.txt for up to 24 hours, so a sitemap added there is found on the next fetch of robots.txt, and submitting it in Search Console the same day costs nothing.

Does resubmitting a sitemap in Search Console make Google crawl it now?

It tells Google the file exists, which helps the first time and rarely after. Submission, through the report or the Search Console API, is a hint, and the ping endpoint that used to announce changes is gone. If Google already reads the file, an accurate lastmod inside it does more than a resubmission. For a single URL, Request indexing in URL Inspection is the direct route, within a daily quota.

Does Google support IndexNow?

No. Bing and Yandex accept IndexNow pings; Google doesn’t, and its Indexing API is limited to job posting pages and livestream videos. For everything else, Google’s routes are the sitemap and links.

What I don’t know: whether Google reads the lastmod on entries in a sitemap index to decide which child sitemap to reread. The sitemaps protocol defines the field, Google’s documentation doesn’t say what it does with it, and I haven’t seen a log comparison large enough to tell. If you want the crawl, render and index path read end to end for your own site, that is our SEO audit.