Google’s John Mueller says content needs to be in HTML in order for it to be indexed quickly.
This is especially true for sites that frequently produce new and/or updated content.
Mueller chimed in with his advice during a discussion on Twitter about Google’s two-pass indexing system.
When Google crawls and indexes content it does two passes. The first pass looks at HTML only. Then, sometime later, it’ll do a second pass looking at the whole site.
Mueller says there is “no fixed timeframe” between the first and second pass.
In some cases it can happen quickly, in other cases it can take a matter of days or weeks.
Yeah, there's no fixed timeframe — the rendering can happen fairly quickly in some cases, but usually it's on the order of days to a few weeks even. If your site produces new / updated content frequently & you want it indexed quickly, you need that content in the HTML.
— 🍌 John 🍌 (@JohnMu) September 13, 2018
As Mueller says, that could take weeks.
So a piece of content might not get fully indexed in Google Search until weeks after it has been published.
That’s obviously not ideal, which is why it’s critical for Googlebot to see the main content on the first pass.
— Alan Bleiweiss (@AlanBleiweiss) September 14, 2018
Why Doesn’t Googlebot Crawl A Whole Page At Once?
The reason why Googlebot doesn’t crawl and index a whole web page on the first pass comes down to resources.
So Googlebot might index a page before rendering is complete, then it will take some time to complete the rendering.
When the final render does arrive Google will perform a second wave of indexing on the client-side rendered content.
Subscribe to SEJ
Get our weekly newsletter from SEJ's Founder Loren Baker about the latest news in the industry!