Google’s John Mueller responded to a question about whether Google’s guidance on LLMs.txt is contradictory. His answer shows why Google considers LLMs.txt speculative for now and why WebMCP, a Google-backed proposal, might be what SEOs and site owners should be looking into.
LLMs.txt, Chrome Lighthouse, And Google’s Guidance
A discussion on Reddit asked about the apparently conflicting guidance from Google. The person asking the question noted that Google Search Central says publishers do not need special AI files such as LLMs.txt in order to appear in AI search experiences. But Chrome’s Lighthouse Audit documentation includes a check for LLMs.txt, which creates the perception of a conflict between the two guidelines.
The Redditor asked:
“Google Search Central says you don’t need special AI files like LLMs.txt to appear in generative AI search, but Chrome Developers says LLMs.txt can help agents understand a site’s structure and primary content.
I’m anti-LLMs.txt, but is this conflicting guidance from Google? Or am I mixing up “Google Search visibility” with “AI agent usability”?”
Classic Technical Writing That Fails To Communicate
The person asking the question apparently didn’t understand what the Chrome Lighthouse Audit documentation said because the documentation is poorly worded.
Contrary to what the Redditor believes, the page does not say that “LLMs.txt can help agents understand a site’s structure and primary content.”
What it does say is that LLMS.txt is an “emerging convention” and that without that file AI agents “may spend more time crawling the site to understand its high-level structure and primary content.” That word “may” is doing a lot of heavy lifting because it’s a hedge word that communicates a possibility, not a certainty.
- The Redditor believes that the Chrome guidance asserts a benefit.
- But the actual wording only suggests a possibility.
The Chrome Lighthouse guidance is jargony because it uses the phrase “emerging convention” instead of correctly saying that LLMS.txt is just a proposal and not a standard.
The use of the word “may” is a poor choice because it requires the reader to infer that the benefit of an LLMs.txt is a hypothetical and not something that’s happening right now.
Mueller’s Ironic Answer
Mueller’s response contains irony. He points out that if an AI platform needs a file then give it to them. No AI sites actually require an LLMs.txt or uses them, so that’s purely a hypothetical. The ironic part is when he points out that website owners and SEOs are using LLMs to parse the HTML content in order generate an LLMs.txt so that another LLM wouldn’t have to do it.
Mueller’s answer:
“When an AI platform that brings you clients complains that it needs the file for your site, then I’d recommend taking the time to create one. (Aside, if you use an LLM to create the file for you, doesn’t that mean the LLM could just … create it for itself too?)”
Mueller is pointing out the inherent redundancy of LLMs.txt.
LLMs.txt Is Purely Speculative For Now
After Mueller answered the initial question, another Redditor continued the discussion by suggesting that the benefit of the proposed LLMs.txt standard is that it would reduce the amount of crawling and bandwidth used while also improving the interactions between a website and AI agents.
The Redditor wrote:
“Don’t you think it would be a generally nice thing to provide friendly and lightweight interfaces between your site and LLMs? I consider it similar to page speed as a good thing for user experience. LLMs are very resource intensive, so why not do what we can to reduce resource consumption on retrieval?”
Mueller responded:
“I don’t think anyone knows – it’s purely speculative for now (the file has existed for years, yet none of the AI systems use it — what does it mean?).
I like the WebMCP approach, as well as the commerce integrations – they have clear goals & processes: “Given the agent is already on your site, how can it *properly* do task X?” (for example, determine the final price of a product, including all fees & potential discounts).”
Mueller injects an interesting point about WebMCP. WebMCP is a proposed web standard that leverages Model Context Protocol to do a lot more than LLMs.txt.
WebMCP enables AI agents to discover and use website functionality, making it easier for them to interact with a website instead of using the HTML that was created for humans. This enables AI agents to do things like compare products, add items to a shopping cart, and fill out a contact form. WebMCP is currently supported in Chrome.
The Bigger Issue May Be Whether Sites Block Agents
Mueller’s answer continued with one more observation which may be the most important part of the discussion. He suggested that the most basic form of agent optimization is simply making sure agents are not blocked from accessing a site. In his view, that hurdle may be more important for most publishers than questions about LLMs.txt.
He continued his answer:
“I don’t think there’s the agentic equivalent of “let me look at 10 sites and see where I can buy X the fastest” (users aren’t going to be happy if the agent buys a “FerraLamboWagen” just because it was easist to buy).
So speculatively, I’d assume that if an agent is already on your site and tasked to do something, it will be happy to just click around and try to complete the task with the UI too.
Of course, all of this assumes that the most basic agentic optimization is in place, namely: don’t block agents. I think that hurdle will be the biggest, for most sites.”
The reality today is that AI agents can use the HTML-based user interface that’s made for humans. So even if the site isn’t implementing WebMCP, the agent can still accomplish tasks. The takeaway then is that it’s increasingly important to make sure that AI agents aren’t being blocked.
