Do You Need an llms.txt File? What Google Actually Says

This question comes up on nearly every call now. Someone has read that AI needs a special file to understand your website, and they want to know if their site is missing it.

Short answer: you don’t need one to be found by AI or by Google, and adding one won’t move you up the results. There is one place it gets checked, which is worth knowing about, and there’s a right and wrong way to build one. Here’s the full picture.

What an llms.txt file actually is

It’s a plain text file that sits at the root of your website, at yoursite.com.au/llms.txt. It’s written in Markdown and it works like a summary and index of your site for machines: a heading with your business name, a short description of what you do, then grouped lists of links to your important pages with a line of description on each.

The idea came from a community proposal, not from Google or OpenAI. The thinking was that large language models have limited context and have to work through messy HTML, so a clean, curated summary of a site would help them use it correctly. As an idea it’s sensible. As a requirement, it doesn’t exist.

What Google says about it

Google’s Search documentation is direct on this point: you don’t need a special AI text file, special AI markup, or extra schema to be eligible for Search results, AI Overviews, or AI Mode. The things that make you eligible are the ordinary things. Your pages need to be crawlable, indexable, and actually useful to the person reading them.

That’s worth sitting with for a second, because it contradicts a lot of what’s being sold at the moment. There is no separate AI channel you can plug into. The same content and the same technical foundations feed both.

So if your business isn’t showing up in AI answers, an llms.txt file is not the missing piece. The usual culprits are much less exciting: a thin website that doesn’t answer any real questions, an incomplete Google Business Profile, inconsistent business details across directories, and almost no mentions anywhere else on the web. We wrote about that pattern in why your business isn’t showing up in ChatGPT.

The one place it does get checked

Here’s the part that gets left out of most articles on this topic.

Chrome has been trialling a new category in Lighthouse, the tool behind Google PageSpeed Insights, called Agentic Browsing. It looks at how well a page supports automated agents, and one of its checks is whether you have an llms.txt file at your root. It’s experimental and it sits outside the Search ranking systems entirely, but it’s a real check that produces a real pass or fail in a report your web person might send you.

The checks it actually implements are modest. The file needs to be more than about 50 characters, it needs a top-level heading, and it needs to contain Markdown links. That’s the bar. A file can clear all three and still be useless to a human reader, which tells you how much weight to give the result.

Notice what this means: the file has nothing to do with your ranking, and everything to do with a diagnostic score. Those are different things, and conflating them is where most of the confusion in this area comes from.

Do the AI platforms themselves use it?

Not as far as anyone can demonstrate. No AI company has publicly confirmed that its systems read llms.txt to decide what to retrieve or who to recommend.

What they clearly do use is the live web. Perplexity and ChatGPT’s search modes fetch real pages, follow real links, and read what’s on them. Your content, your profile, your reviews, and what other sites say about you are doing the work. A summary file at your root is not what gets you into the answer.

So should you have one?

We build them for clients, and we’re upfront about why. It takes very little effort, it produces a clean index of the site that’s genuinely useful to read, and it clears the Agentic Browsing check if anyone runs one. That’s the whole case for it. It is not part of why those sites get cited in AI answers.

If someone has quoted you a meaningful amount of money for an llms.txt file as an AI visibility service, you’re being overcharged for a text file.

How to do it properly

If you’re going to have one, these are the things that actually go wrong. All of them are from real builds.

Use one generator, not two. Several SEO plugins now offer an llms.txt feature. Turn on two and they fight, and you end up with duplicated headings or empty sections in the published file. Pick one owner for the file and switch the other off.

Make it match your sitemap. The links in the file should be the same set of pages you actually want indexed. If your sitemap has 30 URLs and your llms.txt has 47, including your privacy policy and a thank-you page, something is generating it from the wrong list.

Regenerate it after any domain or host change. This is the one that bites. Move a site to its real domain and the file will happily keep listing the old development URLs until you rebuild it. Every link in it is then broken. Check it the same day you launch, not a month later.

Write real descriptions. One line per link, describing what’s on the page in plain words. If the descriptions are auto-filled with the page title repeated twice, you’ve produced something no human or machine benefits from.

Keep it honest. It’s a summary of your site, not a sales page. Overstating what you do here creates the same problem it creates everywhere else, which is a mismatch between what you claim and what your pages deliver.

Then go and check it. Open yoursite.com.au/llms.txt in a browser. If it returns a 404, is blocked, or shows the wrong host, none of the above matters.

What it will not fix

An llms.txt file will not get you into AI Overviews. It will not make ChatGPT recommend you. It will not improve your position for a single search term.

The things that do move those numbers are slower and less interesting to talk about. Pages that answer the specific questions your customers ask. A complete and accurate Google Business Profile. Consistent business details everywhere your business appears, which we cover in NAP consistency. Reviews. Mentions on other sites that Google and the AI platforms already trust. That’s the work, and there’s no file you can upload instead of doing it.

If you want to know where your business actually stands across Google, Maps, and AI search before spending money on any of this, get in touch and we’ll take a look. We’d rather tell you your llms.txt file is fine and your real problem is elsewhere than sell you a file you don’t need.