Do You Need an llms.txt File? What It Does and What the Evidence Shows

If someone on your team has asked whether your site has an llms.txt file, the short answer is yes, add one. It's free and takes about 15 minutes. Just don't expect it to change how ChatGPT, Gemini, or Google's AI Overviews talk about your company.

This post covers what the file is, how it caught on, and what the evidence shows. It also covers the work that does affect whether AI tools cite you.

What is llms.txt?

llms.txt is a plain text file that sits at the root of your site, like yoursite.com/llms.txt. It gives AI tools a short, curated summary of your site, with links to your most useful pages.

It was proposed in September 2024 by Jeremy Howard. Howard is a well-known AI researcher. He co-founded fast.ai and Answer.AI, and he was previously president of Kaggle. His idea was simple. Web pages are full of navigation, scripts, and ads that make them hard for language models to read. A clean Markdown file could point models to the content that matters.

The file is written in Markdown. The only required part is a heading with your site or company name. The proposal also suggests:

• A one-line summary of what you do
• A few short sections with links to key pages
• An "Optional" section for pages an AI can skip

What llms.txt isn't

The most common mistake is treating llms.txt like robots.txt. They do different jobs.

robots.txt is a long-standing standard that tells crawlers what they may access. Crawlers from Google, OpenAI, Anthropic, and others are built to follow it. llms.txt has no rules or permissions in it. Writing Allow or Disallow lines in it does nothing.

The second mistake is assuming it's a shortcut to AI visibility. There's no evidence that having the file makes AI tools more likely to mention or cite you.

How llms.txt caught on

Howard's proposal was aimed mainly at developer documentation. Coding tools benefit from a clean index of a project's docs. That use case is real, and companies like Anthropic and OpenAI publish llms.txt files for their own developer docs.

Then the idea spread to marketing sites. SEO blogs and agencies started recommending it as a way to "optimize for AI." AI chatbots picked up the advice too, and started telling people to add one. That created a feedback loop. The more people wrote about llms.txt, the more AI tools repeated the advice, and the more people added the file.

Popularity made it feel like a standard. It never became one.

What the evidence shows

No major AI search platform has said it uses llms.txt. Google has been the most direct. At a Google Search Central event in July 2025, Google's Gary Illyes told attendees that Google doesn't support llms.txt and isn't planning to. In May 2026, Google published a guide to optimizing for its generative AI features. It lists "unnecessary AI text files (like llms.txt)" among the tactics you can ignore. A June 2026 update added that it's "completely fine" to create the file for other services, but it won't help or hurt you in Google Search.

OpenAI and Anthropic publish llms.txt files for their own docs. Neither has confirmed that its crawlers or chat products use the llms.txt files on other people's sites.

Large studies haven't found an effect. SE Ranking studied about 300,000 domains and found no clear link between having an llms.txt file and being cited by AI tools.

Crawler visits don't prove anything. I've set up llms.txt files on sites I've worked on. AI crawlers usually visit them, and you can see this in your server logs. But I haven't seen any meaningful change in AI citations or traffic afterward.

The cats.txt experiment shows why crawler visits aren't proof. In 2026, SEO Mark Williams-Cook invented a fake standard called cats.txt, a file for declaring your office cats. He published the spec on his blog and promoted it on LinkedIn, and other SEOs added the file to their sites. GPTBot, ClaudeBot, and PerplexityBot all fetched it. Google indexed it. Google's AI Overview described a cat that only existed in one of those files. About two weeks later, ChatGPT was saying cats.txt could help with rankings.

A joke file produced the same "evidence" people use to argue that llms.txt works. It also showed how quickly AI tools start recommending something just because people are talking about it. Google's own guide makes the same point. Google may crawl and index many kinds of files, but "this doesn't mean that the file is treated in a special way."

So why add one?

It's low effort and low risk. If you have public docs or an API, it's genuinely useful, because AI coding tools can read it easily. That was the original use case in Howard's proposal. Google's John Mueller explained on Bluesky that this is why Google publishes llms.txt files for its developer docs. For other kinds of sites, he said he doesn't think it makes much sense. Google's Lighthouse tool has also added an experimental llms.txt check for AI agents, so the file may matter more for agents in the future.

If you're on WordPress, SEO plugins like Yoast, Rank Math, and AIOSEO can generate the file for you. Yoast also offers it for Shopify. Just keep expectations in check. Any well-structured page on your site can do the same job. An AI that reads your llms.txt summary would get the same information from a clear homepage or About page.

What actually helps AI tools cite your site

AI search tools find and understand your company the same way search engines do. They crawl your pages and compare what you say with what the rest of the web says about you. These are the things worth your time.

Make sure your site can be crawled in the first place. If AI crawlers are blocked, nothing else matters. Google's own AI search guidance says to make sure crawling is allowed in robots.txt and by your CDN or hosting setup. Blocks often come from a firewall or bot-protection setting that someone turned on without realizing it. My free crawlability checker shows whether Google and the major AI crawlers can reach your pages.

Keep a working XML sitemap. Your sitemap tells crawlers which pages exist and when they changed. Make sure it only lists live pages and that it's linked in your robots.txt.

Build a strong About page. AI tools need a clear, reliable source for basic facts about your company, and your About page is the natural one. Say clearly what you do, who you serve, where you're based, and who runs the company. Use plain language that's easy to quote.

Describe your company the same way everywhere. Use the same company description, product names, and core facts across your site. Do the same on your LinkedIn page, review sites, and directory listings. When sources agree, AI tools are more confident repeating them.

If you want to know how AI tools describe your company right now, and which competitors they recommend instead, an AI visibility audit is a good place to start.