What is llms.txt, and does your website need one?

Somewhere in the last year, an agency probably told you your site needs an llms.txt file. Maybe they said it like a tax filing, due soon or else. It is a small file and the idea behind it is reasonable, but the claims around it have run ahead of the evidence. This guide explains what it is, what the people who proposed it say, what Google says, and how to decide.

For the wider picture of getting named in AI answers, read what GEO is.

What is llms.txt?

It is a proposal to put a plain text file, written in markdown, at the root of a website, so the file loads at /llms.txt on your domain. The llmstxt.org page describes it as a proposal to standardize on a file that provides information to help agents use a website. Agents here means AI tools that read pages on someone's behalf. The file gives them a curated overview and links to the pages worth reading, instead of making them guess from your menu and your sitemap.

Who proposed it, and how official is it?

The page credits Jeremy Howard and lists a publication date of September 3, 2024. It was modified on August 10, 2026. It describes itself as an informal overview kept in a GitHub repository and open for community input. So it is a community proposal, not a standard issued by a standards body, and no search company has agreed to follow it as a rule. The page says thousands of sites publish an llms.txt file and names OpenAI, Anthropic and Gemini as publishing one for their developer documentation. That tells you it is in use. It does not tell you it changes what AI tools say about a business.

What goes in the file?

The format is simple. In order, the proposal describes an H1 with the name of the site or project, which is the only required part. Then an optional blockquote summary, optional body text, and optional H2 sections that hold lists of links. Each link item is a markdown link, optionally followed by a colon and a note. By convention an H2 called "Optional" holds secondary links an agent can skip.

To show the shape, picture the file for a roofing company. It opens with the company name as the title line. Under it sits a one-line summary of who it serves. Then comes a section of main pages, each one a link with a short note, such as one for roof repair, one for replacement and one for the service area. A last section marked Optional holds the blog and other longer reading an agent can skip.

The proposal also suggests offering clean markdown versions of pages at the same address with .md added. That part is more work, and it matters mostly to sites with heavy documentation.

What does Google say about AI text files?

Google's page on AI features is plain. It says you do not need to create new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode, and that there is no special schema.org structured data to add. It lists the usual basics instead: allow crawling, link pages internally, give a good page experience and keep key content as text. The page does not mention llms.txt by name. Reading it fairly, Google is telling you the file is not part of how its AI features pick pages.

What matters more than the file?

Robots.txt. OpenAI documents OAI-SearchBot as its crawler for search and says sites opted out of it will not be shown in ChatGPT search answers, though they can still appear as navigational links. GPTBot is separate and relates to training. Each setting is independent. A site that blocks the search crawler by accident is invisible to that product no matter how neat its llms.txt is. Fix that first, and allow around 24 hours for a change to take effect, as OpenAI notes.

After that, the same plain work applies as anywhere: pages that can be indexed, answers stated early, honest facts about your business. We cover those in what GEO is.

Who might get value from llms.txt?

The file makes the most sense for sites with a lot of reference material: software documentation, API guides, large knowledge bases. An agent working for a developer can use a tidy index to find the right page fast. A local service business with twelve pages has less to index, and its customers do not usually send agents to read the site. For that business, the file is a harmless extra, not a priority.

How do you add one if you decide to?

Write the file by hand, keep it short and make sure every link works. Upload it to the root so it loads at the root of your domain, at /llms.txt, and check it in a browser. Link only to pages you are happy for an agent to read. Update it when you add or retire an important page, the way you would update a menu.

Do not pay for a large package to generate one. The file is an hour of work for most sites. If an agency quotes it as a major deliverable, our guide on choosing a GEO agency lists the questions that sort out real work from padding.

Can you tell whether it did anything?

Not easily. Nobody outside an AI company can see exactly how its tools use a page. You can look at your server logs for requests to the file, which shows that something fetched it, and you can run a fixed set of questions in AI tools before and after to see whether answers move. Both are weak evidence, and a change in answers could have many causes. Be wary of anyone who claims a clear before-and-after result from this file alone.

Where to go from here

Read your robots.txt, decide whether you want the search crawler allowed, and only then think about llms.txt. For the bigger picture, what GEO is covers the work that does pay off. Passieon is an independent agency and is not affiliated with Google or OpenAI.

Sources

• llmstxt.org: The /llms.txt file, read November 13, 2026.
• Google Search Central: AI features and your website, read November 13, 2026.
• OpenAI: Overview of OpenAI crawlers, read November 13, 2026.