Tornado SEO

llms.txt: what it is and whether you need one

By Thibault · Updated · 5 min read

llms.txt is a Markdown file at the root of a site that gives AI models a short summary and a curated list of links to your most useful pages. Google says you don't need one to appear in AI Overviews or AI Mode, so it will not help your rankings, but it takes 20 minutes to write and can help AI agents that read documentation.

What is llms.txt?

llms.txt is a plain Markdown file placed at yoursite.com/llms.txt that tells AI models what your site is about and where its key content lives. Jeremy Howard proposed it on September 3, 2024. The idea is simple: a language model has a limited context window, and a full HTML page full of menus, scripts and ads wastes it. A short, clean file with a summary and links gives the model the essentials in one request.

The proposal also suggests offering a clean Markdown version of important pages at the same URL with .md added, for example page.html.md, or index.html.md for a URL without a filename. A version 2 of the proposal was published on llmstxt.org in August 2026.

What goes in an llms.txt file?

An llms.txt file follows a fixed order, and only the first element is required. Here is the structure from the official proposal:

  1. An H1 with the name of the site or project. This is the only required section.
  2. A blockquote with a short summary of the key information needed to understand the rest of the file.
  3. Optional paragraphs or lists with more detail, without headings.
  4. Optional H2 sections, each containing a Markdown list of links in the form [name](url), optionally followed by a colon and a note.
  5. By convention, an H2 section named "Optional" for secondary links that an AI model can skip when it needs less context.

For a media site like this one, that means an H1 with the brand, one sentence on what the site covers, then an H2 "Guides" listing your best evergreen pages, an H2 "Glossary" with your definitions, and an "Optional" section with archives and legal pages.

Does Google use llms.txt?

No, Google Search does not use llms.txt, and it says so in its own documentation. The Google Search Central page on AI features states: "You don't need to create new machine readable files, AI text files, or markup to appear in these features." The same page adds that there are no additional requirements or special optimizations to appear in AI Overviews or AI Mode.

Google's people have said the same in public. Gary Illyes said at a Search Central Live event in July 2025 that Google does not support llms.txt and is not planning to. John Mueller compared the file to the old keywords meta tag, a signal site owners declare about themselves that systems have no reason to trust.

There is one mixed signal. Chrome's Lighthouse 13.3 added an experimental Agentic Browsing category that includes an llms.txt audit. That audit checks whether the file exists and loads without a server error. It sits outside the SEO audits and says nothing about rankings or citations.

Do ChatGPT and other AI models read llms.txt?

There is no official confirmation from OpenAI, Anthropic or Google that their AI models use llms.txt to choose what to cite. In July 2025, an SEO shared server logs showing OpenAI requesting his llms.txt file about every 15 minutes. A request proves the file was fetched, not that it changes which pages get cited.

The most likely real use today is AI agents and coding tools that read technical documentation. When a developer asks an AI model to work with a library, a clean Markdown index of the docs saves it from parsing dozens of HTML pages.

Thibault's take I treat llms.txt like a sitemap for a reader that may or may not show up. It costs almost nothing, so I add one. But I see sites spend a week on it while their main content only appears after JavaScript runs, and AI crawlers often do not execute JavaScript. Fix that first. Accessibility comes before visibility: if a bot cannot read your HTML, no text file will save you.

Do you need an llms.txt file?

You need one if you publish documentation, an API or a developer product; for everyone else it is optional. Use this table to decide:

  1. Documentation, API, SaaS with technical docs: yes, add llms.txt and Markdown versions of your doc pages. Agents read these.
  2. Media site or blog: optional. A short file listing your best guides is enough.
  3. E-commerce: low priority. Spend the time on product page content and structured data.
  4. Local business: skip it. Your Google Business Profile matters far more.

How to create an llms.txt file in 5 steps

Creating an llms.txt file takes about 20 minutes for most sites. Follow these steps:

  1. List the 10 to 30 pages you would want an AI model to read first: your best guides, your product or service pages, your about page.
  2. Write the H1 with your site name and a one sentence blockquote that says what you do and for whom.
  3. Group the links under H2 sections, one link per line, each with a short note after a colon.
  4. Put archives, legal pages and less important content under an H2 named Optional.
  5. Upload the file to the root of your domain, check that yoursite.com/llms.txt returns a 200 status as plain text, and update it when you publish a major page.

What should you do before llms.txt?

Three checks will do more for your AI visibility than any llms.txt file. First, make sure your robots.txt does not block the AI crawlers you want, such as OAI-SearchBot for ChatGPT search. Second, view your pages with JavaScript disabled: if the main text disappears, AI crawlers probably cannot read it. Third, rank in classic search, because AI Overviews and ChatGPT search lean on pages that search engines already trust.

Thibault's take There are no absolutes in SEO, and llms.txt could become a standard if AI companies start relying on it. Until one of them says so officially, I would not promise a client anything from it. Write it, link your strongest pages in it, and move on to the work that moves numbers.

Frequently asked questions

Is llms.txt a ranking factor?

No. Google states that no AI text files are needed to appear in AI Overviews or AI Mode, and Gary Illyes said Google does not support llms.txt. It has no effect on Google rankings.

What is the difference between llms.txt and robots.txt?

robots.txt tells crawlers what they are allowed to access, and crawlers respect it. llms.txt is a voluntary summary that points AI models to your best content. It grants or blocks nothing.

Where do I put llms.txt?

At the root of your domain, so it loads at yoursite.com/llms.txt. It should return a 200 status and plain text Markdown.

Should I create Markdown versions of my pages?

Only if you run documentation or a developer product, where AI agents read pages directly. For a blog or a shop, clean HTML that loads without JavaScript does the job.

Can llms.txt hurt my site?

No, it has no known negative effect. The only risk is spending time on it instead of fixing crawlability or content problems that matter more.

Sources

  1. llmstxt.org/
  2. developers.google.com/search/docs/appearance/ai-features
  3. searchenginejournal.com/googles-llms-txt-guidance-depends-on-which-product-you-ask/575431/
  4. seroundtable.com/openai-crawling-Ilms-txt-files-39811.html

Stay ahead of every search update

Google, ChatGPT, Gemini, Claude: what changed, why it matters, and what to do about it. Every morning, in 3 minutes, free.

What the...?