Contents
What is llms.txt?
llms.txt is a proposal published by Jeremy Howard in September 2024 and updated in August 2026. The idea: a short file at yoursite.com/llms.txt that says in one sentence what the site does and links to the pages that matter. An AI tool finds the right page there quickly, instead of crawling the whole site.
A map of your site, written for AI tools, that you put together yourself.
- A plain-text file in Markdown
- A title, a summary, annotated links
- Readable by humans, too
Not an access rule, and not a Google ranking factor.
- It doesn't allow or block any crawler
- It doesn't replace your sitemap
- It isn't an official standard
robots.txt tells crawlers what they're allowed to crawl. Your sitemap lists every page for search engines. llms.txt picks the most useful ones and explains what they're for.
No standards body publishes llms.txt. Each AI tool decides whether it reads the file, and how.
Sources: llmstxt.org (proposal of September 3, 2024, version 2 from August 2026, accessed November 4, 2026).
Google, Lighthouse, OpenAI: who reads llms.txt?
Before you start, it helps to know what it actually changes today. Google has been clear: llms.txt plays no role in Google Search, AI Overviews included. Chrome, on the other hand, checks for it in an experimental audit. And AI companies publish one for their own documentation.
According to Google, neither llms.txt nor any other “special” file is needed to appear in Search or its AI features. Creating one won't help you, and won't hurt you either.
Lighthouse's experimental Agentic Browsing category checks that an llms.txt file exists at the root of the domain. It requires Chrome 150 or later.
OpenAI and Anthropic publish an llms.txt for their developer docs. Their crawler documentation talks about robots.txt, not llms.txt.
In its guide to generative AI features, Google writes: “It's completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files.” Google Search itself ignores them. The note was added on June 15, 2026.
It costs almost nothing, does no harm, and helps the tools that read it. It's also a chance to spell out, in black and white, which pages matter most.
Sources: Google Search Central (generative AI guide and documentation updates, June 2026), Chrome for Developers, OpenAI and Anthropic documentation (accessed November 4, 2026).
The llms.txt format, in order.
The file is written in Markdown, in a set order. Only one element is required: the title. Everything else is recommended, and is just as readable for a human as for a machine.
| Element | How to write it | Required? |
|---|---|---|
| Title | One H1 line (#) with the name of the site or project. | Yes, the only one |
| Summary | A blockquote (>) of one or two sentences: what you do, and for whom. | No, recommended |
| Details | Paragraphs or lists, with no heading: useful context, caveats. | No |
| Link lists | H2 sections (##), each listing [name](url) links followed by a colon and a short note. | No, recommended |
| Optional | An H2 section named Optional for secondary links. | No |
An llms.txt example
yoursite.com/llms.txt# Your site > One sentence on what the site does, and for whom. A few lines of context: what you offer, your languages, your markets. ## Guides - [Guide title](https://yoursite.com/guide): what the page teaches - [Pricing](https://yoursite.com/pricing): plans and prices ## Optional - [About](https://yoursite.com/about)
Ours is live at seocreo.com/llms.txt: a title, a summary, then our product pages, pricing and guides, grouped into sections. For developer docs, see the llms.txt OpenAI publishes at developers.openai.com/llms.txt and the llms.txt Anthropic publishes at platform.claude.com/llms.txt.
Since version 2, it no longer tells tools what to skip: by convention, it's simply where secondary links go.
Sources: llmstxt.org, proposal and version 2 changes page (accessed November 4, 2026).
How to create an llms.txt file, with or without a plugin.
Three ways to do it: write it by hand, let your WordPress plugin generate it, or start from an online llms.txt generator. Either way, choosing the pages that matter is up to you.
Write the file in a text editor, save it as llms.txt, then upload it to the root of your site (FTP, your host's file manager, or your project's public folder).
Yoast SEO, Rank Math and AIOSEO create llms.txt in their free versions and keep it up to date. See the paths below.
It crawls your site and suggests a first draft. Review it: cut the pages that add nothing and write the summary yourself.
llms.txt in WordPress: where to turn it on
| Plugin | Path | What you set |
|---|---|---|
| Yoast SEO | Yoast SEO › Settings › Site features › AI tools › llms.txt | Automatic or manual page selection (Customize llms.txt file) |
| Rank Math | Rank Math SEO › Dashboard › LLMS Txt module, then General Settings › Edit llms.txt | Post types, taxonomies, posts/terms limit, additional content |
| AIOSEO | llms.txt settings in the plugin | Post types, taxonomies, number of pages per type |
Yoast SEO won't overwrite an llms.txt it didn't create, or one you've edited by hand: delete it, or turn the feature off. On a multisite network, Yoast's feature isn't available.
Put the file in the folder your site serves as-is (for example public/ in a Next.js project). Then check that the URL shows the raw text, not a page of your site.
Sources: Yoast SEO and Rank Math help, plugin pages on WordPress.org (accessed November 4, 2026).
.md versions and discovery links.
The proposal goes beyond the file itself: it recommends offering a Markdown version of your useful pages and, since version 2, pointing to those versions in each page's code. It's optional, and mostly useful for documentation sites.
A page's Markdown version lives at the same URL with .md added (page.md or page.html.md). For a URL with no file name: index.md.
An llms.txt file can also live in a subfolder. It covers the pages in that folder, and the most specific one wins.
<!-- In the page's <head>: its Markdown version, and the llms.txt that covers it --> <link rel="alternate" type="text/markdown" href="/guide.md"> <link rel="describedby" href="/llms.txt"> # Or in the HTTP Link header, which also works for non-HTML files
Some plugins also create an llms-full.txt, which puts all the site's content in a single file. It isn't part of the llmstxt.org proposal.
Google says the same about Markdown: Google Search doesn't need Markdown versions of your pages.
Sources: llmstxt.org, version 2 (August 2026), and Google Search Central (accessed November 4, 2026).
The file points the way, your content earns the citation
llms.txt helps an AI tool find your pages. What gets them cited is their content: a direct answer, clear headings, a real FAQ.
Your llms.txt in 7 steps.
Seven steps, from the must-dos to the nice-to-haves. Check them off as you go.
Let crawlers read your pages
llms.txt points to your pages. If robots.txt, your firewall or your CDN blocks them, the file leads nowhere.
Write the title and summary
The site name as an H1 title, the only required element. Then a one-sentence blockquote saying what you do, and for whom.
List your key pages
Under H2 headings, links followed by a short note. A selection, not the whole site: secondary pages go in the Optional section.
Upload it to the root
At yoursite.com/llms.txt, as plain text. Open that URL in your browser: you should see the file, not an error page.
Keep it up to date
Added a key page or changed a URL? Update the file. WordPress plugins do this for you.
Offer .md versions
For your most useful pages, especially documentation: a Markdown version at the same URL, flagged in the <head>.
Write the answer before the details
The step that makes the biggest difference isn't in the file: open every page and every section with a clear answer.
Check that your llms.txt is live, and being read.
No tool, not even an llms.txt validator, tells you which AI tools use your file. What you can do is check that it's live, test it with Lighthouse, and see in your server logs who fetches it.
- Open the URLyoursite.com/llms.txt should show your text, with a 200 status code, without redirecting to another page.
- Run LighthouseIn Chrome 150 or later, from DevTools, with the Agentic Browsing category.
- Read your server logsLook for requests to /llms.txt and note which crawlers make them.
- Don't read too much into itA crawler that downloads the file doesn't tell you what it does with it.
The Agentic Browsing category doesn't give a score out of 100: it shows the share of audits passed. llms.txt sits in the discovery audits, alongside layout stability (CLS).
Since Google Search doesn't use llms.txt, no report tracks it. For your visibility in Google's AI answers, open Performance › Generative AI instead.
Sources: Chrome for Developers, Agentic Browsing category scoring (accessed November 4, 2026).
Common myths.
Six claims you hear a lot about llms.txt, and what's actually true.
No. Google says Google Search ignores these files: they don't help, and they don't hurt.
No. According to Google, no special file or markup is needed to appear in its AI features.
No. It sets no access rules. To allow or block a crawler, use robots.txt.
Nothing confirms that. OpenAI and Anthropic publish one for their docs, but their crawler documentation talks about robots.txt.
No. It's a proposal published on llmstxt.org. Each tool decides whether to follow it.
No. Only the title is required. The whole point of the file is to choose your key pages.
Frequently asked questions.
What is llms.txt?
A Markdown text file, placed at the root of a site, that introduces the site to AI tools in one sentence and lists its most useful pages. It's a proposal published in 2024 on llmstxt.org, not an official standard.
What is an llms.txt file used for?
It helps an AI tool, such as an agent browsing on your behalf, find the right page quickly without crawling the whole site. It plays no part in Google rankings.
Does Google use llms.txt?
No. Google says Google Search, including AI Overviews and AI Mode, ignores llms.txt: the file doesn't help and doesn't hurt. Only Chrome's Lighthouse tool checks for it, in an experimental audit.
How do I create an llms.txt file?
Write an H1 title with the site name, a one-sentence summary as a blockquote, then H2 sections with your key links and a short note for each. Save it all as llms.txt.
Where do I put the llms.txt file?
At the root of the site, so it loads at yoursite.com/llms.txt. An llms.txt file can also cover a subfolder, such as /docs/.
What's the difference between llms.txt and robots.txt?
robots.txt gives crawlers access rules: what they can and can't crawl. llms.txt sets no rules: it introduces your most useful pages.
Which llms.txt generator should I use?
On WordPress, the one in your SEO plugin: Yoast SEO, Rank Math and AIOSEO each include a free llms.txt generator. Otherwise, an online generator gives you a first draft to review and trim before you publish it.
How do I add llms.txt in WordPress?
With Yoast SEO: Settings › Site features › AI tools, then turn on llms.txt. With Rank Math: turn on the LLMS Txt module in the dashboard. The file is then created and kept up to date for you.
Do ChatGPT and Claude read llms.txt?
Neither OpenAI nor Anthropic says their crawlers do: their crawler documentation covers robots.txt. An assistant can still open the file, like any other page, when it browses your site.
Do I also need an llms-full.txt file?
No, it isn't necessary. llms-full.txt, which gathers all the site's content into one file, isn't part of the llmstxt.org proposal. Some plugins offer it as an option.
- The /llms.txt filellmstxt.org · Jeremy Howard · September 3, 2024, updated August 10, 2026
- Changes in v2llmstxt.org · August 2026
- Optimizing your website for generative AI features on Google SearchGoogle Search Central · accessed November 4, 2026
- Latest documentation updatesGoogle Search Central · June 15, 2026
- AI features and your websiteGoogle Search Central · accessed November 4, 2026
- Lighthouse agentic browsing scoringChrome for Developers · accessed November 4, 2026
- Overview of OpenAI CrawlersOpenAI · accessed November 4, 2026
- Does Anthropic crawl data from the web, and how can site owners block the crawler?Anthropic Help · accessed November 4, 2026
- How to enable llms.txt with Yoast SEOYoast Help · accessed November 4, 2026
- How to Use llms.txt in Rank Math SEORank Math Help · accessed November 4, 2026
GuideSEO for ChatGPT
GuideAI Overviews
GuideBing Webmaster Tools