Guide
What is llms.txt?
By Jaeden Doody, StillAwake Media. Examples measured .
llms.txt is a plain Markdown file at the root of a website (yoursite.com/llms.txt) that summarises what the site is and links to the pages a language model should read first. It was proposed in September 2024 and is not a standard. Google says Search doesn't use it. It can help tools that fetch it on purpose, but no AI product has publicly committed to reading it for everyday answers.
What the file looks like
The proposal on llmstxt.org defines a short structure. Only the first line is required; the rest is optional and goes in this order:
- An H1 with the site or project name. The only required part.
- A blockquote with a one-paragraph summary.
- Any plain paragraphs or lists with more detail (no headings).
- H2 sections, each a list of links in the form
- [Name](url): note. - An
## Optionalsection for links a model can skip when it is short on space.
A minimal file for a small business looks like this:
# Example Plumbing
> Example Plumbing is a licensed plumbing company serving Laval and
> Montréal, Québec. Emergency repairs, water heaters and renovations.
> Service calls from $129 CAD.
Open Monday to Saturday, 7:00–19:00. Bilingual (English and French).
## Services
- [Emergency repairs](https://example.com/emergency): 24/7 call-outs and response times
- [Water heaters](https://example.com/water-heaters): installation and replacement, with prices
## About
- [Licences and insurance](https://example.com/about): RBQ licence number and coverage
- [Contact](https://example.com/contact): phone, email and service area map
## Optional
- [Blog](https://example.com/blog): maintenance tips and seasonal adviceFictional business. Notice what it states: location, service area, hours, languages and a starting price. Those are the facts an assistant needs and most sites bury.
Eight real files, measured
We fetched each file on 29 October 2026 and counted its size, links and H2 sections. They range from a third of a page to almost 100 KB, which says a lot about how loosely the proposal is followed.
| Site | Size | Links | H2 sections | ## Optional | What stands out |
|---|---|---|---|---|---|
| llmstxt.org | 0.6 KB | 3 | 1 | No | The proposal's own file. A title, a one-line summary, and three links to Markdown docs. |
| Svelte | 1.7 KB | 7 | 3 | No | Mostly links to full, medium and small versions of the docs, so a model can pick a size that fits its context. |
| Vercel | 5.4 KB | 28 | 9 | Yes | Opens with a 'When to use Vercel' section, then points to deeper indexes. Uses the Optional section as the spec suggests. |
| Shopify | 12.5 KB | 71 | 12 | No | A marketing-site file: what Shopify is, then sections by task (start a business, sell online), each link with a one-line description. |
| Next.js | 14.4 KB | 51 | 5 | No | Says what the site is the source of truth for, which tells a model when to prefer it over other sources. |
| Cloudflare developer docs | 16.8 KB | 112 | 9 | No | A file of files: each product links to its own llms.txt rather than listing every page here. |
| StillAwake Media (ours) | 77.0 KB | 245 | 10 | No | Too long for what the spec intends: it listed every article and game wiki page with no Optional section. Measuring it for this page is how we noticed; those lists now sit under ## Optional. |
| Stripe docs | 97.3 KB | 467 | 27 | Yes | The largest here, and the only one that starts with direct instructions to coding agents (check the latest package version) instead of a summary blockquote. |
Links counts list items starting with a Markdown link. Only two of the eight use the Optional section, and Stripe skips the summary blockquote. Large companies treat the format as a loose convention, not a specification.
What llms.txt does and doesn't do
What Google says
In June 2026 Google added a note to its Search documentation: llms.txt files aren't needed for Google Search and won't positively or negatively affect visibility or rankings, though it's fine to keep one for other systems. Its guide to AI features also says you don't need new machine-readable files, AI text files or special markup to appear in AI Overviews or AI Mode.
What other tools do with it
Chrome's Lighthouse has an llms.txt audit in its agentic-browsing category. It flags a server error when fetching the file and marks a missing file as not applicable, because providing one is optional. Developer tools and documentation assistants sometimes fetch a site's llms.txt deliberately, which is why documentation sites (Stripe, Cloudflare, Next.js, Svelte) were early adopters.
What it can't do
- It can't make an AI assistant mention or recommend you. Nothing can guarantee that.
- It can't block or allow AI crawlers. That's robots.txt.
- It can't fix a site that never states its own facts. If your pages don't say where you work or what you charge, a summary file is the only place that does, and that's a weak signal.
So why publish one? It costs little, it's useful to the tools that do read it, and writing it forces you to decide what your one-paragraph description should be. Then make sure your homepage and service pages say the same thing.
llms.txt next to the files you already have
| File | What it tells machines | Used by search engines? |
|---|---|---|
| robots.txt | Which crawlers may fetch which URLs. | Yes: a long-standing standard that major crawlers follow. |
| sitemap.xml | Which URLs exist and when they changed. | Yes: search engines use it to discover pages. |
| Structured data (JSON-LD) | Facts about the page: business, article, product. | Partly: Google uses it for some rich results, without guarantees. |
| llms.txt | A short Markdown summary of the site and its most useful pages, written for language models. | No: a 2024 proposal. Google says Search doesn't need it. |
How to add one
- Write the summary first. One paragraph: what you do, for whom, where, and a starting price if you publish one. Check that your homepage says the same.
- Pick 5 to 20 pages. Services, pricing, about, contact and your best guides. Give each a one-line note on what it answers.
- Put everything else under ## Optional, or leave it to your sitemap.
- Serve it at /llms.txt as plain text with a 200 status. How depends on your platform; see the guides below.
- Keep it in sync. Generate it from the same data as your pages if you can, or set a reminder to review it when prices or services change.
Our free llms.txt generator drafts the file from your site and reports which of those facts it couldn't find.
Common mistakes
- Listing every URL
- That's what the sitemap is for. The file is meant to be short: the pages a model should read first, with a line on why. Put the long tail under ## Optional.
- Writing marketing copy
- "The leading solution for…" gives a model nothing to repeat. Say plainly what you do, for whom, where, and what it costs.
- Facts that contradict the site
- If the file says one price and the pricing page another, you've created the inconsistency you were trying to remove. Generate it from the same data as your pages if you can.
- Serving it as HTML
- It should return plain text or Markdown at /llms.txt with a 200 status. A 404 page or a redirect to your homepage doesn't count.
- Expecting it to fix visibility
- If your site never states your service area, prices or who runs it, the file can't make an assistant describe you accurately. Fix the pages first.
Questions
What is llms.txt in one sentence?
A Markdown file at yoursite.com/llms.txt that gives language models a short summary of your site and links to the pages most worth reading.
Who created llms.txt?
Jeremy Howard proposed it on llmstxt.org on 3 September 2024. The site describes it as a proposal open for community input, not a standard any company is obliged to follow.
Does Google use llms.txt?
No. In June 2026 Google added a note to its documentation saying these files aren't needed for Google Search and won't positively or negatively affect visibility or rankings. Google also says no special files or markup are needed to appear in AI Overviews or AI Mode.
Does ChatGPT, Claude or Perplexity read it?
There's no public, verifiable commitment from those products to read llms.txt when answering ordinary questions, so we don't claim one. It is used by some developer tools and documentation workflows that fetch it deliberately. Treat it as cheap, harmless and possibly useful, not as a ranking lever.
Is llms.txt the same as robots.txt?
No. robots.txt controls which crawlers may fetch what. llms.txt doesn't allow or block anything; it only describes. Blocking AI crawlers is done in robots.txt.
What is llms-full.txt?
A convention some sites use for a second file containing full page text rather than links. It isn't part of the llmstxt.org proposal, which instead suggests offering a Markdown version of each page at the same URL with .md appended.
What actually moves AI visibility
Clear pages that state your facts, consistent information across the web, and independent sources that mention you. Read what generative engine optimization is, whether ChatGPT can recommend your business, or see how we approach answer engine optimization. For page-level facts, try the schema markup generator.
Sources: llmstxt.org (proposal, v2); Google Search Central documentation updates, June 2026; Google Search Central, “AI features and your website”; Chrome for Developers, Lighthouse llms.txt audit. File measurements by StillAwake Media, 2026-10-29.