What is llms.txt and does your website need one?

llms.txt is a proposed convention — a plain-markdown file served at /llms.txt — that gives AI systems a curated, token-efficient map of a website's most important pages, proposed by Jeremy Howard of Answer.AI on September 3, 2024, and specified at llmstxt.org; whether your website needs one is more honest as "it's a cheap hedge": no major AI engine has officially committed to reading it, so it is insurance for an emerging behavior, not a compliance requirement.

The qualifier matters because llms.txt gets sold as if it were robots.txt for AI — a file the engines are bound to honor. It isn't. Google's AI-features documentation explicitly says you "don't need to create new machine readable files, AI text files, or markup" to appear in its AI features (verified live 2026-08-18), and neither OpenAI nor Perplexity documents reading the file in their crawler pages. Where it does get used today is by AI agents and tools that fetch a site on a user's behalf and benefit from a clean map instead of parsing navigation HTML.

The format, from the actual spec

• H1 site name — the only required element. ("# Strategic AI Architects")
• Blockquote summary — a short description carrying the key facts needed to interpret the rest.
• H2 sections of link lists — each list item a markdown link plus an optional one-line note; an "Optional" section marks skippable depth.
• Companion file — content-rich sites can also serve /llms-full.txt with expanded detail.

The spec is live and maintained: the version at llmstxt.org was updated August 10, 2026, reflecting two years of adoption experience since the original proposal (verified live 2026-08-18).

Our first-hand position

We serve llms.txt on every site we run — this one included: /llms.txt is generated on every build from the same registries that produce our page indexes, so it cannot drift stale, and it includes a "Tools (agent-callable)" section that points agents at the live WebMCP endpoint they can actually call. The reasoning is an asymmetric bet: the cost is one generated file, the downside is zero — Google confirms its absence doesn't hurt you and its presence doesn't rank you — and the upside is being legible to every agent that does read it, in a period when agent behavior is changing quarterly. That is also exactly how we'd describe it to a client: hedge, not moat. The moat is content worth citing; llms.txt is the map to it.