llms.txt for agent-readable docs

/llms.txt is a proposed convention (llmstxt.org) for a Markdown file at a well-known path that gives LLMs and coding agents a short, curated map of a site. It is not a sitemap dump and not a ranking signal. It is a token-efficient index: project summary, the links that matter, and optional secondary material agents can skip when context is tight.

This page tracks v2 of the proposal, published on 10 August 2026 and the first revision since the format appeared in 2024. The file skeleton did not change, so a valid v1 file is still a valid v2 file. What changed is everything around the file: how agents discover it, where markdown versions are allowed to live, and what a file in a subpath means. See What changed in v2 below.

DaloyJS already ships one for the project site at https://daloyjs.dev/llms.txt. Every docs page also has a Markdown sibling (append .md to the HTML URL). The index is generated from the same docs nav the human UI uses, so the map and the site cannot silently disagree about which pages exist.

What it is (and is not)

• Is: a curated Markdown index at /llms.txt, optionally paired with clean .md versions of linked pages.
• Is not: a replacement for robots.txt (access policy for crawlers).
• Is not: a replacement for sitemap.xml (exhaustive indexable URLs).
• Is not: a guaranteed boost in ChatGPT or Google AI citations. Major model providers have not committed to treating it like robots.txt. Chrome Lighthouse includes an experimental agentic-browsing check that fails only on server errors for the path; a missing file is treated as not applicable.

Use it so coding agents, IDE helpers, and research tools that do fetch the file spend tokens on your contract and auth docs instead of navigation chrome. Keep it honest: stale links train everything that reads them to ignore the map.

Spec shape

The informal spec at llmstxt.org is Markdown with a fixed skeleton:

1. H1 with the project or site name (required).
2. Optional blockquote summary that grounds the rest of the file.
3. Optional free-form paragraphs (not headings) with more background.
4. Zero or more H2 sections. Each is a list of - [title](url): optional note entries.
5. By convention an H2 named Optional for secondary links an agent can skip when it needs a shorter context. In v1 this heading was machine-readable: it told the context-expansion tool which links to omit. v2 dropped that tool, so Optional is now a hint to the reader rather than an instruction to a parser. It is still worth using, and still belongs last.

# DaloyJS

> Runtime-portable, contract-first TypeScript web framework with OpenAPI 3.1,
> typed clients, and security-first defaults.

Current release notes and package pins live on the homepage and CHANGELOG.

## Docs

- [Getting started](https://daloyjs.dev/docs/getting-started.md): first route and test
- [Routing](https://daloyjs.dev/docs/routing.md): contract-first app.route shape
- [MCP](https://daloyjs.dev/docs/mcp.md): runtime tool surface for agents

## Project

- [Docs MCP server](https://daloyjs.dev/mcp): search_docs / get_doc / list_docs
- [npm @daloyjs/core](https://www.npmjs.com/package/@daloyjs/core): published package

## Optional

- [Blog](https://daloyjs.dev/blog): design write-ups and release posts

Prefer linking to Markdown when you have it. Put blog posts and historical material under Optional. Curate aggressively: a thousand-link index recreates the context-window problem the file exists to solve.

What changed in v2

v1 told you what to put in the file but left the surrounding questions open. The spec now poses the one that mattered most: given a page, how does an agent find its markdown version, or the llms.txt file that covers it, without guessing? Five changes answer it.

1. Link relations for discovery. The headline change. rel="alternate" type="text/markdown" points from a page to its markdown version, and rel="describedby" points to the llms.txt file covering it. Both can be HTML <link> elements or an HTTP Link: response header.
2. Two markdown URL forms. v1 allowed only .md appended to the full page URL (page.html.md). Many publishing tools replace the extension instead (page.md), so v2 permits both. URLs without a file name append index.html.md or index.md.
3. Subpath coverage is defined. v1 permitted files at subpaths without saying what they meant. v2 states it: a file covers the pages under its path, and where several apply the most specific one wins. So /docs/llms.txt covers everything in /docs/. This is what lets a project that controls only a path, such as a GitHub Pages site, participate properly.
4. The consumption model is stated. v1 said nothing about how the file should be read and shipped an llms_txt2ctx tool that expanded it into one large context. v2 drops the tool and states the expectation instead: agents view or search the llms.txt, then follow the links they need. The file stays small; the detail lives behind the links.
5. Optional lost its mechanical meaning. It existed to tell the expansion tool what to leave out. With the tool gone, the section remains a useful convention and nothing more.

Nothing here invalidates an existing file. If you already publish a v1 llms.txt, adding the two link relations is the whole upgrade.

Link relations

The header form is usually the better one to reach for. It works on non-HTML resources, including the markdown files themselves, and can be added in web-server or CDN configuration without touching a single template:

Link: </docs/routing.md>; rel="alternate"; type="text/markdown",
      </docs/llms.txt>; rel="describedby"

The equivalent in a page's <head>:

<link rel="alternate" type="text/markdown" href="/docs/routing.md">
<link rel="describedby" href="/docs/llms.txt">

Relative URLs are fine in both forms. Point rel="describedby" at the most specific llms.txt that covers the page, not always the one at the site root.

How daloyjs.dev implements it

The site route website/app/llms.txt/route.ts builds the file at request time (cached for an hour):

• Project links (homepage, docs MCP server, GitHub, npm, JSR, create-daloy).
• Every docs page from getDocsSearchSections(), grouped like the sidebar, with URLs pointing at the .md siblings.
• Blog posts under the final ## Optional section.

Two more surfaces implement the v2 half. There is a second, narrower index at /docs/llms.txt covering just the documentation, built from the same getDocsSearchSections() call. And website/proxy.ts stamps the Link: header onto every response, so a docs page advertises its .md sibling and points at /docs/llms.txt, while the rest of the site points at the root /llms.txt. The matching <link rel="alternate"> element comes from buildMetadata() in website/lib/seo.ts, so every docs page gets it without per-page wiring.

Agents that prefer structured tools over page fetches can use https://daloyjs.dev/mcp (MCP docs) with search_docs, get_doc, and list_docs. The llms.txt file points at that endpoint so the two surfaces stay discoverable together.

For your own API or docs site

@daloyjs/core does not force every API process to serve /llms.txt. That file belongs on the host that publishes human documentation (marketing site, docs portal, or static docs deploy). For a DaloyJS API you still give agents a short list of canonical machine surfaces:

• OpenAPI from your app (see OpenAPI generation).
• Validated route examples via AI-friendly route metadata and daloy inspect --ai.
• Optional dedicated MCP server for live tools.
• In-repo AGENTS.md for agents that edit the codebase (the scaffolder ships one; see the scaffolder docs).

If you publish a docs site, add /llms.txt there, keep .md siblings if you can, and regenerate the index from the same source of truth as your human nav so the map cannot rot alone.

Checklist

• File is reachable at https://<host>/llms.txt with text/plain (or Markdown) and a 2xx status.
• H1 + short summary describe the product in plain language.
• Links are curated, described, and mostly point at Markdown or other low-chrome content.
• ## Optional is last when you include secondary material (a convention now, not a parser directive).
• Pages advertise their markdown sibling with rel="alternate" type="text/markdown" and their index with rel="describedby", as a Link: header, <link> elements, or both.
• A file at a subpath covers the pages beneath it. If you publish /docs/llms.txt, point the docs pages' describedby at it rather than at the root file.
• robots.txt still allows the bots you intend to read docs; blocking them makes the map unread.
• Review the file when you add or remove docs routes (same cadence as nav updates).