llms.txt for agent-readable docs

/llms.txt is a proposed convention (llmstxt.org) for a Markdown file at a well-known path that gives LLMs and coding agents a short, curated map of a site. It is not a sitemap dump and not a ranking signal. It is a token-efficient index: project summary, the links that matter, and optional secondary material agents can skip when context is tight.

DaloyJS already ships one for the project site at https://daloyjs.dev/llms.txt. Every docs page also has a Markdown sibling (append .md to the HTML URL). The index is generated from the same docs nav the human UI uses, so the map and the site cannot silently disagree about which pages exist.

What it is (and is not)

• Is: a curated Markdown index at /llms.txt, optionally paired with clean .md versions of linked pages.
• Is not: a replacement for robots.txt (access policy for crawlers).
• Is not: a replacement for sitemap.xml (exhaustive indexable URLs).
• Is not: a guaranteed boost in ChatGPT or Google AI citations. Major model providers have not committed to treating it like robots.txt. Chrome Lighthouse includes an experimental agentic-browsing check that fails only on server errors for the path; a missing file is treated as not applicable.

Use it so coding agents, IDE helpers, and research tools that do fetch the file spend tokens on your contract and auth docs instead of navigation chrome. Keep it honest: stale links train everything that reads them to ignore the map.

Spec shape

The informal spec at llmstxt.org is Markdown with a fixed skeleton:

1. H1 with the project or site name (required).
2. Optional blockquote summary that grounds the rest of the file.
3. Optional free-form paragraphs (not headings) with more background.
4. Zero or more H2 sections. Each is a list of - [title](url): optional note entries.
5. A final H2 named Optional for secondary links that a short context budget may drop.

# DaloyJS

> Runtime-portable, contract-first TypeScript web framework with OpenAPI 3.1,
> typed clients, and security-first defaults.

Current release notes and package pins live on the homepage and CHANGELOG.

## Docs

- [Getting started](https://daloyjs.dev/docs/getting-started.md): first route and test
- [Routing](https://daloyjs.dev/docs/routing.md): contract-first app.route shape
- [MCP](https://daloyjs.dev/docs/mcp.md): runtime tool surface for agents

## Project

- [Docs MCP server](https://daloyjs.dev/mcp): search_docs / get_doc / list_docs
- [npm @daloyjs/core](https://www.npmjs.com/package/@daloyjs/core): published package

## Optional

- [Blog](https://daloyjs.dev/blog): design write-ups and release posts

Prefer linking to Markdown when you have it. Put blog posts and historical material under Optional. Curate aggressively: a thousand-link index recreates the context-window problem the file exists to solve.

How daloyjs.dev implements it

The site route website/app/llms.txt/route.ts builds the file at request time (cached for an hour):

• Project links (homepage, docs MCP server, GitHub, npm, JSR, create-daloy).
• Every docs page from getDocsSearchSections(), grouped like the sidebar, with URLs pointing at the .md siblings.
• Blog posts under the final ## Optional section.

Agents that prefer structured tools over page fetches can use https://daloyjs.dev/mcp (MCP docs) with search_docs, get_doc, and list_docs. The llms.txt file points at that endpoint so the two surfaces stay discoverable together.

For your own API or docs site

@daloyjs/core does not force every API process to serve /llms.txt. That file belongs on the host that publishes human documentation (marketing site, docs portal, or static docs deploy). For a DaloyJS API you still give agents a short list of canonical machine surfaces:

• OpenAPI from your app (see OpenAPI generation).
• Validated route examples via AI-friendly route metadata and daloy inspect --ai.
• Optional dedicated MCP server for live tools.
• In-repo AGENTS.md for agents that edit the codebase (the scaffolder ships one; see the scaffolder docs).

If you publish a docs site, add /llms.txt there, keep .md siblings if you can, and regenerate the index from the same source of truth as your human nav so the map cannot rot alone.

Checklist

• File is reachable at https://<host>/llms.txt with text/plain (or Markdown) and a 2xx status.
• H1 + short summary describe the product in plain language.
• Links are curated, described, and mostly point at Markdown or other low-chrome content.
• ## Optional is last when you include secondary material.
• robots.txt still allows the bots you intend to read docs; blocking them makes the map unread.
• Review the file when you add or remove docs routes (same cadence as nav updates).