OPEN LINK ↗
// 1h agoBENCHMARK RESULT
Mintlify: llms.txt cuts agent 404s by 90%
Mintlify's new benchmark across 20 documentation sites shows that publishing a standard llms.txt index reduces AI agent 404 errors by up to 90%. By testing formats like raw HTML, plain markdown, and inlined indexes, the study finds that serving markdown via content negotiation alongside llms.txt is the most token-friendly setup.
// ANALYSIS
The developer docs of the future will be designed for machine-readability first, optimized as clean markdown feeds and llms.txt maps for AI agents.
- –Codex vs. Claude: Codex exhibits a thorough but verbose strategy, reading 2-4x more pages and achieving slightly higher accuracy, whereas Claude is highly efficient, maintaining 94-99% accuracy while reading far fewer pages.
- –The llms.txt Standard: The study shows clear developer behavior: agents reflexively reach for /llms.txt and markdown versions of URLs.
- –Token Efficiency: Inlining the index on every page is a waste of context window; publishing a single llms.txt file coupled with content negotiation provides the same error reduction at a fraction of the cost.
// TAGS
llms.txtdocumentationagentclaude-codecodexmintlifybenchmarkmintlify-docs-url-discovery-bench
DISCOVERED
1h ago
2026-07-17
PUBLISHED
1h ago
2026-07-17
RELEVANCE
8/ 10
AUTHOR
mintlify