What is llms.txt and do you need one?
llms.txt is a proposed Markdown map of your site for AI tools. What it is, who reads it, what it does not do, and which sites should publish one.
By the Paperlink AI teamPublished 6 min read
Short answer
llms.txt is a proposed Markdown file at /llms.txt that tells AI tools what your site is and which pages explain it best. Support varies between AI tools, and search engines have said they do not use it for ranking. It is worth publishing for docs and software sites because it is cheap, but fix crawler access and page text first.
llms.txt gets a lot of attention for a small text file. Some people call it the robots.txt of AI, and some call it useless. Both are wrong. It is a short, optional map of your site for AI tools. It helps some tools today, it does nothing for search rankings, and it takes less than an hour to publish. This post explains what it is, who reads it and how to decide whether your site needs one.
What llms.txt is
llms.txt is a proposal published at llmstxt.org in September 2024. The idea is simple: AI tools have little room to read a site, and HTML pages are full of menus, scripts and layout. So the site owner publishes one Markdown file at /llms.txt that says, in a few lines, what the site is and which pages explain it best. A tool can read that file first and then open only the pages it needs.
A file for a small online flower shop could look like this:
# Petal Lane
> Petal Lane is a flower shop in Leeds, UK. It delivers fresh bouquets in Leeds the same day for orders placed before 1 pm.
Delivery outside Leeds takes 1 to 2 working days. Prices are in GBP and include VAT.
## Help
- [Delivery areas and times](https://petallane.example/help/delivery): Same-day zones, cut-off times and costs
- [Returns and refunds](https://petallane.example/help/refunds)
- [Flower care](https://petallane.example/help/care)
## Optional
- [Our story](https://petallane.example/about)The title, the one-line summary and the sections of links are the whole format. The llms.txt generator guide explains each part and the rules that the validator checks.
Who reads it today
Be clear about what the file does and does not do in 2026:
- Search engines do not use it for ranking. They have said so. Search results come from their own crawls of your pages.
- Support among AI tools varies. Some coding assistants, agents and docs tools read it, often when a user points them at a site. Others ignore it. The list changes, and no vendor has to tell you.
- Docs platforms generate it. Several hosted docs platforms can create llms.txt for you, so many developer docs sites have one without anyone writing it.
The most reliable way to learn who reads your file is to look. Check your server or CDN logs for requests to /llms.txt and note the user agents. A month of logs tells you more than any list on the internet, because it is about your site.
llms.txt, robots.txt and sitemap.xml
The three files sit next to each other at the root of your site, and they do different jobs.
| robots.txt | sitemap.xml | llms.txt | |
|---|---|---|---|
| Job | Says which paths crawlers may read | Lists every page you want found | Points AI tools to the few pages that explain the site |
| Read by | Search engines and most crawlers | Search engines and many crawlers | Some AI tools |
| Format | Plain text rules | XML | Markdown |
| Size | A few lines | Up to 50,000 URLs per file | Usually 10 to 50 links |
| Controls access? | Yes, for crawlers that follow it | No | No |
| Affects search ranking? | Indirectly, through what can be crawled | Helps pages get found | No |
The row that matters most is access. llms.txt cannot block anything. If you want to keep AI crawlers away from some pages, that is a job for robots.txt, and you can test your rules with the robots.txt checker.
Do you need one?
| Type of site | Recommendation | Why |
|---|---|---|
| Developer docs or an API | Yes | Coding assistants and agents are among the tools that read the file |
| Software product with a help center | Worth it | It takes under an hour, and it makes you choose the pages that describe the product |
| Online shop | Low priority | Product pages, structured data and a clean sitemap matter more |
| Local business with a few pages | Optional | Fix crawler access and page text first; then add a short file if you like |
| Content you do not want AI tools to use | Not the right tool | llms.txt invites reading; use robots.txt to block |
If you are unsure, check the basics first. A site that blocks AI search crawlers, or builds its text with JavaScript, gets nothing from llms.txt, because the pages it points to cannot be read. The AI readiness grader checks crawler access, page text and llms.txt together, so you can see which one to fix first.
What makes a file useful
- Facts in the summary. Where you are, what you sell, who it is for, the one number people ask about most. An assistant can repeat a fact correctly. It cannot do much with a slogan.
- Few links, well chosen. The pages that answer real questions: pricing, the main help articles, getting started. Not every blog post.
- Links that work. Every link should load with a 200 status, on pages that robots.txt allows.
- Generated from the same data as your pages. A hand-written file goes stale the first time a price changes. If your site is built from a CMS or code, generate the file from the same source.
You can also publish llms-full.txt, a second file with the full text of the key pages in one Markdown document. It is useful for docs, where a tool may want everything at once, and less useful for a small business site.
What we do
Paperlink AI publishes both files: /llms.txt with a summary, plan facts and links to the main pages, docs, blog posts and free tools, and /llms-full.txt, which adds the full text of the docs. Both are built from the same data as the website, so a price on the pricing page and a price in llms.txt cannot drift apart.
Questions people ask
Will llms.txt get my site cited by ChatGPT or other assistants?
No file can promise that. Assistants cite pages they can fetch and read that answer the question well. llms.txt can help a tool find those pages, but the pages do the work.
Can llms.txt hurt my site?
Not in search. The only risk is a file with wrong facts or broken links, which tells AI tools something that is not true. Check it every time your prices or main pages change.
Is llms.txt an official standard?
No. It is a proposal with a public specification. There is no standards body behind it, and no AI vendor is required to read it.