What is llms.txt?
llms.txt is a short file written in Markdown that sits at the root of your domain, the same place as robots.txt. Where robots.txt tells crawlers what they may not read, llms.txt tries to do the opposite: it points AI tools at the pages that best explain who you are and what you offer.
The idea was put forward by Jeremy Howard of Answer.AI in September 2024. His reasoning was practical. Language models have limited context, and a typical web page is full of menus, cookie banners and scripts. A clean list of links with short summaries is much easier for a model to use when it needs to answer a question about your product or documentation.
What goes inside the file
- One H1 heading with the name of your site or project. This is the only required part.
- A short summary in a blockquote, one or two sentences on what you do.
- Optional notes in plain paragraphs, for anything a model should know before reading further.
- H2 sections with lists of links, each link followed by a colon and a one-line description.
- An "Optional" section for pages that can be skipped when space is tight.
# Acme Payroll
> Payroll and tax filing software for small businesses in India, from 1 to 200 employees.
## Product
- [Features](https://acme.example/features): payroll runs, payslips, PF and ESI filing
- [Pricing](https://acme.example/pricing): plans from Rs 999 a month, free trial
## Help
- [Getting started](https://acme.example/help/start): set up your company in 15 minutes
- [Compliance calendar](https://acme.example/help/calendar): monthly filing deadlines
## Optional
- [Blog](https://acme.example/blog): guides on payroll and labour lawSome sites also publish llms-full.txt, a single file with the full text of their key pages. That's mostly useful for software documentation.
Do Google, ChatGPT or Claude actually use llms.txt?
This is where the hype and the evidence part ways. Google's own guidance on AI features in Search says you don't need to create new machine-readable files or special markup to appear in AI Overviews or AI Mode. Google's search team has also said it has no plans to support llms.txt, and John Mueller has compared it to the keywords meta tag, a signal that was easy to add and ended up being ignored.
None of the big AI search products, including ChatGPT search, Gemini, Claude and Perplexity, has publicly said it reads llms.txt when deciding what to cite. Several site owners who checked their server logs report that AI crawlers rarely request the file at all.
There is one twist. In 2026 Google's Lighthouse tool added an audit for llms.txt under a new agent-focused category, describing it as an emerging convention for AI agents that browse sites on a user's behalf. So the file may matter more for agents that act inside a browser than for search answers. That's worth watching, but it's not a ranking factor today.
Should you add an llms.txt file anyway?
Our view: yes if it's quick, no if it's replacing real work. We publish one on growmarketingpro.com. It took about 20 minutes, it costs nothing to host, and writing it was a useful exercise in describing our product in plain words. That description now matches what's on our home page, pricing page and help centre, which does matter for AI.
Where it goes wrong is when a team ships llms.txt and calls their AI search work done. The things that measurably affect whether assistants mention you are elsewhere:
- 1Let the right crawlers in
If OAI-SearchBot, Claude-SearchBot or PerplexityBot are blocked in robots.txt or by your firewall, no file will help. Our guide to robots.txt for AI crawlers covers the full list.
- 2Put your answers in the HTML
AI crawlers often don't run JavaScript. If your pricing or features only appear after a script loads, many assistants never see them.
- 3Get mentioned where AI looks
Review sites, comparison articles and forums shape answers more than anything on your own domain. See brand mentions vs backlinks.
- 4Then add llms.txt as a tidy extra
List your 10 to 20 most useful pages with honest one-line descriptions, and keep it updated when pages change.
Common llms.txt mistakes
- Listing every URL. The point is curation. A file with 2,000 links is a sitemap with extra steps.
- Marketing copy instead of descriptions. "The world's best platform" tells a model nothing. "Payroll software for teams of 1 to 200" does.
- Linking to pages that are blocked. If robots.txt disallows a page, listing it in llms.txt just sends a crawler to a closed door.
- Letting it go stale. Old prices or discontinued products in a file meant for AI is worse than no file.
Frequently asked questions
What is llms.txt?
A proposed Markdown file at the root of a website that lists its most useful pages with short descriptions, so AI tools can find them quickly. It was suggested by Jeremy Howard in September 2024 and is a community convention, not an official standard.
Does Google use llms.txt?
Google says you don't need special files or markup to appear in AI Overviews or AI Mode, and its search team has said it doesn't plan to support llms.txt. Google's Lighthouse tool does check for it as part of an agent-browsing audit.
Does llms.txt help you rank in ChatGPT?
There's no public evidence that it does. OpenAI hasn't said ChatGPT search reads it. Crawler access, clear page content and mentions on trusted sites have a much bigger effect.
What's the difference between llms.txt and robots.txt?
robots.txt tells crawlers which parts of a site they may not access, and major crawlers follow it. llms.txt suggests which pages are most useful, and nothing is required to read it.
What is llms-full.txt?
A companion file that contains the full text of a site's key pages in one place. It's mostly used for software documentation, where developers paste it into an AI tool.