llms.txt: what it is and how to add one

Picture a travel agency in Thrissur whose web developer says the site "needs an llms.txt for ChatGPT". Is that true? Partly. The file is quick to add and does no harm. But on its own it won't get the agency named in ChatGPT's answers, and Google Search will ignore it completely.

What is an llms.txt file?

It's a plain Markdown file (a simple text format where # marks a heading), served at yoursite.com/llms.txt. It gives AI language models a short, hand-picked list of a site's most important pages. It began in 2024 as a community proposal, described at llmstxt.org, not as an official standard.

The idea is that an AI tool can read one clean page to learn what a site covers. It doesn't have to crawl thousands of pages or a homepage heavy with JavaScript. Our glossary entry on llms.txt has more detail.

How llms.txt differs from robots.txt and sitemap.xml

These three files do different jobs, and it's worth having all three.

Put simply, robots.txt is a "do not enter" sign, sitemap.xml is a floor plan, and llms.txt is a recommended reading list.

Does llms.txt help AI tools find you?

Not much, so far. But it does no harm and costs almost nothing.

• Google Search ignores it. Google's AI optimisation guide (May 2026) says you don't need llms.txt, chunked content or AI-specific rewriting to appear in its AI features. The file won't help or hurt your Google rankings.
• Few files get read at all. In Ahrefs' study, 97% of llms.txt files got no requests. Of the requests that did come, most were from SEO tools and other bots. Coding agents asked for the file more often than AI search engines did.
• Other AI tools may use it. Google doesn't decide for ChatGPT, Perplexity or Claude. When an AI tool or coding assistant does read it, the file points it to your best pages instead of a busy homepage.

If AI visibility is a ladder, llms.txt is a rung near the top, not the foundation. Put most of your effort into the foundation:

• Useful, answer-first content on your own site.
• The same name, address and phone number on every listing, and real reviews.
• Mentions on other trusted sites, such as directories, LinkedIn and comparison sites, so AI models know your business before they search.

How to set up llms.txt

Create a file called llms.txt in your website's root folder, so it opens at yoursite.com/llms.txt. A simple one looks like this:

# Your Business Name
> One sentence that describes what your business does and who it serves.

## Core Services
- [Service One](https://yoursite.com/service-one)

## Important Guides
- [AEO Guide for 2026](https://yoursite.com/blog/aeo-guide-2026)

Notice what's left out: menus, cookie banners and repeated links. Only the pages you'd want an AI tool to read first go in. You can see ours at /llms.txt. Then follow three rules.

1. Keep it short and current. Use a few dozen links at most: the pages that define your business. Add important new pages, remove retired ones, and update it whenever you update your sitemap.
2. Check that AI crawlers can get in. llms.txt only helps AI tools that can reach your site. If robots.txt or your CDN blocks crawlers such as GPTBot or CCBot (Common Crawl's crawler), your pages stay out of the data AI models learn from. A CDN (a service such as Cloudflare that sits in front of your website) can do this without anyone noticing. Check its bot settings, and make sure they never block Googlebot.
3. Make your pages easy to read. Serve key pages as clean HTML, and if your site can offer Markdown copies, link to those from llms.txt. Put the direct answer in the first sentence or two of each page, under headings that match real questions.

Common llms.txt mistakes

1. Treating it as a Google ranking tactic. Google Search ignores it. It's a courtesy to AI tools.
2. Blocking the crawlers that would read it. Set up crawler access and llms.txt together.
3. Listing every page. A copy of your sitemap is noise. Dozens of links, not hundreds.
4. Letting it go stale. Links to removed pages send AI tools to dead ends.
5. Using it to fake coverage. List real pages only. Google's guide also warns that chasing inauthentic mentions across the web helps less than it might seem.

What to do this week

1. Run a free AI-visibility check to see what AI crawlers can and can't read on your site.
2. Check your robots.txt and CDN settings, so you aren't blocking crawlers you want.
3. Write a short llms.txt: your name, one line about what you do, and your key pages under a few headings.
4. Upload it to your site's root, check it opens at yoursite.com/llms.txt, and set a reminder to update it when key pages change.

How Simeon helps

We set up llms.txt as part of every AEO and GEO project. First we check your robots.txt and CDN for accidental AI-crawler blocks. Then we write a short, curated llms.txt. Alongside it, we rework your key pages answer-first and help you get mentioned on other trusted sites.