SignalCrawler
Sign inStart free auditFree audit

llms.txt: what it is, an example, and whether you need one

Updated 11 October 2026

llms.txt is a plain Markdown file at https://yoursite.com/llms.txt that tells AI tools what your site is and links to the pages worth reading. It was proposed in September 2024 at llmstxt.org as a way to give language models a compact, clean overview instead of making them parse menus, ads and scripts.

Does it help?

Be realistic. It is a proposal, not an adopted standard. Google has said it does not use llms.txt for Search, and other large AI companies have not committed to reading it. It is used by some developer tools and AI coding assistants, which read documentation sites' llms.txt files, and it costs little to maintain.

Worth doing if you run documentation, an API, a software product or a site with clear reference pages. Low priority for a typical small business site; robots.txt rules, crawlable content and clear pages matter far more for AI answers.

The format

In order:

  1. An H1 with the name of the site or project. The only required part.
  2. A blockquote with a short summary.
  3. Optional paragraphs with details, without headings.
  4. H2 sections with Markdown lists of links: - [Title](URL): optional note.
  5. An optional ## Optional section for links a tool can skip.

A complete example

# Acme Analytics

> Acme Analytics is a privacy-friendly web analytics tool for small sites. No cookies, EU hosting, from $9 a month.

Acme works with any website through one script tag. Data is kept for two years.

## Product

- [Features](https://acme.example/features): what Acme measures and what it does not
- [Pricing](https://acme.example/pricing): plans, limits and what counts as a page view

## Docs

- [Install](https://acme.example/docs/install): add the script to any site, WordPress or Shopify
- [API](https://acme.example/docs/api): export data, authentication, rate limits

## Policies

- [Privacy](https://acme.example/privacy): what data is collected and where it is stored

## Optional

- [Changelog](https://acme.example/changelog): release notes

Good practice

  • Keep it short. It is a map, not a copy of the site. Link the pages that answer what people ask: what you do, prices, how it works, policies, contact.
  • Use absolute URLs that answer 200, not redirects.
  • Write the notes for a reader with no context: what is on the page, not marketing.
  • Serve it as text, with status 200. A site that answers unknown addresses with its normal page gives tools an HTML page where they expect Markdown.
  • Update it when you add or rename key pages.

Some sites add llms-full.txt with the full text of the key pages in one file. It is optional.

What llms.txt does not do

It does not grant or deny access; robots.txt does that. Blocking AI crawlers in robots.txt while publishing an llms.txt sends mixed signals. See how to block or allow AI crawlers.

Check yours

The free llms.txt checker checks the structure and whether the linked pages answer.

Related