Skip to content

Web design, software and technical support

Log in +90 539 219 79 78 WhatsApp
SEO & GEO

What is llms.txt and should your website have one?

llms.txt is a file that summarises your key pages for language models as a plain Markdown list. Its structure, how it differs from robots.txt and sitemap.xml, and realistic expectations.

Boran KaradenizKarınca Bilişim Last updated: 7 min

Summarise this article with an AI tool

The tool opens in a new tab with a summary prompt that includes this article’s address.

llms.txt is a plain text file placed at the root of a website that summarises its most important pages as a simple Markdown list that large language models can read easily. Jeremy Howard proposed it in 2024. It is not an official web standard, and the major search engines have not said they use it as a ranking signal; it is quick and harmless to add, but its effect should be treated as limited.

What is llms.txt?

llms.txt is a file that tells language models, in short and orderly text, what your site is and which of its pages matter. It is published at the root of your domain, for example at yourdomain.com/llms.txt.

The problem behind the idea is simple: web pages are full of menus, cookie notices, scripts and repeated template fragments. When a language model reads a page, it has to dig the actual information out of that noise. llms.txt offers a clean table of contents instead.

Jeremy Howard, co-founder of Answer.AI, published the proposal in 2024, and the full description is at llmstxt.org. Since then a number of software documentation sites and developer tools have adopted it, and several content management systems and SEO plugins have started offering to generate the file automatically.

How is an llms.txt file structured?

llms.txt is a simple text file written in Markdown that follows a set order. Under the proposal only the first heading is required; every other section is optional.

The suggested structure runs in this order:

  1. Title: the name of the site or project, written as a heading with a single hash. This is the only mandatory part.
  2. Short summary: one or two sentences describing the site, in a quotation line that starts with a greater-than sign.
  3. Notes: optional paragraphs about how the site should be read. No headings are used here.
  4. Link sections: subheadings opened with two hashes (for example "Services" or "Guides"), each followed by a bulleted list. Every item contains the page name in square brackets, the URL in round brackets and, optionally, a short description after a colon.
  5. Optional section: a section titled "Optional" collects secondary links that can be skipped when context is short.

The proposal also discusses serving Markdown versions of pages by adding ".md" to the same URL, and expanded files such as llms-full.txt that hold all the content in one place. None of this is required; for most corporate sites a single llms.txt is enough.

On Karınca Bilişim's own site, llms.txt is generated automatically from the published services, company pages and blog posts, so it stays up to date whenever a new page goes live.

How does llms.txt differ from robots.txt and sitemap.xml?

llms.txt is neither an access rule nor a complete list of URLs; it is a curated summary for language models. robots.txt tells crawlers what they may access, and sitemap.xml tells search engines about every URL that can be indexed.

Featurerobots.txtsitemap.xmlllms.txt
PurposeCrawler access rulesList of URLs to indexSummary and selected links for language models
FormatPlain text directivesXMLMarkdown
Read byAll crawlersSearch enginesLanguage models and AI tools
ScopeBlocking and allowingAll important URLsKey pages with descriptions
StatusEstablished standardEstablished standardProposal, limited adoption
Does it block access?Yes (for compliant crawlers)NoNo

The practical consequence matters: adding a page to llms.txt does not make a page blocked in robots.txt accessible, and removing a page from llms.txt does not hide it. Access decisions are always made in robots.txt and server settings.

Which platforms use llms.txt?

In short: adoption is growing, but how far the major AI companies and search engines read the file has not been publicly confirmed. Google representatives have said on several occasions that no special file or markup is needed for its AI features.

An honest assessment therefore looks like this:

  • Known: more sites publish the file, and some developer tools and coding assistants can use llms.txt when reading documentation.
  • Unknown: whether general-purpose chat assistants read the file systematically when generating answers, and how much weight it carries if they do.
  • Stated plainly: Google says normal SEO practice is enough to appear in its AI features.

By watching requests for /llms.txt in your server logs you can see which crawlers actually ask for the file on your own site.

Is llms.txt worth adding?

For most corporate sites, yes, as long as expectations stay modest and the file does not replace basic SEO work. It takes little time to write, maintenance can be automated and no negative effect is known.

It is particularly useful when:

  • you publish technical documentation, an API guide or a knowledge base;
  • your services are spread across many pages and you want to offer a short map;
  • you want to write your brand and service definition once, consistently, in one place.

Other work should come first when:

  • the site has indexing or crawling problems (start with our technical SEO checklist);
  • service pages are thin or repeat each other;
  • structured data and brand information are inconsistent.

How do you write a good llms.txt?

A good llms.txt is short, current and matches the visible content of the site exactly. Do not include claims or pages that are not on the site.

  • Describe what the business does in one plain sentence in the summary line.
  • Group links by topic: services, company pages, guides.
  • Add a one-sentence description to each link, consistent with the page's meta description.
  • Leave out drafts, archived pages and pages closed to indexing.
  • Generate the file from your content management system, or update it after every release.
  • On multilingual sites, label the language versions clearly.

llms.txt is a small part of wider GEO work. For the full picture read our guide to GEO, and for an assessment of your own site see our GEO and search visibility service. On WordPress sites we decide together, as part of the WordPress SEO service, whether the file should come from a plugin or from the theme.

Frequently asked questions

Will adding llms.txt improve my rankings?

There is no public evidence that it does. The major search engines have not said they use llms.txt as a ranking signal. It is better seen as a helpful summary of your content than as a ranking tool.

Can I block AI crawlers with llms.txt?

No. llms.txt contains no access rules. To block specific crawlers you need robots.txt and, where necessary, server or CDN settings.

Where does the llms.txt file go?

In the root directory of your domain, reachable at yourdomain.com/llms.txt. It should be served as plain text and be publicly accessible.

How often should I update it?

Whenever you add a service, publish an important guide or change a URL. The most reliable approach is to generate the file from your content management system so it always matches what is published.

All articles
Call nowWhatsApp