Why every website needs an llms.txt file now

More and more search is moving from Google into chat interfaces — people ask ChatGPT, Claude or Perplexity directly about companies, products and specific people, instead of clicking through ten links. These tools don't read websites the way a classic search engine does: they don't have the time or the context window to wade through navigation, cookie banners and ads to get to what actually matters on the page.

That's why the llms.txt standard exists — a plain markdown file at /llms.txt that lets a site tell a model directly what it's about, with no clutter. The format is simple and precisely specified: an H1 heading with the site or project name, a blockquote below it with a short summary, and then sections separated by H2 headings containing links with short descriptions — typically Projects, Writing and Contact sections. It's essentially a sitemap.xml, but written for a model instead of a crawler.

Beyond llms.txt, it's worth doing the things that help classic SEO just as much as AI search: schema.org structured data (typically type Person or Organization for a person or company), a clean sitemap.xml, a robots.txt that doesn't block anything important, and above all content that's readable without JavaScript — if a model or its crawler can't render the page, it doesn't exist for it.

I build this in for clients right from the start of a project, not as an afterthought later. We're at the very beginning of this shift — the earlier a site is readable for AI search, the earlier a company or person shows up in it once more people actually move over to it. It's cheaper and simpler to do now, while few others are doing it, than to catch up once it becomes standard.