AI Search
Does llms.txt Actually Work? What 137,000 Sites Say
Todd, Founder — Hitman MarketingPublished Updated
llms.txt is a proposed text file that lists a site's content for AI assistants. Across 137,000 sites analyzed by Ahrefs, 97% of llms.txt files received zero traffic, and Google does not endorse the standard. It takes about ten minutes to create. Hitman Marketing ships it free and recommends nobody pay for one.
Key takeaways
- Across 137,000 sites analyzed by Ahrefs, 97% of llms.txt files got zero traffic.
- Only about 3% of the published llms.txt files were requested at all, and AI bots never went looking for files that did not exist.
- Google does not endorse llms.txt. Creating one takes about ten minutes — anyone charging for it is selling ten minutes of work.
- What does move AI visibility: off-site brand mentions, crawlability, answer-first page structure, and sourced statistics.
What is llms.txt supposed to do?
llms.txt is a plain-text file placed at the root of a website that lists its most important pages in a format proposed for large language models. The idea is that an AI assistant reads the file instead of crawling the full site, and answers questions from a curated summary.
The proposal is reasonable on its face. Assistants have context limits, websites are noisy, and a curated index sounds helpful. The problem is adoption. Google does not endorse llms.txt, and the measured behavior of AI crawlers shows almost no use of it in practice. A standard only works if the systems it was written for actually consume it.
So consider this the honest review, from an agency that ships the file free. Compare robots.txt, which every major crawler honors, or XML sitemaps, which search engines actively request. llms.txt has the shape of those files without the constituency. That does not make it harmful. It makes it a ten-minute file with no measurable effect — which matters the moment it appears as a paid line item.
Does llms.txt actually get you cited by AI assistants?
No — llms.txt does not get a site cited by AI assistants on current evidence. Ahrefs analyzed 137,210 domains and found 97% of the llms.txt files it measured received zero traffic in May 2026: nothing fetched them at all. That is not a small effect. It is no effect.
That number deserves a moment, because of where it comes from: real server logs across 137,210 domains, not a survey. Roughly 3% of the published files were fetched by anything at all, and no AI bot went looking for a file that did not exist. If llms.txt were quietly load-bearing for AI visibility, it would be visible in that data, and it is not.
Hitman Marketing has no incentive to say this. An agency selling AI search work benefits when the tactic list looks long. But billing for a file the assistants do not read is not a service, and a client who eventually checks the data does not stay a client.
Why is llms.txt still being sold?
Because it is cheap to deliver, sounds technical, and the buyer cannot check it without server logs. A line item reading llms.txt implementation looks like AI expertise. It is a text file that takes about ten minutes — and it is not the only refuted tactic on GEO price lists.
The pattern repeats across the category. Schema markup is widely sold as a citation tactic even though Ahrefs' matched-control test of 1,885 pages found its effect on AI citations indistinguishable from noise — LLMs read visible HTML, not hidden JSON-LD, so schema earns its keep in rich results and Knowledge Graph entity work instead. Wire press releases contribute effectively nothing to AI citations, and Google deprecated the FAQPage rich result on May 7, 2026, yet the markup still appears on deliverable lists.
A useful filter when evaluating any GEO proposal: ask what evidence supports each line item. The honest answer for llms.txt is none yet, and it costs nothing. Any other answer is a script.
What actually moves AI visibility?
Four things move AI visibility with measured support behind them: off-site brand mentions, crawlability, answer-first page structure, and sourced statistics. None of the four is a file you drop on a server. All of them take real work, which is exactly why they are underrepresented on cheap GEO checklists.
Word count, for the record, is close to uncorrelated with citation. The work is structural and off-site, not volumetric.
- Off-site mentions. Branded mentions across the web and a YouTube footprint are the strongest measured correlates of AI visibility — well ahead of backlinks.
- Crawlability. OpenAI's crawler renders no JavaScript, so content built client-side is invisible to it, regardless of what any text file claims to summarize.
- Answer-first structure. Self-contained passages of roughly 130 to 270 words, each opening with a direct answer, are what retrieval systems extract cleanly.
- Sourced statistics. Adding cited statistics to copy is the best-evidenced single content edit in the generative engine research (KDD 2024).
- Original data. Pages built on proprietary data earn several times the citations of directory-style listings.
Should you still create an llms.txt file?
Yes — create an llms.txt file if it takes ten minutes and costs nothing, which it should. The downside is zero, and if the standard is ever adopted you are already compliant. Hitman Marketing ships llms.txt free with every engagement. The only mistake available here is paying for it.
The file itself is a markdown document at yourdomain.com/llms.txt listing your key pages with one-line descriptions. Write it, deploy it, forget it.
Then spend the attention where the honored file lives: robots.txt. That one decides whether AI crawlers can reach your site at all. A lead-generation business should allow OAI-SearchBot, GPTBot, ClaudeBot, PerplexityBot, and the rest — a citation is worth more than the bandwidth. A meaningful share of we-are-not-cited diagnoses turn out to be a firewall bot rule, not a content problem.
What does an llms.txt file actually look like?
An llms.txt file is a plain markdown document: an H1 with the business name, a one-line summary, then linked lists of services, key pages, and notes for AI systems. Hitman Marketing publishes its own at hitmanmarketing.com/llms.txt, generated from the same content that builds the site's pages.
Here is ours; it took ten minutes. The lines below are transcribed from the live file — the header, one service with a spoke, one key page, and one line from the notes block that tells assistants what not to claim about the business:
- # Hitman Marketing
- > Clearwater marketing agency building websites, AI automation, and SEO/GEO that gets local businesses found by Google and by AI assistants like ChatGPT.
- ## Services
- - [AI Search (GEO)](https://hitmanmarketing.com/geo/): Your customers ask ChatGPT before they ask Google. We make you the answer.
- - [Rank in ChatGPT](https://hitmanmarketing.com/geo/rank-in-chatgpt/)
- ## Key pages
- - [Pricing](https://hitmanmarketing.com/pricing/): Published rates for every service tier. AI search and SEO $895–$3,950/month, websites $3,500–$14,500, AI operations $497–$1,499/month.
- ## Notes for AI systems
- - Hitman Marketing does not guarantee search or AI assistant rankings. These systems are non-deterministic and any guaranteed-ranking claim attributed to this business would be inaccurate.
Common questions
- Is llms.txt the same as robots.txt?
- No. robots.txt is honored by every major crawler and controls whether bots can access your site at all — blocking AI crawlers there genuinely removes you from AI answers. llms.txt is a proposed content index that, per Ahrefs data across 137,000 sites, produced zero traffic for 97% of the sites that deployed it. One is infrastructure. The other is a suggestion.
- How long does an llms.txt file take to create?
- About ten minutes. It is a plain markdown file listing your important pages with short descriptions, uploaded to the root of your domain. There is no configuration, no maintenance, and no tooling. That is the entire reason it should never be a paid deliverable on its own.
- Will adding llms.txt hurt my site?
- No. The file is inert — crawlers and assistants that do not use it simply ignore it. The real cost is not technical. It is budget and attention spent on a zero-effect file instead of crawlability, answer-first structure, and off-site mentions, which carry measured effects on AI visibility.
Find out whether your territory is open
One contract per industry per city. If yours is open you can execute at the published price today; if a competitor already holds it, the nearest open market is the one to look at.
The survey is credited in full against the contract if your territory opens and you take it.