97% Of LLMS.txt Files Got No Requests, Ahrefs Data Shows
Ahrefs analyzed 137,000 domains and discovered that 97% of llms.txt files receive zero traffic. The data indicates that while AI bots are the largest segment of the small amount of traffic that does exist, they are significantly less active than anticipated by industry advocates.
Why it matters
This suggests that the adoption of llms.txt as a standard for AI-friendly web content is currently minimal, challenging assumptions about how AI crawlers interact with the open web.
Ahrefs analyzed 137K domains and found 97% of llms.txt files received zero requests. AI retrieval bots accounted for 1% of total requests.
The article reports on empirical data analysis without taking a stance on the utility of the file format.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in