B
Evidence Grade B
Graded
accuracy verified by the review judge
Safe to Cite
Current
87.5%
87.5% -- sites without ai crawler policy for top websites (Tranco sample) (Q1 2026)
87.5% of websites in the sample had no AI crawler policy in their robots.txt file.
Attribution:
View source
Lighthouse Research Team
Last verified:
·
How we verify data →
Source verified
Evidence graded
Value
87.5%
Unit
Percentage
Data Vintage
Q1 2026
Entity
Sample Size
1,460
Geography
EMEA
Evidence Quality
B
Strong, reliable evidence
Sourced from a well-regarded research firm or industry leader with transparent methodology and a reasonably large sample.
Single source - not yet corroborated
Score breakdown
Extraction
95
Source Trust
60
Citation Chain
20
Independence
30
Methodology
45
Freshness
95
Source
AI Visibility
Primary
Tier A
https://www.ai-visibility.org.uk/blog/how-ai-ready-is-the-web/ ↗
Source Article
How AI-Ready Is the Web? What Our Research Data Reveals | AI Visibility
The headline finding
We crawled nearly 2,000 of the web's most prominent domains to answer a simple question: how prepared is the internet for AI?
The answer is worse than we expected. Not because websites aren't trying, but because most of the ones that are trying are getting it wrong.
6.5% of the …
Original Research
AI Visibility
· Original study ↗
Methodology
robots.txt checked for 15 known AI user agents
Methodology documented
Segmentation
IndustryTechnology
RegionEMEA
RoleAll marketers
Channelcontent
Cite this stat
Plain Text
top websites (Tranco sample): 87.5% sites without ai crawler policy. Source: AI Visibility (Q1 2026). Via Lighthouse Intelligence -- https://lighthousedata.io/data/no-ai-crawler-policy-q1-2026
HTML Embed
<blockquote cite="https://lighthousedata.io/data/no-ai-crawler-policy-q1-2026" style="border-left:3px solid #2563eb;padding:12px 16px;margin:16px 0;font-family:system-ui,sans-serif;background:#f0f4ff;"><strong>87.5%</strong> -- sites without ai crawler policy for top websites (Tranco sample)<br><small>Source: AI Visibility (Q1 2026) · <a href="https://lighthousedata.io/data/no-ai-crawler-policy-q1-2026" target="_blank" rel="noopener">Lighthouse Intelligence</a></small></blockquote>
Share
Use this data in your work
Copy the citation, embed the widget, or explore thousands more verified marketing statistics -- all free.