B
Evidence Grade B
Graded
accuracy verified by the review judge
Safe to Cite
Recent
1700x
1700x -- scrapes per referral ratio for OpenAI crawler (2025-06)
In June 2025, Cloudflare found that OpenAI's crawler scraped websites approximately 1,700 times for every referral sent back.
- Approximation ('about 1,700 times')
Attribution:
View source
Lighthouse Research Team
Last verified:
·
How we verify data →
Source verified
Evidence graded
Value
1700x
Unit
Ratio
Data Vintage
2025-06
Entity
Evidence Quality
B
Strong, reliable evidence
Sourced from a well-regarded research firm or industry leader with transparent methodology and a reasonably large sample.
Single source - not yet corroborated
Score breakdown
Extraction
95
Source Trust
60
Citation Chain
60
Independence
30
Methodology
45
Freshness
95
Source
Cloudflare
Primary
Tier A
https://www.ai-visibility.org.uk/blog/do-ai-crawlers-read-ai-discovery-files/ ↗
Source Article
Do AI Crawlers Read Your AI Discovery Files? | AI Visibility
Update, 1 July 2026: a month on, they arrived
When this went out, our 11-day log showed only Meta and Bing. A month on, the picture changed: over the 30 days to 29 June, 11 AI crawlers made 73 requests to this site's files, none blocked, and the major LLM crawlers were among them this time. Nine of …
Original Research
Cloudflare
· Original study ↗
Methodology
Cloudflare network-wide crawler-to-referral analysis
Methodology documented
Segmentation
Industrymarketing
Cite this stat
Plain Text
OpenAI crawler: 1700x scrapes per referral ratio. Source: Cloudflare (2025-06). Via Lighthouse Intelligence -- https://lighthousedata.io/data/cloudflare-openai-scrape-to-referral-ratio-2025
HTML Embed
<blockquote cite="https://lighthousedata.io/data/cloudflare-openai-scrape-to-referral-ratio-2025" style="border-left:3px solid #2563eb;padding:12px 16px;margin:16px 0;font-family:system-ui,sans-serif;background:#f0f4ff;"><strong>1700x</strong> -- scrapes per referral ratio for OpenAI crawler<br><small>Source: Cloudflare (2025-06) · <a href="https://lighthousedata.io/data/cloudflare-openai-scrape-to-referral-ratio-2025" target="_blank" rel="noopener">Lighthouse Intelligence</a></small></blockquote>
Share
Use this data in your work
Copy the citation, embed the widget, or explore thousands more verified marketing statistics -- all free.