87.5%
87.5% -- sites without ai crawler policy for top websites (Tranco sample) (Q1 2026)
87.5% of the websites in the crawled Tranco sample have no AI crawler policy in their robots.txt file.
Questions about this data
What is the sites without ai crawler policy for top websites (Tranco sample)?
The sites without ai crawler policy for top websites (Tranco sample) is 87.5% as of Q1 2026 according to AI Visibility. This figure is the primary benchmark in the Lighthouse dataset for this segment. Individual results vary by region, channel mix, and audience quality.
What is a good sites without ai crawler policy benchmark for top websites (Tranco sample)?
A strong sites without ai crawler policy benchmark for top websites (Tranco sample) is 87.5%, using the current serving value as of Q1 2026. This is one serving-eligible observation, not a distribution median; top-quartile comparisons require the underlying distribution. Vertical, audience size, and campaign type all shift what 'good' means.
How does top websites (Tranco sample) sites without ai crawler policy compare to the industry average?
The published sites without ai crawler policy serving value for top websites (Tranco sample) is 87.5% as of Q1 2026. Individual sectors and audiences may differ substantially; this single observation does not establish an aggregate mean. Quartile comparisons require the underlying distribution, which is not implied by this record.
What factors affect sites without ai crawler policy for top websites (Tranco sample)?
Primary drivers of sites without ai crawler policy variation for top websites (Tranco sample) include audience segmentation, timing, and channel selection. Methodology documents campaign factors, but this serving record does not quantify the effect of personalization. The current serving value is 87.5% as of Q1 2026.
What sites without ai crawler policy should I aim for as a digital marketer in 2026?
For a digital marketer, the published sites without ai crawler policy serving value is 87.5%; it is an evidence-grade reference, not a personalized target as of Q1 2026. Any performance goal should account for channel mix and audience. Review the methodology before operational decisions; a gap from this serving value does not by itself indicate a structural issue.
Approved for serving with evidence grade B. The record passes the documented quality and serving gates; consult the methodology for provenance details.
Single source - not yet corroborated
robots.txt checked for 15 known AI user agents
<blockquote cite="https://lighthousedata.io/data/no-ai-crawler-policy-q1-2026" style="border-left:3px solid #2563eb;padding:12px 16px;margin:16px 0;font-family:system-ui,sans-serif;background:#f0f4ff;"><strong>87.5%</strong> -- sites without ai crawler policy for top websites (Tranco sample)<br><small>Source: AI Visibility (Q1 2026) · <a href="https://lighthousedata.io/data/no-ai-crawler-policy-q1-2026" target="_blank" rel="noopener">Lighthouse Intelligence</a></small></blockquote>
Use this data in your work
Copy the citation, embed the widget, or explore thousands more verified marketing statistics -- all free.