B
Evidence Grade B
Graded
accuracy verified by the review judge
Safe to Cite
Current
33%
33% -- share of test wins that are random noise for A/B tests (2026)
According to Wharton research, 33% of A/B test 'wins' are actually just random noise under standard conditions (rising to 42% when stopped early).
- Baseline noise level before p-hacking/early stopping, which increases to 42%
Attribution:
View source
Lighthouse Research Team
Last verified:
·
How we verify data →
Source verified
Evidence graded
Value
33%
Unit
Percentage
Data Vintage
2026
Entity
Evidence Quality
B
Strong, reliable evidence
Sourced from a well-regarded research firm or industry leader with transparent methodology and a reasonably large sample.
Single source - not yet corroborated
Score breakdown
Extraction
75
Source Trust
60
Citation Chain
20
Independence
30
Methodology
45
Freshness
95
Source
Wharton
Primary
Tier B
https://kirro.io/cro-program ↗
Source Article
CRO Program: Build One That Actually Lasts (2026)
A CRO program is the system that makes conversion rate optimization ongoing instead of one-off. Not a single audit. Not a random test someone runs when they have an idea. It’s the people, tools, governance, and habits that keep testing running month after month.
Most businesses skip this entirely. T…
Original Research
Wharton
· Original study ↗
Methodology
Noise share rises from 33% to 42% when tests are called early (p-hacking)
Methodology documented
Segmentation
Industrymarketing
Cite this stat
Plain Text
A/B tests: 33% share of test wins that are random noise. Source: Wharton (2026). Via Lighthouse Intelligence -- https://lighthousedata.io/data/false-win-rate-noise-33-42
HTML Embed
<blockquote cite="https://lighthousedata.io/data/false-win-rate-noise-33-42" style="border-left:3px solid #2563eb;padding:12px 16px;margin:16px 0;font-family:system-ui,sans-serif;background:#f0f4ff;"><strong>33%</strong> -- share of test wins that are random noise for A/B tests<br><small>Source: Wharton (2026) · <a href="https://lighthousedata.io/data/false-win-rate-noise-33-42" target="_blank" rel="noopener">Lighthouse Intelligence</a></small></blockquote>
Share
Use this data in your work
Copy the citation, embed the widget, or explore thousands more verified marketing statistics -- all free.