B
Evidence Grade B
Graded
accuracy verified by the review judge
Safe to Cite
Recent
26.1%
26.1% -- inflated false alarm rate for A/B tests with continuous peeking (2025)
Continuous peeking in A/B testing inflates the false alarm rate from 5% to 26.1%, according to calculations by Evan Miller.
Attribution:
View source
Lighthouse Research Team
Last verified:
·
How we verify data →
Source verified
Evidence graded
Value
26.1%
Unit
Percentage
Data Vintage
2025
Entity
Evidence Quality
B
Strong, reliable evidence
Sourced from a well-regarded research firm or industry leader with transparent methodology and a reasonably large sample.
Single source - not yet corroborated
Score breakdown
Extraction
75
Source Trust
60
Citation Chain
20
Independence
30
Methodology
45
Freshness
95
Source
Evan Miller
Primary
Tier B
https://kirro.io/ab-test-sample-size-formula ↗
Source Article
A/B Test Sample Size Formula, Explained With Examples
The A/B test sample size formula is:
n = (Zα/2 + Zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)²
If that looks like gibberish, you’re not alone. Most articles either throw this formula at you with zero explanation, or skip it and say “just use a calculator.”
This guide does both. We’ll break down every piec…
Original Research
Evan Miller
· Original study ↗
Methodology
Evan Miller's statistical calculation on peeking
Methodology documented
Trend Over Time
| Period | Value | Change |
|---|---|---|
| 2025 (current) | 26.1% | |
| 2026 | 26.1% | 0% |
Segmentation
Industrymarketing
Cite this stat
Plain Text
A/B tests with continuous peeking: 26.1% inflated false alarm rate. Source: Evan Miller (2025). Via Lighthouse Intelligence -- https://lighthousedata.io/data/peeking-inflates-false-alarm-26-1pct
HTML Embed
<blockquote cite="https://lighthousedata.io/data/peeking-inflates-false-alarm-26-1pct" style="border-left:3px solid #2563eb;padding:12px 16px;margin:16px 0;font-family:system-ui,sans-serif;background:#f0f4ff;"><strong>26.1%</strong> -- inflated false alarm rate for A/B tests with continuous peeking<br><small>Source: Evan Miller (2025) · <a href="https://lighthousedata.io/data/peeking-inflates-false-alarm-26-1pct" target="_blank" rel="noopener">Lighthouse Intelligence</a></small></blockquote>
Share
Related Statistics
Use this data in your work
Copy the citation, embed the widget, or explore thousands more verified marketing statistics -- all free.