MDE & A/B Test Duration Calculator

The Minimum Detectable Effect (MDE) is the smallest relative improvement in a conversion rate that an A/B test is mathematically powered to detect over a specific timeframe. Calculating your MDE and test duration ensures you gather a sufficient sample size, preventing you from calling tests too early and falling victim to statistical “peeking.”

How do you use this Test Duration calculator?

1. Enter Baseline Conversion Rate: Input your current site or landing page conversion rate.
2. Enter Target Uplift: Input the relative percentage increase you hope to achieve.
3. Enter Daily Visitors: Estimate the amount of traffic this specific page receives per day.
4. Get Your Roadmap: The calculator will output the total Sample Size Needed and the Estimated Days the test must run untouched.

What is MDE and why is test duration important?

Before launching a test, data scientists must balance traffic volume with the size of the expected impact. This requires setting:

• Statistical Power (80-95%): The probability that the test will correctly reject the null hypothesis when a true effect exists (avoiding Type II errors). For marketing purposes, 80% is strong enough, but some teams or executives prefer 95%.
• Target Relative Uplift: The percentage improvement you are trying to prove (e.g., a 10% lift on a 2% baseline).
• Sample Size: The total traffic required to reach validity without succumbing to variance.

How do you calculate MDE and required sample size?

Calculating required sample size involves complex statistics factoring in Z-scores for alpha (confidence) and beta (power).

For data teams using Python, the statsmodels library handles this calculation perfectly:

import statsmodels.stats.api as sms

# Define baseline CR and expected minimum effect
baseline_rate = 0.02
expected_uplift = 0.10
new_rate = baseline_rate * (1 + expected_uplift)

# Calculate effect size
effect_size = sms.proportion_effectsize(baseline_rate, new_rate)

# Calculate required sample size per variant (80% power, alpha 0.05)
sample_size = sms.NormalIndPower().solve_power(
    effect_size, 
    power=0.8, 
    alpha=0.05, 
    ratio=1
)
print(f"Required Sample Size Per Variant: {int(sample_size)}")

Why use an MDE Calculator instead of guessing test length?