Purchase conversion
ALL ASSIGNED CUSTOMERSRates include all assigned customers. Each group has approximately 21,300 customers.
HILLSTROM EMAIL EXPERIMENT / 2008
Did the email
make a difference?
A randomised holdout reveals what promotional email actually changed. Explore the evidence behind two campaigns, 64,000 customers and one business decision.
64,000assigned customers
3randomised groups
2 weeksoutcome window
36 passedautomated project tests
01 / THE EXPERIMENT
Follow the outcome.
Same customers, three ways to measure impact.
Choose an outcome to update the evidence.
CAMPAIGN EFFECTS VS CONTROL
Absolute rate changes are in percentage points; relative lift is a percentage of the control rate. All displayed intervals are pointwise 95% intervals, not simultaneous intervals.
WHAT THE EVIDENCE SUPPORTS
Both email assignments increased purchasing in this experiment. The men’s campaign is the stronger candidate for a contemporary replication. Profitability still depends on costs and margins.
02 / INSIDE THE WORKSPACE
From evidence to a decision.
A Python and Streamlit application connects
experiment integrity, inference and economics.
03 / BEHIND THE NUMBERS
Evidence you can inspect.
Original experimental data
Original Hillstrom customer assignments are preserved. Validation covers all 12 fields, missingness, category coding and outcome consistency. Duplicate-looking records are retained.
Inference with uncertainty
Two primary conversion tests use Holm correction. Newcombe intervals describe risk differences; Welch tests and a seeded bootstrap assess mean spending, including zeros.
Reproducible calculations
Twelve DuckDB analyses are reconciled against Python. The application and reports share tested calculations. All 36 automated checks passed in the recorded local verification.
PYTHONPANDAS / NUMPYSCIPY / STATSMODELSDUCKDBPLOTLYSTREAMLITPYTEST
Keep the study in context.
This is a 2008 experiment with two-week outcomes, not a current campaign. Assignment is observed; delivery, opens, clicks and actual campaign costs are unavailable. None of the 14 exploratory interaction tests survives Holm correction. The study does not establish individual treatment effects or present-day profitability.
Original-data redistribution permission is unresolved. This page publishes aggregate results and dashboard captures, with no customer-level records. Source acquisition used the publisher’s original HTTP endpoint after its HTTPS certificate failed; the recorded checksum supports byte reproducibility.
EXPLORE THE PROJECT
Take a closer look.
Read the reasoning, inspect the estimates or reproduce the analysis.