Measurement · Experimentation · Causal Inference
Paid-media measurement, A/B and causal inference case studies, the
experiment-toolkit
Python package, and short essays on the methods behind each one.
Each one is end-to-end, tested, and reviewable.
A compact paid-media measurement lab covering geo lift, attribution, incrementality, MMM, and web-attribution caveats. Synthetic data, recovered ground truth, decision memos.
Daily demand forecasting on public retail data, evaluated like production: a seasonal-naive baseline, ETS / SARIMA / LightGBM, and a leakage-free rolling-origin backtest. SARIMA reaches MASE 0.74 — 18.5% below baseline — plus a difference-in-differences read that recovers a known +15% promo uplift.
Six worked case studies covering A/B with CUPED, propensity-score methods, difference-in-differences (including Callaway-Sant'Anna), synthetic control, sequential testing, and switchback designs. Each paired with a short memo and the diagnostics that back it.
A small, typed, tested Python package. Sample size and MDE (CUPED-aware), sequential testing (mSPRT), ratio-metric delta method, staggered DiD, and observational sensitivity analysis. Each function cites the paper it implements.
pip install experiment-toolkitSQL-first product analytics case studies. Funnels, retention cohorts, engagement segmentation, and a memo proposing a defensible north-star metric under Goodhart's law.
Eighteen SQL problems in the style of large consumer-tech interview loops. Windowed aggregations, sessionisation, retention curves, cohort funnels, with notes on pitfalls and dialect differences.
For fun — algorithms and puzzles outside the day job.
Short essays on the methods behind the case studies.
Why a LightGBM lost to a plain SARIMA on a daily retail series — and how the right baseline, metric, and backtest made that the trustworthy answer.
The structural bias in LTA, why it compounds into budget, and what to reach for instead.
Cluster randomisation at the DMA level, synthetic control as its observational cousin, and MDE before you commit.
A measurement stack that earns credibility: MMM for allocation, geo for calibration, incrementality for campaigns.
Why the default DiD regression is biased under staggered adoption, and what to reach for instead.
Quantifying robustness to unmeasured confounding, with Rosenbaum bounds as the matched-pair companion.
When and how to combine CUPED with mSPRT without breaking the always-valid guarantee.
A case for property-based tests in statistical code.
What overlap, double robustness, and a known experimental benchmark look like on a classic dataset.
A framework for metric selection under Goodhart's law.
The projects above are the fastest way in. Open an issue and I'll see it.