Statistics & experimentation

Pick a test statistic, compare it to its distribution under "no effect", read off a p-value, and control error rates across one test or many.

Estimate a conversion or event rate from few observations with a Beta-Binomial or Gamma-Poisson update: two counters, an honest interval, and P(B beats A) directly.

Location and scale estimates (median, MAD, trimmed mean, Huber M-estimator) that stay correct when a minority of the data is garbage.

Randomly split units into arms by a salted hash, log exposure, compare metrics with a confidence interval, and guard the result with sample-ratio, guardrail and peeking checks.

Subtract the part of each unit's outcome that a pre-experiment covariate predicts; the effect estimate stays unbiased and its variance shrinks by 1 - rho^2.

Partition the tunable parameters into layers, hash each unit with a per-layer salt so layers are independent, and give experiments inside a layer disjoint bucket ranges.

Add up the log-likelihood ratio per observation and stop when it crosses a fixed bound; the least data of any test with the same error rates (for two point hypotheses).

Score = (C·m + sum of ratings)/(C + n): the posterior mean under a prior worth C ratings, so items with few ratings sit near the global mean.

Answer probability is a logistic curve of ability minus item difficulty (1PL), with item slope (2PL) and a guessing floor (3PL); fit items by marginal ML, score people, measure precision by information.

Predict the expected value from a seasonal baseline, scale the residual by a robust spread (MAD), threshold at a chosen false-alarm rate, then merge flags into incidents.

Estimate normal behaviour from healthy data, then signal when values leave limits (Shewhart) or small deviations add up (CUSUM, EWMA); tune by average run length.

The end-to-end recipe: fix horizon and uncertainty, beat naive baselines, choose a model family, and validate with rolling-origin backtests and scale-free metrics.

Forecast by keeping a few running states (level, trend, seasonal) updated by a fraction of each forecast error; O(1) memory per series and a strong baseline.

Split a series into trend, seasonal and remainder with iterated loess smoothing; the seasonal shape may drift and robust weights keep outliers out of it.

Estimate forecast error by repeatedly training on the past and testing on the next h steps, moving the origin forward; never trains on the future.