← Back
Home › Systems › AI and ML infrastructure

model-monitoring

model drift and quality monitoring

Arize

SystemTechniques · 8Primitives · 32 model-monitoring Windowed stream aggregationdistributed · medium: Group events by event-time window per key, keep one partial aggregate per slice, and emit a window when the watermark passes its end.›Distribution drift detectionmachine-learning · medium: PSI, KS and other tests for feature and score drift between training and productionChange point detectionstatistics · medium: CUSUM/E-divisive/PELT to find shifts in a time series mean or distributionHypothesis testingstatistics · medium: Pick a test statistic, compare it to its distribution under "no effect", read off a p-value, and control error rates across one test or many.›Columnar storagestorage · medium: Store each column contiguously so analytic scans read only the columns they need, compress well and run as tight SIMD loops.›Watermarksdistributed · medium: Event-time progress markers that bound lateness and trigger window resultsLSM-treestorage · medium: Buffer writes in memory, flush them as immutable sorted files and merge in the background; turns random writes into sequential I/O.›Quantile sketchesdata-structures · low: Mergeable quantile sketches: t-digest, DDSketch, HDR histogramsCount-min sketchdata-structures · low: d rows of w counters estimate any item's frequency in a stream, overestimating by at most ε·N with probability 1-δ, in fixed tiny memory.›HyperLogLogdata-structures · low: Counts distinct elements in kilobytes with about 1 % error by tracking the longest run of leading zero bits in hashed values per bucket.›Calibrationmachine-learning · low: Make predicted probabilities match observed frequencies: measure with ECE, fix post hoc with temperature, Platt or isotonic scaling on held-out data.›Embeddingsmachine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.›Evaluation metricsmachine-learning · low: Precision, recall, ROC AUC, average precision, log loss, NDCG and pass@k computed correctly, plus paired confidence intervals for comparing models.›PCA and SVDmachine-learning · low: Find orthogonal directions of maximum variance by an SVD of the centred data; randomized SVD gets the top k components in a few passes.›Hash aggregationstorage · low: GROUP BY via hash tables with partitioning and spill to diskHierarchical timing wheelsystems · low: Timers bucketed by expiry into wheels of slots, coarser wheels for far deadlines: O(1) insert and cancel for millions of timeouts.›Bayesian rate estimationstatistics · low: Estimate a conversion or event rate from few observations with a Beta-Binomial or Gamma-Poisson update: two counters, an honest interval, and P(B beats A) directly.›Robust statisticsstatistics · low: Location and scale estimates (median, MAD, trimmed mean, Huber M-estimator) that stay correct when a minority of the data is garbage.›Bootstrap confidence intervalsstatistics · low: Resampling-based confidence intervals for arbitrary statisticsWilson score intervalstatistics · low: Wilson lower bound for ranking by up/down votes and proportions with few samplesBit packing and PFORcompression · low: Store blocks of integers with exactly b bits each and patch rare outliers separately (PFOR); the fastest integer codec family, billions of values per second.›Delta encodingcompression · low: Store differences between consecutive values instead of the values, turning sorted IDs, timestamps and counters into small numbers that pack into few bits.›Dictionary encodingcompression · low: Map values to small integer codes via a dictionary for compact, comparable columns+ 1 more techniques+ 16 more primitives

Similar systems

Share the most building blocks with this one.

metric-anomaly-detection 33 %fraud-detection 27 %data-quality 25 %rum 20 %recommender 19 %