← Back
Home › Systems › CI/CD and developer tooling

flaky-test-detection

flaky test detection and quarantine

Buildkite Test Analytics

SystemTechniques · 6Primitives · 20 flaky-test-detection Windowed stream aggregationdistributed · medium: Group events by event-time window per key, keep one partial aggregate per slice, and emit a window when the watermark passes its end.›Near-duplicate detection with MinHa…machine-learning · medium: Estimate set Jaccard similarity from k min-hashes and find similar pairs with banded LSH buckets; the standard dedup for web crawls, training corpora and sequence collections.›Log template miningsearch-sort · medium: Extract log templates and parameters by prefix-tree clustering (Drain)Hypothesis testingstatistics · medium: Pick a test statistic, compare it to its distribution under "no effect", read off a p-value, and control error rates across one test or many.›Watermarksdistributed · medium: Event-time progress markers that bound lateness and trigger window resultsLSM-treestorage · medium: Buffer writes in memory, flush them as immutable sorted files and merge in the background; turns random writes into sequential I/O.›Failure bisectionsearch-sort · low: Binary search over commits, batches or inputs to isolate a failureStack trace fingerprintingsearch-sort · low: Normalize and hash stack frames to group crashesBayesian rate estimationstatistics · low: Estimate a conversion or event rate from few observations with a Beta-Binomial or Gamma-Poisson update: two counters, an honest interval, and P(B beats A) directly.›Exponential smoothing (ETS, Holt-Wi…statistics · low: Forecast by keeping a few running states (level, trend, seasonal) updated by a fraction of each forecast error; O(1) memory per series and a strong baseline.›Hash aggregationstorage · low: GROUP BY via hash tables with partitioning and spill to diskHierarchical timing wheelsystems · low: Timers bucketed by expiry into wheels of slots, coarser wheels for far deadlines: O(1) insert and cancel for millions of timeouts.›Fast hash functionsdata-structures · low: Non-cryptographic hashes built from 64×64→128-bit multiplies and xor-shifts; pick by key size, trust level and whether the hash is persisted.›Rolling hashdata-structures · low: Rabin-Karp polynomial rolling hashes and winnowing for substring fingerprints and document matchingSwiss tabledata-structures · low: Open-addressing hash table that filters 8-16 slots at once with 1-byte hash tags and SIMD, giving fast lookups at 87.5 % load.›Radix triedata-structures · low: Compressed prefix tree for path routing, domain matching and prefix lookupsUnicode text tokenization for searchsearch-sort · low: Turn multilingual text into index terms: segment words (UAX 29 style, CJK bigrams), normalize, case-fold and accent-fold, identically at index and query time.›Bootstrap confidence intervalsstatistics · low: Resampling-based confidence intervals for arbitrary statisticsWilson score intervalstatistics · low: Wilson lower bound for ranking by up/down votes and proportions with few samplesd-ary heapdata-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.›Consumer offsetsdistributed · low: Each consumer group stores one integer per partition, the next position to read, so a restarted consumer resumes where it left off with at-least-once delivery.›Fencing tokensdistributed · low: A number that grows with every lock grant and is checked by the protected resource, so a paused ex-holder's late write is rejected.›+ 4 more primitives

Similar systems

Share the most building blocks with this one.

metric-anomaly-detection 18 %error-tracking 16 %