← Back
Home › Systems › IT operations, analytics and integration

product-analytics

product event analytics with funnels

Amplitude, PostHog

SystemTechniques · 9Primitives · 35 product-analytics Cohort retentionstorage · medium: Retention matrices: group users by first event period and count returns per later periodColumnar storagestorage · medium: Store each column contiguously so analytic scans read only the columns they need, compress well and run as tight SIMD loops.›Funnel analysisstorage · medium: Ordered multi-step conversion within a time window per user, computed in one passIdentity resolutionstorage · medium: Merge anonymous and known identifiers into one profile (union-find over identity edges)Vectorized executionstorage · medium: Query operators pass batches of about 1-2K values per column instead of single tuples, amortizing interpretation overhead and enabling tight SIMD loops.›Entity resolutionstorage · medium: Decide which records describe the same real-world thing: block to cut the pair space, score pairs with a probabilistic model, cluster the matches transitively.›Record linkagesearch-sort · medium: Probabilistic record matching (Fellegi-Sunter) with blocking for patient or entity indexesRoaring bitmapscompression · low: Compressed set of 32-bit integers in 64K-value chunks, each a sorted array, bitmap or run list; fast intersections, the standard for bitmap indexes.›Dictionary encodingcompression · low: Map values to small integer codes via a dictionary for compact, comparable columnsZone mapsstorage · low: Per-block min/max (and sparse primary) indexes to skip data during scansSessionizationstorage · low: Group events into sessions by inactivity gapsUnion-findgraphs · low: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.›Hash based samplingdata-structures · low: Deterministic sampling by hashing an entity id so all its events are kept or dropped togetherHyperLogLogdata-structures · low: Counts distinct elements in kilobytes with about 1 % error by tracking the longest run of leading zero bits in hashed values per bucket.›Quantile sketchesdata-structures · low: Mergeable quantile sketches: t-digest, DDSketch, HDR histogramsExponential backoff with jittersystems · low: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.›Hash aggregationstorage · low: GROUP BY via hash tables with partitioning and spill to diskWindow functionsstorage · low: ROW_NUMBER, rank, running and frame aggregates via sort and segment treesCalendar date arithmeticsystems · low: Civil dates, time zones (tzdb), DST gaps, month-end anchoring and duration mathBit packing and PFORcompression · low: Store blocks of integers with exactly b bits each and patch rare outliers separately (PFOR); the fastest integer codec family, billions of values per second.›Delta encodingcompression · low: Store differences between consecutive values instead of the values, turning sorted IDs, timestamps and counters into small numbers that pack into few bits.›Hash joinstorage · low: Build a hash table on the smaller input's join key and probe it with every row of the larger one; the linear-time workhorse for equi-joins.›Embeddingsmachine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.›+ 2 more techniques+ 19 more primitives

Similar systems

Share the most building blocks with this one.

realtime-olap 29 %columnar-warehouse 16 %web-analytics 15 %