mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
Consumer applications
short-video
short-video app
TikTok
System
Techniques · 28
Primitives · 82
short-video
CDN and edge
cloud · medium: CDNs cache content in hundreds of PoPs near users, cutting latency and egress bills; edge compute runs small code there. Bandwidth price spreads 10x between providers.
›
Deterministic simulation testing
distributed · medium: Run the whole system in one thread under a seeded scheduler with simulated network, disk and clock, so every failure replays from its seed.
›
Partitioned commit log
distributed · medium: A topic split into replicated append-only partitions: key hash keeps per-key order, offsets give position, and a high watermark limits readers to fully replicated records.
›
Primary-backup replication
distributed · medium: One node orders all writes and ships them to standbys; the acknowledgement rule (none, one, all) sets what a crash loses, and an external arbiter decides who takes over.
›
Windowed stream aggregation
distributed · medium: Group events by event-time window per key, keep one partial aggregate per slice, and emit a window when the watermark passes its end.
›
Deep ranking models for recommendat…
machine-learning · medium: Embedding tables plus a feature-interaction net predict several feedback signals per candidate; a weighted sum of them is the ranking score.
›
Multi-armed bandits
machine-learning · medium: Choose among K options repeatedly while learning their payoffs: epsilon-greedy, UCB1 and Thompson sampling keep total regret low where a fixed split or a fixed exploration rate pays forever.
›
Embeddings
machine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Count-min sketch
data-structures · low: d rows of w counters estimate any item's frequency in a stream, overestimating by at most ε·N with probability 1-δ, in fixed tiny memory.
›
Cuckoo hashing
data-structures · low: Every key lives in one of two candidate buckets, so lookups read at most two cache lines; inserts evict residents to their alternate bucket.
›
Hdr histogram
data-structures · low: Log-linear bucketed histograms with bounded relative error for latency percentiles
Distributed ID generation
distributed · low: Mint unique, roughly time-sorted IDs on many nodes with no per-ID coordination: Snowflake (time, worker, sequence) or UUIDv7 (time, random).
›
Process supervision
hardware-os · low: A parent process starts services, reaps their exits and restarts them under a delay and burst-limit policy, then stops them with SIGTERM before SIGKILL.
›
Perceptual hashing
search-sort · low: Compact fingerprints (64 to 256 bits) of an image that stay close under resizing and recompression, compared by Hamming distance; pHash, dHash and Meta's PDQ.
›
Mutual TLS
security · low: TLS where the client also proves a certificate key, so every connection carries a verified, rotatable machine identity.
›
Graceful draining
systems · low: On a termination signal, stop taking new work, finish in-flight work until a deadline, tell clients to reconnect elsewhere, then exit; clients resume from a token.
›
HTTP caching semantics
networking · low: The rules (RFC 9111) that decide whether a stored response is still fresh, and how a cheap conditional request revalidates it when it is not.
›
Cooperative scheduling
concurrency · low: Green threads/coroutines yielding at await points on a user-space scheduler
Record replay
distributed · low: Record every nondeterministic input (time, randomness, I/O, model and tool outputs, per-tick player input) and re-execute deterministically for debugging, replays and state rebuild
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Fast hash functions
data-structures · low: Non-cryptographic hashes built from 64×64→128-bit multiplies and xor-shifts; pick by key size, trust level and whether the hash is persisted.
›
Consumer offsets
distributed · low: Each consumer group stores one integer per partition, the next position to read, so a restarted consumer resumes where it left off with at-least-once delivery.
›
+ 21 more techniques
+ 66 more primitives
System
Techniques · 28
Primitives · 82
short-video
CDN and edge
cloud · medium: CDNs cache content in hundreds of PoPs near users, cutting latency and egress bills; edge compute runs small code there. Bandwidth price spreads 10x between providers.
›
Deterministic simulation testing
distributed · medium: Run the whole system in one thread under a seeded scheduler with simulated network, disk and clock, so every failure replays from its seed.
›
Partitioned commit log
distributed · medium: A topic split into replicated append-only partitions: key hash keeps per-key order, offsets give position, and a high watermark limits readers to fully replicated records.
›
Primary-backup replication
distributed · medium: One node orders all writes and ships them to standbys; the acknowledgement rule (none, one, all) sets what a crash loses, and an external arbiter decides who takes over.
›
Windowed stream aggregation
distributed · medium: Group events by event-time window per key, keep one partial aggregate per slice, and emit a window when the watermark passes its end.
›
Deep ranking models for recommendat…
machine-learning · medium: Embedding tables plus a feature-interaction net predict several feedback signals per candidate; a weighted sum of them is the ranking score.
›
Multi-armed bandits
machine-learning · medium: Choose among K options repeatedly while learning their payoffs: epsilon-greedy, UCB1 and Thompson sampling keep total regret low where a fixed split or a fixed exploration rate pays forever.
›
Two-tower retrieval
machine-learning · medium: Two separate encoders map users/queries and items into one vector space; item vectors are precomputed and searched by nearest neighbour, so recall over millions costs one small forward pass.
›
Adaptive bitrate streaming
media · medium: The client fetches short HTTP segments and picks each segment's rendition from throughput estimates, buffer level, or both, trading stalls against quality and switches.
›
Video transcoding ladder
media · medium: Turn one source into resolution x bitrate x codec renditions chosen from measured rate-quality points, cut on aligned keyframes so players can switch mid-stream.
›
HNSW
search-sort · medium: Hierarchical Navigable Small World: a layered proximity graph searched greedily from a sparse top layer; the default in-memory approximate nearest-neighbour index.
›
Coverage-guided fuzzing
security · medium: Mutate inputs, run an instrumented target, and keep any input that reaches a new code edge; feedback turns blind guessing into incremental search.
›
A/B testing (online controlled expe…
statistics · medium: Randomly split units into arms by a salted hash, log exposure, compare metrics with a confidence interval, and guard the result with sample-ratio, guardrail and peeking checks.
›
Point-in-time recovery
storage · medium: Periodic base backups plus a continuously archived change log let you rebuild the state as of any instant, then branch from there.
›
Delta debugging
compilers · medium: Ddmin minimization of failing inputs or change sets
Fault injection
distributed · medium: Inject latency, errors, kills and partitions via proxies, netem or hooks
Watermarks
distributed · medium: Event-time progress markers that bound lateness and trigger window results
Contrastive learning
machine-learning · medium: Train an encoder so two views of the same thing score higher than every other pair in the batch; InfoNCE with a temperature, no labels needed.
›
Chunked parallel encoding
media · medium: Split video at keyframes, encode chunks in parallel, stitch with rate consistency
TLS 1.3
security · medium: The standard encrypted, authenticated transport; one round trip, only forward-secret AEAD suites, and little left to misconfigure.
›
Hypothesis testing
statistics · medium: Pick a test statistic, compare it to its distribution under "no effect", read off a p-value, and control error rates across one test or many.
›
Sequential hypothesis testing
statistics · medium: Group-sequential and alpha-spending designs for interim analyses without inflating error
ARIES recovery
storage · medium: Three-pass crash recovery (analysis, redo, undo) for steal/no-force buffer pools; repeats history, then rolls back losers with compensation log records.
›
LSM-tree
storage · medium: Buffer writes in memory, flush them as immutable sorted files and merge in the background; turns random writes into sequential I/O.
›
Video rate control
media · medium: CBR/VBR/CRF rate control with VBV buffer model and lookahead
Syscall interception
security · medium: Intercept syscalls via ptrace, seccomp user notify or a user-space kernel
Sequential probability ratio test
statistics · medium: Add up the log-likelihood ratio per observation and stop when it crosses a fixed bound; the least data of any test with the same error rates (for two point hypotheses).
›
Buffer pool
storage · medium: The database's own page cache (frames, page table, pin counts, dirty bits, replacement) so it, not the OS, controls eviction, write order and I/O.
›
Embeddings
machine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Count-min sketch
data-structures · low: d rows of w counters estimate any item's frequency in a stream, overestimating by at most ε·N with probability 1-δ, in fixed tiny memory.
›
Cuckoo hashing
data-structures · low: Every key lives in one of two candidate buckets, so lookups read at most two cache lines; inserts evict residents to their alternate bucket.
›
Hdr histogram
data-structures · low: Log-linear bucketed histograms with bounded relative error for latency percentiles
Distributed ID generation
distributed · low: Mint unique, roughly time-sorted IDs on many nodes with no per-ID coordination: Snowflake (time, worker, sequence) or UUIDv7 (time, random).
›
Process supervision
hardware-os · low: A parent process starts services, reaps their exits and restarts them under a delay and burst-limit policy, then stops them with SIGTERM before SIGKILL.
›
Perceptual hashing
search-sort · low: Compact fingerprints (64 to 256 bits) of an image that stay close under resizing and recompression, compared by Hamming distance; pHash, dHash and Meta's PDQ.
›
Mutual TLS
security · low: TLS where the client also proves a certificate key, so every connection carries a verified, rotatable machine identity.
›
Graceful draining
systems · low: On a termination signal, stop taking new work, finish in-flight work until a deadline, tell clients to reconnect elsewhere, then exit; clients resume from a token.
›
HTTP caching semantics
networking · low: The rules (RFC 9111) that decide whether a stored response is still fresh, and how a cheap conditional request revalidates it when it is not.
›
Cooperative scheduling
concurrency · low: Green threads/coroutines yielding at await points on a user-space scheduler
Record replay
distributed · low: Record every nondeterministic input (time, randomness, I/O, model and tool outputs, per-tick player input) and re-execute deterministically for debugging, replays and state rebuild
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Fast hash functions
data-structures · low: Non-cryptographic hashes built from 64×64→128-bit multiplies and xor-shifts; pick by key size, trust level and whether the hash is persisted.
›
Consumer offsets
distributed · low: Each consumer group stores one integer per partition, the next position to read, so a restarted consumer resumes where it left off with at-least-once delivery.
›
Segmented log
distributed · low: Split an append-only log into files named by their first offset, write only the newest, and find any record by binary search plus a sparse index.
›
Zero-copy I/O
networking · low: Move bytes between files, sockets and NICs by passing page references instead of copying through user space: sendfile, splice, MSG_ZEROCOPY, kTLS.
›
Leases
distributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.
›
Write-ahead log
storage · low: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.
›
Health checking
systems · low: Decide per backend whether to route to it using probes and real-traffic errors, with consecutive-count hysteresis, capped ejection and a panic fallback.
›
Hash aggregation
storage · low: GROUP BY via hash tables with partitioning and spill to disk
Hierarchical timing wheel
systems · low: Timers bucketed by expiry into wheels of slots, coarser wheels for far deadlines: O(1) insert and cancel for millions of timeouts.
›
Feature hashing (the hashing trick)
machine-learning · low: Hash each feature name to an index in a fixed-size vector, with a second hash for the sign, so no vocabulary is needed and memory is fixed.
›
Mixture of experts (MoE)
machine-learning · low: Replace one FFN with many expert FFNs and a router sending each token to its top-k; parameters scale with E, per-token compute with k.
›
Embedding table sharding
ml-training · low: Sharding huge embedding tables across devices (row/column/table-wise) with all-to-all lookups and caching
Bayesian rate estimation
statistics · low: Estimate a conversion or event rate from few observations with a Beta-Binomial or Gamma-Poisson update: two counters, an honest interval, and P(B beats A) directly.
›
Negative sampling
machine-learning · low: Train by contrasting each observed positive with a few sampled negatives instead of normalising over every candidate; the choice of sampler decides bias and quality.
›
Adaptive streaming manifests
media · low: HLS playlists and DASH MPDs: renditions, segment timelines, codecs strings
GOP structure
media · low: How an encoder orders I, P and B frames into groups of pictures: the knob behind seekability, compression, latency and stream switching.
›
Media containers
media · low: MP4/ISO BMFF, fragmented MP4/CMAF, Matroska and MPEG-TS mux/demux
HTTP range requests
networking · low: Ask for byte offsets with Range and get a 206 slice back; the standard way to resume, seek and read big remote files piecewise.
›
Exponential smoothing (ETS, Holt-Wi…
statistics · low: Forecast by keeping a few running states (level, trend, seasonal) updated by a fraction of each forecast error; O(1) memory per series and a strong baseline.
›
Image resampling filters
graphics · low: Resize an image by convolving with a kernel (box, triangle, cubic, Lanczos) whose width scales with the shrink factor, done as two separable 1-D passes with precomputed weights.
›
Perceptual quality metrics (PSNR, S…
media · low: Full-reference scores that say how bad a compressed picture looks: PSNR (pixel error), SSIM (structure), VMAF (learned fusion), and when each one lies.
›
Video codecs
media · low: H.264/HEVC/AV1 basics: frames, GOPs, hardware decode/encode APIs and latency settings
d-ary heap
data-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.
›
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
Scalar quantization
search-sort · low: Per-dimension int8/binary quantization of vectors with rescoring
Code coverage instrumentation
compilers · low: Insert counters into compiled code to record which blocks, edges or lines ran; a spanning tree of the control-flow graph cuts the counters needed.
›
Sanitizers
security · low: ASan/UBSan/TSan shadow memory and instrumentation to catch memory bugs
Bootstrap confidence intervals
statistics · low: Resampling-based confidence intervals for arbitrary statistics
CUPED (variance reduction with pre-…
statistics · low: Subtract the part of each unit's outcome that a pre-experiment covariate predicts; the effect estimate stays unbiased and its variance shrinks by 1 - rho^2.
›
Experiment layering (overlapping ex…
statistics · low: Partition the tunable parameters into layers, hash each unit with a per-layer salt so layers are independent, and give experiments inside a layer disjoint bucket ranges.
›
Deterministic bucketing
systems · low: Hash a stable unit id with a per-purpose salt into a bucket; "in" means bucket below the percentage, so assignment is stateless, repeatable and only grows on ramp-up.
›
Copy on write snapshots
storage · low: Point-in-time snapshots by sharing pages/extents and copying only on write
Failure bisection
search-sort · low: Binary search over commits, batches or inputs to isolate a failure
Transparent interception
networking · low: Redirect traffic to a local proxy (iptables TPROXY/REDIRECT, eBPF) without app changes
Data augmentation
machine-learning · low: Generate label-preserving variants of each training example on the fly to enlarge the effective dataset and bake in invariances.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Scene cut detection
media · low: Detect shot changes from frame-difference statistics
AEAD (AES-GCM, ChaCha20-Poly1305)
security · low: Authenticated encryption that encrypts and detects tampering in one call; the only kind of symmetric encryption you should use.
›
Cryptographic hashes (SHA-2, BLAKE3)
security · low: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.
›
ECDH key exchange (X25519, P-256)
security · low: Each side multiplies the other's public curve point by its own scalar and both get the same point; always validate the peer key and run the result through a KDF.
›
HKDF key derivation (extract-and-ex…
security · low: HMAC-based two-step key derivation: extract a uniform pseudorandom key from any secret, then expand it into as many context-bound keys as you need.
›
X509 certificates
security · low: Certificate chains, path validation, SANs, revocation and short-lived issuance
Wilson score interval
statistics · low: Wilson lower bound for ranking by up/down votes and proportions with few samples
Adaptive quantization
media · low: Per-block QP offsets driven by local texture energy: spend bits on flat areas where blocking shows, save them where texture masks the error.
›
Ebpf
hardware-os · low: Verified in-kernel programs on hooks (kprobes, XDP, tc) with maps
Ptrace process control
hardware-os · low: Ptrace: attach, read and write registers and memory, single-step, intercept signals
Seccomp bpf
security · low: Syscall filtering with seccomp BPF programs
Loglikelihood scoring
machine-learning · low: Scoring choices by summed token log-probabilities, normalization by length, perplexity computation
fsync, O_DSYNC and O_DIRECT
hardware-os · low: What fsync, fdatasync, O_DSYNC, O_DIRECT and F_FULLFSYNC really guarantee on Linux and macOS, and the exact call sequences for crash-safe files.
›
LRU and CLOCK
systems · low: LRU evicts the least recently used entry via hash map plus linked list; CLOCK approximates it with a reference bit, so hits take no lock.
›
S3-FIFO
systems · low: Three FIFO queues (small, main, ghost) that evict one-hit wonders quickly and keep reused objects; low miss ratios with no list moves on hits.
›
Control-flow graph
compilers · low: Split a linear instruction list into basic blocks and connect them by jumps; the graph every optimizer, SSA builder and dataflow pass walks.
›
Fencing tokens
distributed · low: A number that grows with every lock grant and is checked by the protected resource, so a paused ex-holder's late write is rejected.
›
Message sequence numbers
distributed · low: A per-stream counter on every message lets the receiver detect loss, duplicates and reordering, ack cheaply and resume after reconnect with "send from N".
›
Convolution image filters
graphics · low: Separable 2D convolution kernels for blur, sharpen (unsharp mask) and edges with border handling
BFS and DFS
graphs · low: The two basic graph traversals; BFS gives hop distances and levels, DFS gives discovery/finish order for cycles, components and topological order.
›
Minimum spanning tree
graphs · low: Cheapest set of edges connecting all vertices; Kruskal sorts edges and joins components with union-find, Prim grows one tree, Boruvka merges in parallel rounds.
›
Union-find
graphs · low: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.
›
CRC32C checksums
hardware-os · low: The Castagnoli CRC-32 with dedicated instructions on x86 (SSE4.2) and ARMv8. It is the standard fast integrity check for storage blocks, logs and network frames.
›
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Collective communication
ml-training · low: The family of group exchanges (broadcast, reduce, all-gather, reduce-scatter, all-to-all) and the latency-plus-bytes cost model that picks ring, tree or direct algorithms by message size and topology.
›
Reverse-mode automatic differentiat…
ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
Discrete cosine transform
numerics · low: Fast integer DCT/DST block transforms used by image and video codecs
Sparse matrices (CSR)
numerics · low: Compressed sparse row stores only nonzeros as row pointers, column indices and values; SpMV streams them once, so it is memory-bandwidth bound.
›
Constant time code
security · low: Writing secret-dependent code without branches or memory lookups on secrets; constant-time compare and select
Csprng
security · low: OS-backed cryptographically secure random numbers (getrandom), and when a fast PRNG is not enough
Slab allocator
systems · low: Pre-carve pages into equal-size slots per size class with an in-place free list: O(1) alloc and free, no external fragmentation, cache-warm reuse.
›
Copy for AI
Similar systems
Share the most building blocks with this one.
social-feed
45 %
dating
36 %
photo-backup
36 %
music-streaming
35 %
digital-asset-management
35 %