mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
Databases and storage engines
db-sharding
sharding layer for SQL databases
Vitess, Citus
System
Techniques · 20
Primitives · 57
db-sharding
Deterministic simulation testing
distributed · medium: Run the whole system in one thread under a seeded scheduler with simulated network, disk and clock, so every failure replays from its seed.
›
Scatter-gather query
distributed · medium: Send a query only to shards that can match, merge their small partial results, and bound the wait for stragglers.
›
Online resharding
distributed · medium: Split or move a shard while it serves traffic by copying a snapshot, replaying changes until the lag is tiny, then flipping a versioned routing table.
›
Primary-backup replication
distributed · medium: One node orders all writes and ships them to standbys; the acknowledgement rule (none, one, all) sets what a crash loses, and an external arbiter decides who takes over.
›
Raft
distributed · medium: Leader-based consensus on a replicated log; one leader per term, majority commit; the default for a strongly consistent replicated state machine.
›
Two-phase commit
distributed · medium: Atomic commit across shards: participants durably vote yes, then the coordinator logs and broadcasts one decision; blocks if the coordinator is lost.
›
Coverage-guided fuzzing
security · medium: Mutate inputs, run an instrumented target, and keep any input that reaches a new code edge; feedback turns blind guessing into incremental search.
›
K way merge
search-sort · low: Merge k sorted streams with a heap or loser tree (merging iterators)
Consistent hashing
distributed · low: Hash nodes and keys onto a ring; each key belongs to the next node clockwise, so membership changes move only about 1/n of keys.
›
Range partitioning
distributed · low: Cut the sorted key space into contiguous half-open ranges kept in a boundary map; split hot or big ranges, merge small ones, move ranges between nodes.
›
Leases
distributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.
›
Hash join
storage · low: Build a hash table on the smaller input's join key and probe it with every row of the larger one; the linear-time workhorse for equi-joins.
›
Recursive-descent parsing
compilers · low: Hand-written top-down parser with one function per grammar rule; simple, fast, with full control over error messages and recovery.
›
Hdr histogram
data-structures · low: Log-linear bucketed histograms with bounded relative error for latency percentiles
Distributed ID generation
distributed · low: Mint unique, roughly time-sorted IDs on many nodes with no per-ID coordination: Snowflake (time, worker, sequence) or UUIDv7 (time, random).
›
Process supervision
hardware-os · low: A parent process starts services, reaps their exits and restarts them under a delay and burst-limit policy, then stops them with SIGTERM before SIGKILL.
›
Mutual TLS
security · low: TLS where the client also proves a certificate key, so every connection carries a verified, rotatable machine identity.
›
Chunked backfill
storage · low: Process old data in bounded key ranges, one short transaction per chunk, with a saved checkpoint and idempotent writes, so a crash resumes and live traffic never waits.
›
Connection pooling
systems · low: Reuse and multiplex backend connections with bounded pools, checkout queues and health eviction
Graceful draining
systems · low: On a termination signal, stop taking new work, finish in-flight work until a deadline, tell clients to reconnect elsewhere, then exit; clients resume from a token.
›
Cooperative scheduling
concurrency · low: Green threads/coroutines yielding at await points on a user-space scheduler
Record replay
distributed · low: Record every nondeterministic input (time, randomness, I/O, model and tool outputs, per-tick player input) and re-execute deterministically for debugging, replays and state rebuild
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
+ 13 more techniques
+ 41 more primitives
System
Techniques · 20
Primitives · 57
db-sharding
Deterministic simulation testing
distributed · medium: Run the whole system in one thread under a seeded scheduler with simulated network, disk and clock, so every failure replays from its seed.
›
Scatter-gather query
distributed · medium: Send a query only to shards that can match, merge their small partial results, and bound the wait for stragglers.
›
Online resharding
distributed · medium: Split or move a shard while it serves traffic by copying a snapshot, replaying changes until the lag is tiny, then flipping a versioned routing table.
›
Primary-backup replication
distributed · medium: One node orders all writes and ships them to standbys; the acknowledgement rule (none, one, all) sets what a crash loses, and an external arbiter decides who takes over.
›
Raft
distributed · medium: Leader-based consensus on a replicated log; one leader per term, majority commit; the default for a strongly consistent replicated state machine.
›
Two-phase commit
distributed · medium: Atomic commit across shards: participants durably vote yes, then the coordinator logs and broadcasts one decision; blocks if the coordinator is lost.
›
Coverage-guided fuzzing
security · medium: Mutate inputs, run an instrumented target, and keep any input that reaches a new code edge; feedback turns blind guessing into incremental search.
›
Change data capture
storage · medium: Decode the database's own commit log into row-change events in commit order, with a resume position, so any consumer can mirror the data exactly.
›
Point-in-time recovery
storage · medium: Periodic base backups plus a continuously archived change log let you rebuild the state as of any instant, then branch from there.
›
Query optimizer
storage · medium: Cost-based plan search: join ordering (DP/greedy), rewrites, access path selection
Delta debugging
compilers · medium: Ddmin minimization of failing inputs or change sets
Fault injection
distributed · medium: Inject latency, errors, kills and partitions via proxies, netem or hooks
Incremental sync cursors
distributed · medium: The client keeps one opaque token; each pull returns only the changes after it plus the next token, so sync cost scales with what changed, not with the dataset.
›
State-machine snapshots
distributed · medium: Periodically save the state a log has produced at index i so the log prefix can be deleted; recovery and catch-up become load snapshot plus replay suffix.
›
ARIES recovery
storage · medium: Three-pass crash recovery (analysis, redo, undo) for steal/no-force buffer pools; repeats history, then rolls back losers with compensation log records.
›
Cardinality estimation
storage · medium: Row-count estimates from histograms, MCVs, sketches and sampling for the optimizer
Join order dynamic programming
storage · medium: DPccp and DPhyp enumeration of join orders with cost models; greedy fallback for many joins
Subquery decorrelation
storage · medium: Unnest correlated subqueries into joins via dependent join and magic sets
Syscall interception
security · medium: Intercept syscalls via ptrace, seccomp user notify or a user-space kernel
Buffer pool
storage · medium: The database's own page cache (frames, page table, pin counts, dirty bits, replacement) so it, not the OS, controls eviction, write order and I/O.
›
K way merge
search-sort · low: Merge k sorted streams with a heap or loser tree (merging iterators)
Consistent hashing
distributed · low: Hash nodes and keys onto a ring; each key belongs to the next node clockwise, so membership changes move only about 1/n of keys.
›
Range partitioning
distributed · low: Cut the sorted key space into contiguous half-open ranges kept in a boundary map; split hot or big ranges, merge small ones, move ranges between nodes.
›
Leases
distributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.
›
Hash join
storage · low: Build a hash table on the smaller input's join key and probe it with every row of the larger one; the linear-time workhorse for equi-joins.
›
Recursive-descent parsing
compilers · low: Hand-written top-down parser with one function per grammar rule; simple, fast, with full control over error messages and recovery.
›
Hdr histogram
data-structures · low: Log-linear bucketed histograms with bounded relative error for latency percentiles
Distributed ID generation
distributed · low: Mint unique, roughly time-sorted IDs on many nodes with no per-ID coordination: Snowflake (time, worker, sequence) or UUIDv7 (time, random).
›
Process supervision
hardware-os · low: A parent process starts services, reaps their exits and restarts them under a delay and burst-limit policy, then stops them with SIGTERM before SIGKILL.
›
Mutual TLS
security · low: TLS where the client also proves a certificate key, so every connection carries a verified, rotatable machine identity.
›
Chunked backfill
storage · low: Process old data in bounded key ranges, one short transaction per chunk, with a saved checkpoint and idempotent writes, so a crash resumes and live traffic never waits.
›
Connection pooling
systems · low: Reuse and multiplex backend connections with bounded pools, checkout queues and health eviction
Graceful draining
systems · low: On a termination signal, stop taking new work, finish in-flight work until a deadline, tell clients to reconnect elsewhere, then exit; clients resume from a token.
›
Cooperative scheduling
concurrency · low: Green threads/coroutines yielding at await points on a user-space scheduler
Record replay
distributed · low: Record every nondeterministic input (time, randomness, I/O, model and tool outputs, per-tick player input) and re-execute deterministically for debugging, replays and state rebuild
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Top-k selection
search-sort · low: Keep the k best of n items without sorting all n - bounded min-heap for streams, quickselect for arrays, threshold pruning to skip most items.
›
Deadline propagation
systems · low: Carry an absolute deadline and cancellation along a call chain
Hedged requests
systems · low: Sending a backup request after a latency percentile and taking the first answer to cut tail latency
Bulk loading
storage · low: Build an index bottom-up from sorted input (SST ingest, B-tree bulk load) instead of point inserts
Write-ahead log
storage · low: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.
›
Health checking
systems · low: Decide per backend whether to route to it using probes and real-traffic errors, with consecutive-count hysteresis, capped ejection and a panic fallback.
›
Exponential backoff with jitter
systems · low: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.
›
fsync, O_DSYNC and O_DIRECT
hardware-os · low: What fsync, fdatasync, O_DSYNC, O_DIRECT and F_FULLFSYNC really guarantee on Linux and macOS, and the exact call sequences for crash-safe files.
›
Code coverage instrumentation
compilers · low: Insert counters into compiled code to record which blocks, edges or lines ran; a spanning tree of the control-flow graph cuts the counters needed.
›
Fast hash functions
data-structures · low: Non-cryptographic hashes built from 64×64→128-bit multiplies and xor-shifts; pick by key size, trust level and whether the hash is persisted.
›
Sanitizers
security · low: ASan/UBSan/TSan shadow memory and instrumentation to catch memory bugs
Consumer offsets
distributed · low: Each consumer group stores one integer per partition, the next position to read, so a restarted consumer resumes where it left off with at-least-once delivery.
›
Message sequence numbers
distributed · low: A per-stream counter on every message lets the receiver detect loss, duplicates and reordering, ack cheaply and resume after reconnect with "send from N".
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Copy on write snapshots
storage · low: Point-in-time snapshots by sharing pages/extents and copying only on write
Partition pruning
storage · low: Skip whole partitions using partition transforms and predicate analysis
Predicate pushdown
storage · low: Move filters and projections into scans and remote sources
Relational algebra
storage · low: Relational operators and equivalence rules: selection pushdown, join reordering, projections
Failure bisection
search-sort · low: Binary search over commits, batches or inputs to isolate a failure
Transparent interception
networking · low: Redirect traffic to a local proxy (iptables TPROXY/REDIRECT, eBPF) without app changes
Cursor pagination
networking · low: Keyset/opaque-cursor pagination for stable incremental reads from APIs and tables
Copy-on-write fork
hardware-os · low: fork() gives the child the parent's whole address space by sharing pages read-only and copying one only when someone writes it; vfork and posix_spawn skip even that.
›
HyperLogLog
data-structures · low: Counts distinct elements in kilobytes with about 1 % error by tracking the longest run of leading zero bits in hashed values per bucket.
›
Quantile sketches
data-structures · low: Mergeable quantile sketches: t-digest, DDSketch, HDR histograms
Reservoir sampling
data-structures · low: Uniform k-sample from a stream of unknown length (Algorithm R/L)
Ebpf
hardware-os · low: Verified in-kernel programs on hooks (kprobes, XDP, tc) with maps
Ptrace process control
hardware-os · low: Ptrace: attach, read and write registers and memory, single-step, intercept signals
Seccomp bpf
security · low: Syscall filtering with seccomp BPF programs
LRU and CLOCK
systems · low: LRU evicts the least recently used entry via hash map plus linked list; CLOCK approximates it with a reference bit, so hits take no lock.
›
S3-FIFO
systems · low: Three FIFO queues (small, main, ghost) that evict one-hit wonders quickly and keep reused objects; low miss ratios with no list moves on hits.
›
Control-flow graph
compilers · low: Split a linear instruction list into basic blocks and connect them by jumps; the graph every optimizer, SSA builder and dataflow pass walks.
›
d-ary heap
data-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.
›
Fencing tokens
distributed · low: A number that grows with every lock grant and is checked by the protected resource, so a paused ex-holder's late write is rejected.
›
Segmented log
distributed · low: Split an append-only log into files named by their first offset, write only the newest, and find any record by binary search plus a sparse index.
›
BFS and DFS
graphs · low: The two basic graph traversals; BFS gives hop distances and levels, DFS gives discovery/finish order for cycles, components and topological order.
›
Minimum spanning tree
graphs · low: Cheapest set of edges connecting all vertices; Kruskal sorts edges and joins components with union-find, Prim grows one tree, Boruvka merges in parallel rounds.
›
Union-find
graphs · low: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.
›
CRC32C checksums
hardware-os · low: The Castagnoli CRC-32 with dedicated instructions on x86 (SSE4.2) and ARMv8. It is the standard fast integrity check for storage blocks, logs and network frames.
›
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
Sparse matrices (CSR)
numerics · low: Compressed sparse row stores only nonzeros as row pointers, column indices and values; SpMV streams them once, so it is memory-bandwidth bound.
›
Csprng
security · low: OS-backed cryptographically secure random numbers (getrandom), and when a fast PRNG is not enough
Copy for AI
Similar systems
Share the most building blocks with this one.
distributed-sql
35 %
party-lobby
31 %
iac
30 %
browser-test-grid
30 %
ingestion
29 %