mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
Security and identity
dlp
data loss prevention
Google DLP
System
Techniques · 20
Primitives · 70
dlp
Deterministic simulation testing
distributed · medium: Run the whole system in one thread under a seeded scheduler with simulated network, disk and clock, so every failure replays from its seed.
›
Primary-backup replication
distributed · medium: One node orders all writes and ships them to standbys; the acknowledgement rule (none, one, all) sets what a crash loses, and an external arbiter decides who takes over.
›
Named entity recognition
machine-learning · medium: Find the character spans of entities in text and give each a type, by labelling every token and decoding a valid label sequence.
›
Near-duplicate detection with MinHa…
machine-learning · medium: Estimate set Jaccard similarity from k min-hashes and find similar pairs with banded LSH buckets; the standard dedup for web crawls, training corpora and sequence collections.
›
Optical character recognition (text…
machine-learning · medium: Find text lines in an image, read each crop with a CTC or attention model, then correct with a lexicon and score confidence for review routing.
›
Text classification
machine-learning · medium: Assign one or more labels to a text; climb a ladder from regexes to linear models on n-grams, embeddings, fine-tuned encoders and LLM prompts, only as far as the data and budget demand.
›
Linear-time regular expression matc…
search-sort · medium: Match a regular expression in O(pattern x text) worst case by tracking all live parse positions at once, then cache the state sets as a lazily built DFA for speed.
›
Rolling hash
data-structures · low: Rabin-Karp polynomial rolling hashes and winnowing for substring fingerprints and document matching
Aho-Corasick multi-pattern matching
search-sort · low: Find all occurrences of many fixed patterns in one pass over the text, in time independent of the pattern count, using a trie with failure links.
›
Format-preserving encryption
security · low: Encrypt a digit or character string into another string of the same length and alphabet with a keyed Feistel network, reversible with the key and a context tweak, without a lookup table.
›
Message authentication codes (HMAC,…
security · low: A keyed short tag proving a message is unmodified and came from a key holder; pick HMAC, CMAC, KMAC or a one-time polynomial MAC by what primitive you already have.
›
Cryptographic hashes (SHA-2, BLAKE3)
security · low: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.
›
Bloom filter
data-structures · low: Bit array answering "definitely not present" or "probably present" in about 10 bits per key for 1 % false positives; inserts are cheap, deletes impossible.
›
Hdr histogram
data-structures · low: Log-linear bucketed histograms with bounded relative error for latency percentiles
Luhn check
data-structures · low: Luhn mod-10 check digit validation for card numbers and IDs
Process supervision
hardware-os · low: A parent process starts services, reaps their exits and restarts them under a delay and burst-limit policy, then stops them with SIGTERM before SIGKILL.
›
PDF text extraction
machine-learning · low: Rebuild words, lines, columns and reading order from the glyph positions a born-digital PDF stores, and route pages without usable text to OCR.
›
Unicode text tokenization for search
search-sort · low: Turn multilingual text into index terms: segment words (UAX 29 style, CJK bigrams), normalize, case-fold and accent-fold, identically at index and query time.
›
Mutual TLS
security · low: TLS where the client also proves a certificate key, so every connection carries a verified, rotatable machine identity.
›
Graceful draining
systems · low: On a termination signal, stop taking new work, finish in-flight work until a deadline, tell clients to reconnect elsewhere, then exit; clients resume from a token.
›
Cooperative scheduling
concurrency · low: Green threads/coroutines yielding at await points on a user-space scheduler
Record replay
distributed · low: Record every nondeterministic input (time, randomness, I/O, model and tool outputs, per-tick player input) and re-execute deterministically for debugging, replays and state rebuild
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
+ 13 more techniques
+ 54 more primitives
System
Techniques · 20
Primitives · 70
dlp
Deterministic simulation testing
distributed · medium: Run the whole system in one thread under a seeded scheduler with simulated network, disk and clock, so every failure replays from its seed.
›
Primary-backup replication
distributed · medium: One node orders all writes and ships them to standbys; the acknowledgement rule (none, one, all) sets what a crash loses, and an external arbiter decides who takes over.
›
Named entity recognition
machine-learning · medium: Find the character spans of entities in text and give each a type, by labelling every token and decoding a valid label sequence.
›
Near-duplicate detection with MinHa…
machine-learning · medium: Estimate set Jaccard similarity from k min-hashes and find similar pairs with banded LSH buckets; the standard dedup for web crawls, training corpora and sequence collections.
›
Optical character recognition (text…
machine-learning · medium: Find text lines in an image, read each crop with a CTC or attention model, then correct with a lexicon and score confidence for review routing.
›
Text classification
machine-learning · medium: Assign one or more labels to a text; climb a ladder from regexes to linear models on n-grams, embeddings, fine-tuned encoders and LLM prompts, only as far as the data and budget demand.
›
Linear-time regular expression matc…
search-sort · medium: Match a regular expression in O(pattern x text) worst case by tracking all live parse positions at once, then cache the state sets as a lazily built DFA for speed.
›
Coverage-guided fuzzing
security · medium: Mutate inputs, run an instrumented target, and keep any input that reaches a new code edge; feedback turns blind guessing into incremental search.
›
Data masking
security · medium: Hide sensitive field values from people and systems that don't need them, by masked copies or role-based masking at read time, while keeping joins and formats working.
›
Tls termination
security · medium: Terminate TLS at a proxy: SNI cert selection, session resumption, OCSP, kTLS offload (listed in Moved to the wiki)
Point-in-time recovery
storage · medium: Periodic base backups plus a continuously archived change log let you rebuild the state as of any instant, then branch from there.
›
Delta debugging
compilers · medium: Ddmin minimization of failing inputs or change sets
Fault injection
distributed · medium: Inject latency, errors, kills and partitions via proxies, netem or hooks
Acme protocol
security · medium: Automated TLS certificate issuance and renewal via ACME HTTP-01/DNS-01 challenges
Key rotation
security · medium: Rotating keys without downtime: key ids, overlap windows, re-wrapping, JWKS publication, revocation
TLS 1.3
security · medium: The standard encrypted, authenticated transport; one round trip, only forward-secret AEAD suites, and little left to misconfigure.
›
ARIES recovery
storage · medium: Three-pass crash recovery (analysis, redo, undo) for steal/no-force buffer pools; repeats history, then rolls back losers with compensation log records.
›
Envelope encryption
security · medium: Encrypt each object with its own data key and wrap that key under a KMS-held key, so rotation re-wraps 60 bytes and deleting a key shreds data.
›
Syscall interception
security · medium: Intercept syscalls via ptrace, seccomp user notify or a user-space kernel
Buffer pool
storage · medium: The database's own page cache (frames, page table, pin counts, dirty bits, replacement) so it, not the OS, controls eviction, write order and I/O.
›
Rolling hash
data-structures · low: Rabin-Karp polynomial rolling hashes and winnowing for substring fingerprints and document matching
Aho-Corasick multi-pattern matching
search-sort · low: Find all occurrences of many fixed patterns in one pass over the text, in time independent of the pattern count, using a trie with failure links.
›
Format-preserving encryption
security · low: Encrypt a digit or character string into another string of the same length and alphabet with a keyed Feistel network, reversible with the key and a context tweak, without a lookup table.
›
Message authentication codes (HMAC,…
security · low: A keyed short tag proving a message is unmodified and came from a key holder; pick HMAC, CMAC, KMAC or a one-time polynomial MAC by what primitive you already have.
›
Cryptographic hashes (SHA-2, BLAKE3)
security · low: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.
›
Bloom filter
data-structures · low: Bit array answering "definitely not present" or "probably present" in about 10 bits per key for 1 % false positives; inserts are cheap, deletes impossible.
›
Hdr histogram
data-structures · low: Log-linear bucketed histograms with bounded relative error for latency percentiles
Luhn check
data-structures · low: Luhn mod-10 check digit validation for card numbers and IDs
Process supervision
hardware-os · low: A parent process starts services, reaps their exits and restarts them under a delay and burst-limit policy, then stops them with SIGTERM before SIGKILL.
›
PDF text extraction
machine-learning · low: Rebuild words, lines, columns and reading order from the glyph positions a born-digital PDF stores, and route pages without usable text to OCR.
›
Unicode text tokenization for search
search-sort · low: Turn multilingual text into index terms: segment words (UAX 29 style, CJK bigrams), normalize, case-fold and accent-fold, identically at index and query time.
›
Mutual TLS
security · low: TLS where the client also proves a certificate key, so every connection carries a verified, rotatable machine identity.
›
Graceful draining
systems · low: On a termination signal, stop taking new work, finish in-flight work until a deadline, tell clients to reconnect elsewhere, then exit; clients resume from a token.
›
Cooperative scheduling
concurrency · low: Green threads/coroutines yielding at await points on a user-space scheduler
Record replay
distributed · low: Record every nondeterministic input (time, randomness, I/O, model and tool outputs, per-tick player input) and re-execute deterministically for debugging, replays and state rebuild
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Leases
distributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.
›
Write-ahead log
storage · low: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.
›
Health checking
systems · low: Decide per backend whether to route to it using probes and real-traffic errors, with consecutive-count hysteresis, capped ejection and a panic fallback.
›
Byte-pair encoding (BPE) tokenizati…
machine-learning · low: Learn a subword vocabulary by repeatedly merging the most frequent adjacent pair; byte-level BPE encodes any input losslessly with 32k-200k tokens.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
Convolutional codes and the Viterbi…
signal-processing · low: Streaming error-correcting code: each output bit is a parity of the last K input bits; the Viterbi decoder finds the most likely path through a 2^(K-1)-state trellis.
›
Fast hash functions
data-structures · low: Non-cryptographic hashes built from 64×64→128-bit multiplies and xor-shifts; pick by key size, trust level and whether the hash is persisted.
›
Swiss table
data-structures · low: Open-addressing hash table that filters 8-16 slots at once with 1-byte hash tags and SIMD, giving fast lookups at 87.5 % load.
›
Connectionist temporal classificati…
machine-learning · low: Alignment-free sequence loss that sums the probability of every frame-to-label alignment with a blank symbol, using a dynamic program.
›
Document image preprocessing
machine-learning · low: Turn a scan or phone photo into an upright, evenly lit, binarised or normalised grey page so layout analysis and OCR read it reliably.
›
Convolutional networks
ml-training · low: Layers that slide a small shared kernel over a grid, giving translation equivariance and few parameters; still the most compute-efficient choice for many vision tasks.
›
Calibration
machine-learning · low: Make predicted probabilities match observed frequencies: measure with ECE, fix post hoc with temperature, Platt or isotonic scaling on held-out data.
›
Linear and logistic regression
machine-learning · low: Fit a weighted sum of features to a number (least squares) or a probability (logistic); the baseline every model must beat.
›
Recursive-descent parsing
compilers · low: Hand-written top-down parser with one function per grammar rule; simple, fast, with full control over error messages and recovery.
›
SIMD substring search
search-sort · low: Find bytes or substrings by comparing 16-64 haystack positions per instruction against the needle's first and last byte, then verifying the rare candidates.
›
LRU and CLOCK
systems · low: LRU evicts the least recently used entry via hash map plus linked list; CLOCK approximates it with a reference bit, so hits take no lock.
›
Code coverage instrumentation
compilers · low: Insert counters into compiled code to record which blocks, edges or lines ran; a spanning tree of the control-flow graph cuts the counters needed.
›
Sanitizers
security · low: ASan/UBSan/TSan shadow memory and instrumentation to catch memory bugs
X509 certificates
security · low: Certificate chains, path validation, SANs, revocation and short-lived issuance
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Copy on write snapshots
storage · low: Point-in-time snapshots by sharing pages/extents and copying only on write
Failure bisection
search-sort · low: Binary search over commits, batches or inputs to isolate a failure
Transparent interception
networking · low: Redirect traffic to a local proxy (iptables TPROXY/REDIRECT, eBPF) without app changes
DNS message format and client looku…
networking · low: Binary query/response format of DNS (RFC 1035) and how a stub client uses it: record types, name compression, UDP/TCP/EDNS sizes, TTL caching.
›
HTTP/1.1 message parsing
networking · low: Parse HTTP/1.1 requests and responses incrementally from a byte stream: RFC 9112 body-length rules, chunked decoding and the strictness that stops request smuggling.
›
ECDSA signatures
security · low: The elliptic-curve signature standardised in FIPS 186-5 (P-256) and used by secp256k1 ledgers; 64-byte signatures, but a repeated or biased nonce leaks the private key.
›
JSON Web Token (JWT)
security · low: A signed, URL-safe token carrying claims; the verifier's job is the checklist (pinned algorithm, key by kid, exp, aud, iss), not the format.
›
Secrets management
security · low: Keep keys and passwords out of code and logs, give workloads short-lived identity instead of static keys, and make rotation routine.
›
Cron scheduling
systems · low: Cron expression parsing and next-fire-time computation, with catch-up and time-zone rules
AEAD (AES-GCM, ChaCha20-Poly1305)
security · low: Authenticated encryption that encrypts and detects tampering in one call; the only kind of symmetric encryption you should use.
›
ECDH key exchange (X25519, P-256)
security · low: Each side multiplies the other's public curve point by its own scalar and both get the same point; always validate the peer key and run the result through a KDF.
›
HKDF key derivation (extract-and-ex…
security · low: HMAC-based two-step key derivation: extract a uniform pseudorandom key from any secret, then expand it into as many context-bound keys as you need.
›
Aes key wrap
security · low: AES Key Wrap (RFC 3394/5649) and AES-GCM-SIV for deterministic, authenticated wrapping of key material
Csprng
security · low: OS-backed cryptographically secure random numbers (getrandom), and when a fast PRNG is not enough
Ebpf
hardware-os · low: Verified in-kernel programs on hooks (kprobes, XDP, tc) with maps
Ptrace process control
hardware-os · low: Ptrace: attach, read and write registers and memory, single-step, intercept signals
Seccomp bpf
security · low: Syscall filtering with seccomp BPF programs
fsync, O_DSYNC and O_DIRECT
hardware-os · low: What fsync, fdatasync, O_DSYNC, O_DIRECT and F_FULLFSYNC really guarantee on Linux and macOS, and the exact call sequences for crash-safe files.
›
S3-FIFO
systems · low: Three FIFO queues (small, main, ghost) that evict one-hit wonders quickly and keep reused objects; low miss ratios with no list moves on hits.
›
Control-flow graph
compilers · low: Split a linear instruction list into basic blocks and connect them by jumps; the graph every optimizer, SSA builder and dataflow pass walks.
›
d-ary heap
data-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.
›
Fencing tokens
distributed · low: A number that grows with every lock grant and is checked by the protected resource, so a paused ex-holder's late write is rejected.
›
BFS and DFS
graphs · low: The two basic graph traversals; BFS gives hop distances and levels, DFS gives discovery/finish order for cycles, components and topological order.
›
Minimum spanning tree
graphs · low: Cheapest set of edges connecting all vertices; Kruskal sorts edges and joins components with union-find, Prim grows one tree, Boruvka merges in parallel rounds.
›
Union-find
graphs · low: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.
›
CRC32C checksums
hardware-os · low: The Castagnoli CRC-32 with dedicated instructions on x86 (SSE4.2) and ARMv8. It is the standard fast integrity check for storage blocks, logs and network frames.
›
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Text normalization
machine-learning · low: Rewrite messy text into one canonical form for matchers and models, or into spoken words for speech output (and back), keeping a map to the original offsets.
›
L-BFGS
numerics · low: Quasi-Newton minimizer that approximates the inverse Hessian from the last m gradient differences via the two-loop recursion; O(m·n) memory and work per step.
›
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Sparse matrices (CSR)
numerics · low: Compressed sparse row stores only nonzeros as row pointers, column indices and values; SpMV streams them once, so it is memory-bandwidth bound.
›
Constant time code
security · low: Writing secret-dependent code without branches or memory lookups on secrets; constant-time compare and select
OAuth 2 and OpenID Connect
security · low: Delegated authorization (OAuth 2) and login on top of it (OIDC); use authorization code + PKCE for every user-facing client.
›
Copy for AI
Similar systems
Share the most building blocks with this one.
llm-dlp
50 %
card-vault
35 %
legal-research
33 %
background-screening
32 %
customer-data-platform
29 %