mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
CI/CD and developer tooling
remote-build-execution
remote build cache and execution
BuildBuddy, REAPI
System
Techniques · 2
Primitives · 26
remote-build-execution
Durable job queue
distributed · medium: Persisted tasks that workers lease for a visibility timeout, ack on success and retry with backoff; at-least-once execution that survives worker and queue crashes.
›
Hermetic execution
systems · medium: Run actions with only declared inputs in a sandbox and fixed environment
Leases
distributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.
›
Cryptographic hashes (SHA-2, BLAKE3)
security · low: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.
›
Content-addressed storage
storage · low: Store each blob under the hash of its bytes: immutable, deduplicated by construction, self-verifying on every read, and composable into Merkle trees.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Zstandard (zstd)
compression · low: LZ77 plus Huffman and FSE entropy coding, with levels from LZ4-like speed to xz-like ratio, all decoding at about 1-2 GB/s.
›
Merkle tree
data-structures · low: Hash tree for verifying and diffing large datasets with logarithmic proofs
Consistent hashing
distributed · low: Hash nodes and keys onto a ring; each key belongs to the next node clockwise, so membership changes move only about 1/n of keys.
›
Fuse userspace filesystem
hardware-os · low: FUSE protocol: kernel forwards VFS requests to a userspace daemon; caching and splice
Grpc protocol
networking · low: GRPC framing over HTTP/2: unary and streaming RPCs, deadlines, status
Power of two choices
networking · low: Pick two random backends and send to the less loaded one; exponentially better balance than random, with no global state and no herding.
›
Resumable uploads
networking · low: Open an upload session, send the body in offset-addressed chunks, and after any failure ask the server how much it committed and continue from there.
›
LRU and CLOCK
systems · low: LRU evicts the least recently used entry via hash map plus linked list; CLOCK approximates it with a reference bit, so hits take no lock.
›
W-TinyLFU
systems · low: An LRU window plus segmented LRU, guarded by a 4-bit Count-Min sketch that admits a newcomer only if it beats the victim's frequency.
›
Dead-letter queue
distributed · low: A side queue for messages that keep failing: after N delivery attempts or a permanent error, park them with the reason so the main queue keeps moving.
›
Idempotency keys
finance · low: Client-chosen unique key per logical operation, stored with a request fingerprint and the final response, so retries replay the first result instead of acting twice.
›
Write-ahead log
storage · low: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.
›
+ 10 more primitives
System
Techniques · 2
Primitives · 26
remote-build-execution
Durable job queue
distributed · medium: Persisted tasks that workers lease for a visibility timeout, ack on success and retry with backoff; at-least-once execution that survives worker and queue crashes.
›
Hermetic execution
systems · medium: Run actions with only declared inputs in a sandbox and fixed environment
Leases
distributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.
›
Cryptographic hashes (SHA-2, BLAKE3)
security · low: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.
›
Content-addressed storage
storage · low: Store each blob under the hash of its bytes: immutable, deduplicated by construction, self-verifying on every read, and composable into Merkle trees.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Zstandard (zstd)
compression · low: LZ77 plus Huffman and FSE entropy coding, with levels from LZ4-like speed to xz-like ratio, all decoding at about 1-2 GB/s.
›
Merkle tree
data-structures · low: Hash tree for verifying and diffing large datasets with logarithmic proofs
Consistent hashing
distributed · low: Hash nodes and keys onto a ring; each key belongs to the next node clockwise, so membership changes move only about 1/n of keys.
›
Fuse userspace filesystem
hardware-os · low: FUSE protocol: kernel forwards VFS requests to a userspace daemon; caching and splice
Grpc protocol
networking · low: GRPC framing over HTTP/2: unary and streaming RPCs, deadlines, status
Power of two choices
networking · low: Pick two random backends and send to the less loaded one; exponentially better balance than random, with no global state and no herding.
›
Resumable uploads
networking · low: Open an upload session, send the body in offset-addressed chunks, and after any failure ask the server how much it committed and continue from there.
›
LRU and CLOCK
systems · low: LRU evicts the least recently used entry via hash map plus linked list; CLOCK approximates it with a reference bit, so hits take no lock.
›
W-TinyLFU
systems · low: An LRU window plus segmented LRU, guarded by a 4-bit Count-Min sketch that admits a newcomer only if it beats the victim's frequency.
›
Dead-letter queue
distributed · low: A side queue for messages that keep failing: after N delivery attempts or a permanent error, park them with the reason so the main queue keeps moving.
›
Idempotency keys
finance · low: Client-chosen unique key per logical operation, stored with a request fingerprint and the final response, so retries replay the first result instead of acting twice.
›
Write-ahead log
storage · low: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.
›
Exponential backoff with jitter
systems · low: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.
›
Cgroups and namespaces
hardware-os · low: Linux namespaces limit what a process can see; cgroup v2 limits and accounts what it may use. Containers, sandboxes and job schedulers are built from the two.
›
Overlay filesystem
hardware-os · low: Union mounts (overlayfs) stacking read-only layers under a writable layer
Merkle dag
data-structures · low: Content-addressed DAG of hashed nodes (git trees/commits) for dedupe and verifiable history
Fencing tokens
distributed · low: A number that grows with every lock grant and is checked by the protected resource, so a paused ex-holder's late write is rejected.
›
Atomic file replacement
hardware-os · low: Write temp file, fsync, rename over target, fsync directory: crash-safe whole-file updates
CRC32C checksums
hardware-os · low: The Castagnoli CRC-32 with dedicated instructions on x86 (SSE4.2) and ARMv8. It is the standard fast integrity check for storage blocks, logs and network frames.
›
fsync, O_DSYNC and O_DIRECT
hardware-os · low: What fsync, fdatasync, O_DSYNC, O_DIRECT and F_FULLFSYNC really guarantee on Linux and macOS, and the exact call sequences for crash-safe files.
›
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Copy for AI
Similar systems
Share the most building blocks with this one.
backup-dr
25 %
monorepo-vcs
25 %
image-snapshotter
22 %
file-sync
19 %
cold-archive
18 %