mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
Consumer applications
file-sync
file sync across devices
Dropbox
System
Techniques · 3
Primitives · 24
file-sync
Rsync delta algorithm
compression · medium: Rsync rolling checksum + strong hash to send only changed blocks
Incremental sync cursors
distributed · medium: The client keeps one opaque token; each pull returns only the changes after it plus the next token, so sync cost scales with what changed, not with the dataset.
›
Envelope encryption
security · medium: Encrypt each object with its own data key and wrap that key under a KMS-held key, so rotation re-wraps 60 bytes and deleting a key shreds data.
›
Cryptographic hashes (SHA-2, BLAKE3)
security · low: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.
›
Write-ahead log
storage · low: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Content defined chunking
compression · low: Rolling-hash chunk boundaries (Rabin, FastCDC) for deduplication that survives insertions
Zstandard (zstd)
compression · low: LZ77 plus Huffman and FSE entropy coding, with levels from LZ4-like speed to xz-like ratio, all decoding at about 1-2 GB/s.
›
Merkle tree
data-structures · low: Hash tree for verifying and diffing large datasets with logarithmic proofs
Vector clocks
distributed · low: One counter per node; comparing two vectors tells whether events are ordered or concurrent, which is exactly what conflict detection needs.
›
File system watching
hardware-os · low: Treat OS change notices as hints, debounce them, and rescan and diff a snapshot whenever notices overflow or may be lost.
›
fsync, O_DSYNC and O_DIRECT
hardware-os · low: What fsync, fdatasync, O_DSYNC, O_DIRECT and F_FULLFSYNC really guarantee on Linux and macOS, and the exact call sequences for crash-safe files.
›
Resumable uploads
networking · low: Open an upload session, send the body in offset-addressed chunks, and after any failure ask the server how much it committed and continue from there.
›
Server-sent events
networking · low: One long-lived HTTP response carries text events one way; each event may carry an id, so a reconnecting client names its last one and the server resumes.
›
B+ tree
storage · low: Balanced high-fanout search tree with all values in linked leaves; the default on-disk index for point lookups and range scans.
›
Exponential backoff with jitter
systems · low: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.
›
Rolling hash
data-structures · low: Rabin-Karp polynomial rolling hashes and winnowing for substring fingerprints and document matching
Swiss table
data-structures · low: Open-addressing hash table that filters 8-16 slots at once with 1-byte hash tags and SIMD, giving fast lookups at 87.5 % load.
›
Message sequence numbers
distributed · low: A per-stream counter on every message lets the receiver detect loss, duplicates and reordering, ack cheaply and resume after reconnect with "send from N".
›
+ 8 more primitives
System
Techniques · 3
Primitives · 24
file-sync
Rsync delta algorithm
compression · medium: Rsync rolling checksum + strong hash to send only changed blocks
Incremental sync cursors
distributed · medium: The client keeps one opaque token; each pull returns only the changes after it plus the next token, so sync cost scales with what changed, not with the dataset.
›
Envelope encryption
security · medium: Encrypt each object with its own data key and wrap that key under a KMS-held key, so rotation re-wraps 60 bytes and deleting a key shreds data.
›
Cryptographic hashes (SHA-2, BLAKE3)
security · low: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.
›
Write-ahead log
storage · low: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Content defined chunking
compression · low: Rolling-hash chunk boundaries (Rabin, FastCDC) for deduplication that survives insertions
Zstandard (zstd)
compression · low: LZ77 plus Huffman and FSE entropy coding, with levels from LZ4-like speed to xz-like ratio, all decoding at about 1-2 GB/s.
›
Merkle tree
data-structures · low: Hash tree for verifying and diffing large datasets with logarithmic proofs
Vector clocks
distributed · low: One counter per node; comparing two vectors tells whether events are ordered or concurrent, which is exactly what conflict detection needs.
›
File system watching
hardware-os · low: Treat OS change notices as hints, debounce them, and rescan and diff a snapshot whenever notices overflow or may be lost.
›
fsync, O_DSYNC and O_DIRECT
hardware-os · low: What fsync, fdatasync, O_DSYNC, O_DIRECT and F_FULLFSYNC really guarantee on Linux and macOS, and the exact call sequences for crash-safe files.
›
Resumable uploads
networking · low: Open an upload session, send the body in offset-addressed chunks, and after any failure ask the server how much it committed and continue from there.
›
Server-sent events
networking · low: One long-lived HTTP response carries text events one way; each event may carry an id, so a reconnecting client names its last one and the server resumes.
›
B+ tree
storage · low: Balanced high-fanout search tree with all values in linked leaves; the default on-disk index for point lookups and range scans.
›
Exponential backoff with jitter
systems · low: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.
›
Rolling hash
data-structures · low: Rabin-Karp polynomial rolling hashes and winnowing for substring fingerprints and document matching
Swiss table
data-structures · low: Open-addressing hash table that filters 8-16 slots at once with 1-byte hash tags and SIMD, giving fast lookups at 87.5 % load.
›
Message sequence numbers
distributed · low: A per-stream counter on every message lets the receiver detect loss, duplicates and reordering, ack cheaply and resume after reconnect with "send from N".
›
Cursor pagination
networking · low: Keyset/opaque-cursor pagination for stable incremental reads from APIs and tables
AEAD (AES-GCM, ChaCha20-Poly1305)
security · low: Authenticated encryption that encrypts and detects tampering in one call; the only kind of symmetric encryption you should use.
›
Aes key wrap
security · low: AES Key Wrap (RFC 3394/5649) and AES-GCM-SIV for deterministic, authenticated wrapping of key material
Csprng
security · low: OS-backed cryptographically secure random numbers (getrandom), and when a fast PRNG is not enough
CRC32C checksums
hardware-os · low: The Castagnoli CRC-32 with dedicated instructions on x86 (SSE4.2) and ARMv8. It is the standard fast integrity check for storage blocks, logs and network frames.
›
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Constant time code
security · low: Writing secret-dependent code without branches or memory lookups on secrets; constant-time compare and select
Copy for AI
Similar systems
Share the most building blocks with this one.
backup-dr
29 %
cold-archive
29 %
e2e-cloud-backup
24 %
remote-build-execution
19 %
filesystem
18 %