← Back
Home › Systems › Cloud platform and compute

distributed-cron

distributed cron / periodic job scheduler

Kubernetes CronJobs at scale

SystemTechniques · 4Primitives · 22 distributed-cron Clock synchronizationdistributed · medium: Estimate a clock's offset from four timestamps, filter out queueing noise, and steer the clock smoothly; NTP gets about a millisecond, PTP with hardware timestamps under a microsecond.›Durable job queuedistributed · medium: Persisted tasks that workers lease for a visibility timeout, ack on success and retry with backoff; at-least-once execution that survives worker and queue crashes.›Raftdistributed · medium: Leader-based consensus on a replicated log; one leader per term, majority commit; the default for a strongly consistent replicated state machine.›State-machine snapshotsdistributed · medium: Periodically save the state a log has produced at index i so the log prefix can be deleted; recovery and catch-up become load snapshot plus replay suffix.›Leasesdistributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.›Idempotency keysfinance · low: Client-chosen unique key per logical operation, stored with a request fingerprint and the final response, so retries replay the first result instead of acting twice.›Exponential backoff with jittersystems · low: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.›d-ary heapdata-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.›Fencing tokensdistributed · low: A number that grows with every lock grant and is checked by the protected resource, so a paused ex-holder's late write is rejected.›Rendezvous hashingdistributed · low: Score every node with hash(key, node) and pick the highest: minimal key movement and exact weights without a ring, at O(n) per lookup.›Calendar date arithmeticsystems · low: Civil dates, time zones (tzdb), DST gaps, month-end anchoring and duration mathCron schedulingsystems · low: Cron expression parsing and next-fire-time computation, with catch-up and time-zone rulesHierarchical timing wheelsystems · low: Timers bucketed by expiry into wheels of slots, coarser wheels for far deadlines: O(1) insert and cancel for millions of timeouts.›PID controllerrobotics · low: Drive a measured value to a setpoint with three terms on the error: proportional, integral, derivative. The default feedback loop for motors, heaters, drones and gimbals.›Phase-locked loop (digital)signal-processing · low: Feedback loop that locks a local oscillator to the phase and frequency of an input tone; the core of carrier recovery, clock recovery and GNSS tracking.›Robust statisticsstatistics · low: Location and scale estimates (median, MAD, trimmed mean, Huber M-estimator) that stay correct when a minority of the data is garbage.›Dead-letter queuedistributed · low: A side queue for messages that keep failing: after N delivery attempts or a permanent error, park them with the reason so the main queue keeps moving.›Write-ahead logstorage · low: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.›Copy-on-write forkhardware-os · low: fork() gives the child the parent's whole address space by sharing pages read-only and copying one only when someone writes it; vfork and posix_spawn skip even that.›Copy on write snapshotsstorage · low: Point-in-time snapshots by sharing pages/extents and copying only on write+ 6 more primitives

Similar systems

Share the most building blocks with this one.

task-queue 32 %workflow-engine 26 %transcoding-farm 18 %alerting 18 %llm-workflow-automation 18 %