← Back
Home › Systems › Data processing and pipelines

event-router

content-based event routing between services

Amazon EventBridge, Azure Event Grid, Google Eventarc

SystemTechniques · 20Primitives · 65 event-router Cell based architecturedistributed · medium: Independent full-stack cells with a thin router to bound blast radiusDeterministic simulation testingdistributed · medium: Run the whole system in one thread under a seeded scheduler with simulated network, disk and clock, so every failure replays from its seed.›Primary-backup replicationdistributed · medium: One node orders all writes and ships them to standbys; the acknowledgement rule (none, one, all) sets what a crash loses, and an external arbiter decides who takes over.›Pub/sub fan-outdistributed · medium: Deliver one published message to every subscriber of a topic by sharing one encoded payload and giving each connection a bounded queue with an explicit slow-consumer policy.›Shuffle shardingdistributed · medium: Assign each tenant a random subset of nodes to isolate failuresWebhook deliverynetworking · medium: Reliable server-to-server HTTP callbacks: signed POSTs, 2xx-means-done, retries over days with jitter, endpoint disabling, SSRF guards and idempotent receivers.›Coverage-guided fuzzingsecurity · medium: Mutate inputs, run an instrumented target, and keep any input that reaches a new code edge; feedback turns blind guessing into incremental search.›Topic matching triedata-structures · low: Trie over topic levels with + and # wildcards for subscription matchingDead-letter queuedistributed · low: A side queue for messages that keep failing: after N delivery attempts or a permanent error, park them with the reason so the main queue keeps moving.›Idempotency keysfinance · low: Client-chosen unique key per logical operation, stored with a request fingerprint and the final response, so retries replay the first result instead of acting twice.›Hmac request signingsecurity · low: HMAC canonical-request signatures and presigned URLs (SigV4 style) with expiryExponential backoff with jittersystems · low: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.›JSON Schema validationsecurity · low: Compile a JSON Schema once, then check every untrusted document against it, within hard size and depth limits, and report failures as JSON Pointers.›Schema inferencestorage · low: Infer column types and structure from sample recordsHierarchical timing wheelsystems · low: Timers bucketed by expiry into wheels of slots, coarser wheels for far deadlines: O(1) insert and cancel for millions of timeouts.›Publish subscribe event bussystems · low: In-process or on-box topic bus decoupling producers and consumers with bounded queuesHdr histogramdata-structures · low: Log-linear bucketed histograms with bounded relative error for latency percentilesEvent deduplication windowdistributed · low: Keep the ids of recently accepted events for a time window and drop repeats; memory stays bounded by rotating time buckets instead of per-entry timers.›Segmented logdistributed · low: Split an append-only log into files named by their first offset, write only the newest, and find any record by binary search plus a sparse index.›Process supervisionhardware-os · low: A parent process starts services, reaps their exits and restarts them under a delay and burst-limit policy, then stops them with SIGTERM before SIGKILL.›Boolean expression indexingsearch-sort · low: Indexing conjunctive targeting expressions so one event finds all matching ads/subscriptions fastMutual TLSsecurity · low: TLS where the client also proves a certificate key, so every connection carries a verified, rotatable machine identity.›OAuth 2 and OpenID Connectsecurity · low: Delegated authorization (OAuth 2) and login on top of it (OIDC); use authorization code + PKCE for every user-facing client.›+ 13 more techniques+ 49 more primitives

Similar systems

Share the most building blocks with this one.

ipaas 51 %customer-data-platform 44 %llm-workflow-automation 42 %reverse-etl 41 %ingestion 35 %