← Back
Home › Systems › Data processing and pipelines

schema-registry

event schema registry with compatibility checks

Confluent Schema Registry

SystemTechniques · 2Primitives · 7 schema-registry Log compactiondistributed · medium: Keep only the latest record per key in a log, with tombstones and a cleanerSchema evolutionstorage · medium: Compatibility rules (backward/forward/full) for adding, removing and renaming fieldsSegmented logdistributed · low: Split an append-only log into files named by their first offset, write only the newest, and find any record by binary search plus a sparse index.›Protobuf wire formatnetworking · low: Protocol Buffers wire format: tags, varints, length-delimited fields and schema evolutionJSON Schema validationsecurity · low: Compile a JSON Schema once, then check every untrusted document against it, within hard size and depth limits, and report failures as JSON Pointers.›Leasesdistributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.›Cryptographic hashes (SHA-2, BLAKE3)security · low: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.›Swiss tabledata-structures · low: Open-addressing hash table that filters 8-16 slots at once with 1-byte hash tags and SIMD, giving fast lookups at 87.5 % load.›Fencing tokensdistributed · low: A number that grows with every lock grant and is checked by the protected resource, so a paused ex-holder's late write is rejected.›

Similar systems

Share the most building blocks with this one.

cdc 21 %