← Back
Home › Systems › Data processing and pipelines

identity-graph

identity resolution and data onboarding for ads

LiveRamp, Epsilon, Acxiom

SystemTechniques · 17Primitives · 57 identity-graph Mapreducedistributed · medium: Map, partition, sort/shuffle, reduce with combiners and re-execution of failed tasksAddress parsinggeometry · medium: Split free-text addresses into components (house, street, city, postcode) with rules or a sequence modelGraph community detectiongraphs · medium: Louvain/label propagation community detection for rings, fraud clusters and campaignsRecord linkagesearch-sort · medium: Probabilistic record matching (Fellegi-Sunter) with blocking for patient or entity indexesData maskingsecurity · medium: Hide sensitive field values from people and systems that don't need them, by masked copies or role-based masking at read time, while keeping joins and formats working.›Envelope encryptionsecurity · medium: Encrypt each object with its own data key and wrap that key under a KMS-held key, so rotation re-wraps 60 bytes and deleting a key shreds data.›Private set intersectionsecurity · medium: Compute the overlap of two parties' sets without revealing the rest (ECDH-PSI, OPRF)Union-findgraphs · low: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.›Fuzzy string matchingsearch-sort · low: Edit distance, Jaro-Winkler and trigram indexes for approximate name and address matchingMessage authentication codes (HMAC,…security · low: A keyed short tag proving a message is unmodified and came from a key holder; pick HMAC, CMAC, KMAC or a one-time polynomial MAC by what primitive you already have.›Bloom filterdata-structures · low: Bit array answering "definitely not present" or "probably present" in about 10 bits per key for 1 % false positives; inserts are cheap, deletes impossible.›E164 number parsingdata-structures · low: Parse, validate and normalize phone numbers to E.164Consent signal propagationsecurity · low: Carry user consent choices as encoded strings (TCF, GPP, GPC) with each request and enforce them per purpose and vendorCryptographic hashes (SHA-2, BLAKE3)security · low: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.›Id syncingsecurity · low: Cookie syncing: redirect chains between ad-tech domains exchange user IDs to build id-mapping tables across partiesK anonymitysecurity · low: Minimum group-size thresholds and suppression so aggregates cannot single out a personObject storagecloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.›Speculative executiondistributed · low: Launch backup copies of straggler tasks and keep the first resultText normalizationmachine-learning · low: Rewrite messy text into one canonical form for matchers and models, or into spoken words for speech output (and back), keeping a map to the original offsets.›Unicode text tokenization for searchsearch-sort · low: Turn multilingual text into index terms: segment words (UAX 29 style, CJK bigrams), normalize, case-fold and accent-fold, identically at index and query time.›PRNGs (PCG, xoshiro)numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.›Sparse matrices (CSR)numerics · low: Compressed sparse row stores only nonzeros as row pointers, column indices and values; SpMV streams them once, so it is memory-bandwidth bound.›Linear and logistic regressionmachine-learning · low: Fit a weighted sum of features to a number (least squares) or a probability (logistic); the baseline every model must beat.›+ 10 more techniques+ 41 more primitives

Similar systems

Share the most building blocks with this one.

background-screening 20 %master-data-management 18 %data-clean-room 15 %