← Back
Home › Systems › Payments and commerce

product-search

product search and ranking

Amazon search

SystemTechniques · 13Primitives · 41 product-search Gradient-boosted decision trees (GB…machine-learning · medium: Add small regression trees one at a time, each fitted to the loss gradient; histogram splits over 256 bins make it the strongest tabular default.›Learning to rankmachine-learning · medium: Train a scorer on query-grouped data so that the order it induces maximizes NDCG; LambdaMART (boosted trees with NDCG-weighted pair gradients) is the production default.›Two-tower retrievalmachine-learning · medium: Two separate encoders map users/queries and items into one vector space; item vectors are precomputed and searched by nearest neighbour, so recall over millions costs one small forward pass.›Top-k pruning with score upper boun…search-sort · medium: Skip documents that provably cannot enter the top-k, using per-list and per-block maximum scores against the current k-th best score (WAND, Block-Max WAND, MaxScore).›HNSWsearch-sort · medium: Hierarchical Navigable Small World: a layered proximity graph searched greedily from a sparse top layer; the default in-memory approximate nearest-neighbour index.›A/B testing (online controlled expe…statistics · medium: Randomly split units into arms by a salted hash, log exposure, compare metrics with a confidence interval, and guard the result with sample-ratio, guardrail and peeking checks.›Change data capturestorage · medium: Decode the database's own commit log into row-change events in commit order, with a resume position, so any consumer can mirror the data exactly.›Evaluation metricsmachine-learning · low: Precision, recall, ROC AUC, average precision, log loss, NDCG and pass@k computed correctly, plus paired confidence intervals for comparing models.›Embeddingsmachine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.›Bit packing and PFORcompression · low: Store blocks of integers with exactly b bits each and patch rare outliers separately (PFOR); the fastest integer codec family, billions of values per second.›BM25 rankingsearch-sort · low: The standard lexical relevance score: IDF-weighted term frequency with saturation (k1) and length normalization (b); default in Lucene, Elasticsearch and SQLite FTS5.›Top-k selectionsearch-sort · low: Keep the k best of n items without sorting all n - bounded min-heap for streams, quickselect for arrays, threshold pruning to skip most items.›Roaring bitmapscompression · low: Compressed set of 32-bit integers in 64K-value chunks, each a sorted array, bitmap or run list; fast intersections, the standard for bitmap indexes.›Hybrid search and rank fusionsearch-sort · low: Merge a keyword (BM25) result list and a vector (ANN) list into one ranking, by ranks (RRF) or by normalized scores (convex combination).›Levenshtein automatasearch-sort · low: Levenshtein automata / edit-distance search over dictionaries for typo toleranceUnicode text tokenization for searchsearch-sort · low: Turn multilingual text into index terms: segment words (UAX 29 style, CJK bigrams), normalize, case-fold and accent-fold, identically at index and query time.›Hedged requestssystems · low: Sending a backup request after a latency percentile and taking the first answer to cut tail latencyW-TinyLFUsystems · low: An LRU window plus segmented LRU, guarded by a 4-bit Count-Min sketch that admits a newcomer only if it beats the victim's frequency.›Quantile sketchesdata-structures · low: Mergeable quantile sketches: t-digest, DDSketch, HDR histogramsLoss functionsmachine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.›Regularizationmachine-learning · low: Techniques that trade training fit for generalization: weight decay, dropout, early stopping, label smoothing and data mixing, with the exact update rules and defaults.›Negative samplingmachine-learning · low: Train by contrasting each observed positive with a few sampled negatives instead of normalising over every candidate; the choice of sampler decides bias and quality.›Sorted-list intersectionsearch-sort · low: Intersect sorted id lists by merging, galloping from the short list into the long one, and skipping whole compressed blocks, shortest list first.›+ 6 more techniques+ 25 more primitives

Similar systems

Share the most building blocks with this one.

news-ranking 30 %professional-network 30 %web-search 27 %search-engine 27 %rag 25 %