mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
Finance and trading
alternative-data
alternative data turned into investable signals
YipitData, M Science, Bloomberg Second Measure
System
Techniques · 19
Primitives · 61
alternative-data
Density-based clustering (DBSCAN)
machine-learning · medium: Group points that are packed densely, label the rest as noise: no k, arbitrary shapes, two parameters (eps, minPts), cost set by the neighbour index.
›
HTML main-text extraction
machine-learning · medium: Decode bytes, tolerate broken markup, drop non-content nodes and score text blocks by length and link density to keep the main run, with headings, metadata and offsets for citation.
›
Linear-time regular expression matc…
search-sort · medium: Match a regular expression in O(pattern x text) worst case by tracking all live parse positions at once, then cache the state sets as a lazily built DFA for speed.
›
Url frontier
search-sort · medium: Prioritised per-host queues with politeness delays for crawling
Data masking
security · medium: Hide sensitive field values from people and systems that don't need them, by masked copies or role-based masking at read time, while keeping joins and formats working.
›
Change point detection
statistics · medium: CUSUM/E-divisive/PELT to find shifts in a time series mean or distribution
Survey weighting raking
statistics · medium: Raking (iterative proportional fitting) of survey weights to population margins
STL decomposition (seasonal-trend w…
statistics · low: Split a series into trend, seasonal and remainder with iterated loess smoothing; the seasonal shape may drift and robust weights keep outliers out of it.
›
Time-series cross-validation (rolli…
statistics · low: Estimate forecast error by repeatedly training on the past and testing on the next h steps, moving the origin forward; never trains on the future.
›
Fuzzy string matching
search-sort · low: Edit distance, Jaro-Winkler and trigram indexes for approximate name and address matching
Kalman filter
robotics · low: Optimal recursive estimator for a linear system with Gaussian noise: fuses a motion model with noisy measurements into a state estimate plus its covariance.
›
Browser automation protocol
agentic-ai · low: CDP / WebDriver BiDi commands and events over websockets to drive browsers
Security master symbology
data-structures · low: Cross-reference instrument identifiers (ISIN, CUSIP, SEDOL, FIGI, tickers) with effective-dated mappings and listing-level resolution
Geospatial cell index (geohash, S2,…
geometry · low: Map a lat/lon to a 64-bit cell id on a hierarchical grid, so "near" becomes an integer range scan in any ordinary sorted index.
›
Point in polygon
geometry · low: Ray-casting/winding tests and indexed polygon lookup for reverse geocoding
K anonymity
security · low: Minimum group-size thresholds and suppression so aggregates cannot single out a person
k-d tree
geometry · low: A binary tree that splits k-dimensional points by one coordinate per level, giving fast nearest-neighbour and box queries on static low-dimensional data.
›
Union-find
graphs · low: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.
›
Text normalization
machine-learning · low: Rewrite messy text into one canonical form for matchers and models, or into spoken words for speech output (and back), keeping a map to the original offsets.
›
Unicode text tokenization for search
search-sort · low: Turn multilingual text into index terms: segment words (UAX 29 style, CJK bigrams), normalize, case-fold and accent-fold, identically at index and query time.
›
Recursive-descent parsing
compilers · low: Hand-written top-down parser with one function per grammar rule; simple, fast, with full control over error messages and recovery.
›
Aho-Corasick multi-pattern matching
search-sort · low: Find all occurrences of many fixed patterns in one pass over the text, in time independent of the pattern count, using a trie with failure links.
›
SIMD substring search
search-sort · low: Find bytes or substrings by comparing 16-64 haystack positions per instruction against the needle's first and last byte, then verifying the rare candidates.
›
+ 12 more techniques
+ 45 more primitives
System
Techniques · 19
Primitives · 61
alternative-data
Density-based clustering (DBSCAN)
machine-learning · medium: Group points that are packed densely, label the rest as noise: no k, arbitrary shapes, two parameters (eps, minPts), cost set by the neighbour index.
›
HTML main-text extraction
machine-learning · medium: Decode bytes, tolerate broken markup, drop non-content nodes and score text blocks by length and link density to keep the main run, with headings, metadata and offsets for citation.
›
Linear-time regular expression matc…
search-sort · medium: Match a regular expression in O(pattern x text) worst case by tracking all live parse positions at once, then cache the state sets as a lazily built DFA for speed.
›
Url frontier
search-sort · medium: Prioritised per-host queues with politeness delays for crawling
Data masking
security · medium: Hide sensitive field values from people and systems that don't need them, by masked copies or role-based masking at read time, while keeping joins and formats working.
›
Change point detection
statistics · medium: CUSUM/E-divisive/PELT to find shifts in a time series mean or distribution
Survey weighting raking
statistics · medium: Raking (iterative proportional fitting) of survey weights to population margins
Time-series forecasting (baselines,…
statistics · medium: The end-to-end recipe: fix horizon and uncertainty, beat naive baselines, choose a model family, and validate with rolling-origin backtests and scale-free metrics.
›
Bitemporal data
storage · medium: Valid time plus transaction time on every record for as-of queries, back-dating and corrections
Columnar storage
storage · medium: Store each column contiguously so analytic scans read only the columns they need, compress well and run as tight SIMD loops.
›
Entity resolution
storage · medium: Decide which records describe the same real-world thing: block to cut the pair space, score pairs with a probabilistic model, cluster the matches transitively.
›
Html parsing
graphics · medium: WHATWG HTML tokenizer state machine and tree construction with error recovery
Record linkage
search-sort · medium: Probabilistic record matching (Fellegi-Sunter) with blocking for patient or entity indexes
Arima
statistics · medium: ARIMA/SARIMA forecasting: differencing, AR and MA terms, order selection, maximum-likelihood fit
Inverted index
storage · medium: Map each term to a sorted, compressed list of the documents containing it; the core of full-text search and other fast set queries.
›
MVCC
storage · medium: Keep timestamped versions of each row so readers see a consistent snapshot without blocking writers; the default concurrency control in modern databases.
›
Vectorized execution
storage · medium: Query operators pass batches of about 1-2K values per column instead of single tuples, amortizing interpretation overhead and enabling tight SIMD loops.
›
Parser error recovery
compilers · medium: Recover from syntax errors via synchronization tokens and error nodes to keep a full tree
Near-duplicate detection with MinHa…
machine-learning · medium: Estimate set Jaccard similarity from k min-hashes and find similar pairs with banded LSH buckets; the standard dedup for web crawls, training corpora and sequence collections.
›
STL decomposition (seasonal-trend w…
statistics · low: Split a series into trend, seasonal and remainder with iterated loess smoothing; the seasonal shape may drift and robust weights keep outliers out of it.
›
Time-series cross-validation (rolli…
statistics · low: Estimate forecast error by repeatedly training on the past and testing on the next h steps, moving the origin forward; never trains on the future.
›
Fuzzy string matching
search-sort · low: Edit distance, Jaro-Winkler and trigram indexes for approximate name and address matching
Kalman filter
robotics · low: Optimal recursive estimator for a linear system with Gaussian noise: fuses a motion model with noisy measurements into a state estimate plus its covariance.
›
Browser automation protocol
agentic-ai · low: CDP / WebDriver BiDi commands and events over websockets to drive browsers
Security master symbology
data-structures · low: Cross-reference instrument identifiers (ISIN, CUSIP, SEDOL, FIGI, tickers) with effective-dated mappings and listing-level resolution
Geospatial cell index (geohash, S2,…
geometry · low: Map a lat/lon to a 64-bit cell id on a hierarchical grid, so "near" becomes an integer range scan in any ordinary sorted index.
›
Point in polygon
geometry · low: Ray-casting/winding tests and indexed polygon lookup for reverse geocoding
K anonymity
security · low: Minimum group-size thresholds and suppression so aggregates cannot single out a person
k-d tree
geometry · low: A binary tree that splits k-dimensional points by one coordinate per level, giving fast nearest-neighbour and box queries on static low-dimensional data.
›
Union-find
graphs · low: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.
›
Text normalization
machine-learning · low: Rewrite messy text into one canonical form for matchers and models, or into spoken words for speech output (and back), keeping a map to the original offsets.
›
Unicode text tokenization for search
search-sort · low: Turn multilingual text into index terms: segment words (UAX 29 style, CJK bigrams), normalize, case-fold and accent-fold, identically at index and query time.
›
Recursive-descent parsing
compilers · low: Hand-written top-down parser with one function per grammar rule; simple, fast, with full control over error messages and recovery.
›
Aho-Corasick multi-pattern matching
search-sort · low: Find all occurrences of many fixed patterns in one pass over the text, in time independent of the pattern count, using a trie with failure links.
›
SIMD substring search
search-sort · low: Find bytes or substrings by comparing 16-64 haystack positions per instruction against the needle's first and last byte, then verifying the rare candidates.
›
LRU and CLOCK
systems · low: LRU evicts the least recently used entry via hash map plus linked list; CLOCK approximates it with a reference bit, so hits take no lock.
›
Bloom filter
data-structures · low: Bit array answering "definitely not present" or "probably present" in about 10 bits per key for 1 % false positives; inserts are cheap, deletes impossible.
›
d-ary heap
data-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.
›
Robots exclusion protocol
search-sort · low: Decide per URL whether a crawler may fetch it from a host's robots.txt (RFC 9309): status-code rules, group choice, longest-match Allow/Disallow, 24 h cache.
›
URL normalization
search-sort · low: Rewrite a URL to one canonical key, in safe RFC 3986 steps plus lossy, site-dependent ones (tracking parameters, query order, fragments), and keep the original beside the key.
›
Token bucket
systems · low: A bucket of up to b tokens refilled at rate r; requests spend tokens, enforcing an average rate while allowing bursts of b.
›
Format-preserving encryption
security · low: Encrypt a digit or character string into another string of the same length and alphabet with a keyed Feistel network, reversible with the key and a context tweak, without a lookup table.
›
Message authentication codes (HMAC,…
security · low: A keyed short tag proving a message is unmodified and came from a key holder; pick HMAC, CMAC, KMAC or a one-time polynomial MAC by what primitive you already have.
›
Bayesian rate estimation
statistics · low: Estimate a conversion or event rate from few observations with a Beta-Binomial or Gamma-Poisson update: two counters, an honest interval, and P(B beats A) directly.
›
Robust statistics
statistics · low: Location and scale estimates (median, MAD, trimmed mean, Huber M-estimator) that stay correct when a minority of the data is garbage.
›
Newton root finding
numerics · low: Linearize at the guess and jump to the tangent root; quadratic convergence near a simple root, made robust by brackets (1D) or line search.
›
Hash aggregation
storage · low: GROUP BY via hash tables with partitioning and spill to disk
Croston method
statistics · low: Forecasting intermittent demand by separately smoothing demand sizes and intervals (Croston, SBA, TSB)
Exponential smoothing (ETS, Holt-Wi…
statistics · low: Forecast by keeping a few running states (level, trend, seasonal) updated by a fraction of each forecast error; O(1) memory per series and a strong baseline.
›
Interval tree
data-structures · low: Index of ranges that answers "which intervals overlap this point or range?" in O(log n + k); static sorted-array form is the compact, cache-friendly choice.
›
Time partitioning
storage · low: Cut a table into fixed-interval time blocks so queries read only overlapping blocks and retention drops a whole block as one metadata change.
›
Bit packing and PFOR
compression · low: Store blocks of integers with exactly b bits each and patch rare outliers separately (PFOR); the fastest integer codec family, billions of values per second.
›
Delta encoding
compression · low: Store differences between consecutive values instead of the values, turning sorted IDs, timestamps and counters into small numbers that pack into few bits.
›
Dictionary encoding
compression · low: Map values to small integer codes via a dictionary for compact, comparable columns
Zone maps
storage · low: Per-block min/max (and sparse primary) indexes to skip data during scans
Embeddings
machine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.
›
Radix trie
data-structures · low: Compressed prefix tree for path routing, domain matching and prefix lookups
String interning
data-structures · low: Store each distinct string once and refer to it by id
Linear and logistic regression
machine-learning · low: Fit a weighted sum of features to a number (least squares) or a probability (logistic); the baseline every model must beat.
›
Finite differences
numerics · low: Replace derivatives with weighted differences of grid values; simple stencils plus a stability condition (CFL) turn PDEs like heat, advection and waves into array loops.
›
L-BFGS
numerics · low: Quasi-Newton minimizer that approximates the inverse Hessian from the last m gradient differences via the two-loop recursion; O(m·n) memory and work per step.
›
Varint (LEB128)
compression · low: Variable-length integers in 7-bit groups with a continuation bit; small numbers take 1 byte, the wire format of Protobuf, WebAssembly and DWARF.
›
BM25 ranking
search-sort · low: The standard lexical relevance score: IDF-weighted term frequency with saturation (k1) and length normalization (b); default in Lucene, Elasticsearch and SQLite FTS5.
›
Sorted-list intersection
search-sort · low: Intersect sorted id lists by merging, galloping from the short list into the long one, and skipping whole compressed blocks, shortest list first.
›
Edit distance
search-sort · low: Levenshtein/Damerau distance by DP, bit-parallel Myers variant
Fast hash functions
data-structures · low: Non-cryptographic hashes built from 64×64→128-bit multiplies and xor-shifts; pick by key size, trust level and whether the hash is persisted.
›
Rolling hash
data-structures · low: Rabin-Karp polynomial rolling hashes and winnowing for substring fingerprints and document matching
Swiss table
data-structures · low: Open-addressing hash table that filters 8-16 slots at once with 1-byte hash tags and SIMD, giving fast lookups at 87.5 % load.
›
Quadtree octree
geometry · low: Adaptive spatial subdivision trees for queries and level of detail; loose and linear variants
Space filling curves
geometry · low: Z-order/Morton (geohash) and Hilbert curves mapping multi-dimensional points to sortable keys
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
Evaluation metrics
machine-learning · low: Precision, recall, ROC AUC, average precision, log loss, NDCG and pass@k computed correctly, plus paired confidence intervals for comparing models.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Reverse-mode automatic differentiat…
ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.
›
Explicit Euler
numerics · low: Step an ODE forward by its current slope, y += h·f(t, y); first-order, one evaluation per step, the baseline every other integrator is measured against.
›
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Thomas tridiagonal solver
numerics · low: O(n) tridiagonal linear solve used in implicit finite-difference and spline schemes
Top-k selection
search-sort · low: Keep the k best of n items without sorting all n - bounded min-heap for streams, quickselect for arrays, threshold pruning to skip most items.
›
Bootstrap confidence intervals
statistics · low: Resampling-based confidence intervals for arbitrary statistics
Copy for AI