mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
AI applications
code-completion
inline code completion
Copilot, Cursor Tab
System
Techniques · 13
Primitives · 55
code-completion
Incremental parsing
compilers · medium: Reparse only the region an edit damaged and splice the new subtrees into the old tree, sharing everything else; O(edit) instead of O(file).
›
Language server protocol
compilers · medium: One JSON-RPC conversation over a pipe lets any editor drive any language's analysis server; the cost is keeping a live, edited document in sync and answering before it changes.
›
Knowledge distillation
ml-training · medium: Train a small student to match a large teacher's softened outputs (plus the hard labels) so it keeps most of the teacher's accuracy at a fraction of the cost.
›
A/B testing (online controlled expe…
statistics · medium: Randomly split units into arms by a salted hash, log exposure, compare metrics with a confidence interval, and guard the result with sample-ratio, guardrail and peeking checks.
›
Code symbol indexing
compilers · medium: Extract definitions and references per file, give each symbol a stable string id, and store symbol to occurrences, re-indexing only files whose content hash changed.
›
Lossless syntax trees
compilers · medium: Concrete syntax trees with trivia, green/red node split (Roslyn, rowan) for tools that rewrite code
Parser error recovery
compilers · medium: Recover from syntax errors via synchronization tokens and error nodes to keep a full tree
Token sampling
ml-training · low: Turn a vector of scores into one chosen item with temperature, top-k, top-p, min-p, repetition penalties and a seed.
›
LRU and CLOCK
systems · low: LRU evicts the least recently used entry via hash map plus linked list; CLOCK approximates it with a reference bit, so hits take no lock.
›
Code context retrieval
agentic-ai · unknown:
Constrained decoding
agentic-ai · low: Force LLM output to match a regex, JSON schema or grammar by masking, at every step, the tokens that would leave the language.
›
Prompt caching
agentic-ai · low: Reuse the attention keys/values of a repeated prompt prefix, so an agent's growing context is prefilled once and later turns pay only for the new tokens.
›
Continuous batching
ml-training · low: Schedule LLM inference per iteration, not per request; finished sequences leave and new ones join the running batch every step, which keeps the GPU full.
›
Fill in the middle
ml-training · low: FIM training and prompting: prefix/suffix/middle reordering with sentinel tokens for code infill
Paged KV cache
ml-training · low: Store each sequence's attention keys/values in fixed-size blocks mapped through a block table, like virtual memory pages; near-zero fragmentation and copy-on-write sharing.
›
Speculative decoding
ml-training · low: A cheap draft proposes γ tokens, the big model verifies them in one pass, and rejection sampling keeps the target's exact distribution at 2-3x speed.
›
Token healing
ml-training · low: Backing up the last prompt token and constraining generation to avoid tokenization artifacts at boundaries
Weight quantization
ml-training · low: Store weights as 8/4-bit integers with per-group scales; decode is memory-bound, so 4-bit weights give about 3-4x tokens/s and fit 4x larger models.
›
Debouncing throttling
systems · low: Coalescing rapid events: debounce (wait for quiet), throttle (max rate), and cancelling stale in-flight requests
Recursive-descent parsing
compilers · low: Hand-written top-down parser with one function per grammar rule; simple, fast, with full control over error messages and recovery.
›
Rope
graphics · low: A balanced tree of small string chunks with cached byte and newline counts, so seek, insert, delete and snapshot cost O(log n) instead of O(n).
›
Data augmentation
machine-learning · low: Generate label-preserving variants of each training example on the fly to enlarge the effective dataset and bake in invariances.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
+ 6 more techniques
+ 39 more primitives
System
Techniques · 13
Primitives · 55
code-completion
Incremental parsing
compilers · medium: Reparse only the region an edit damaged and splice the new subtrees into the old tree, sharing everything else; O(edit) instead of O(file).
›
Language server protocol
compilers · medium: One JSON-RPC conversation over a pipe lets any editor drive any language's analysis server; the cost is keeping a live, edited document in sync and answering before it changes.
›
Knowledge distillation
ml-training · medium: Train a small student to match a large teacher's softened outputs (plus the hard labels) so it keeps most of the teacher's accuracy at a fraction of the cost.
›
A/B testing (online controlled expe…
statistics · medium: Randomly split units into arms by a salted hash, log exposure, compare metrics with a confidence interval, and guard the result with sample-ratio, guardrail and peeking checks.
›
Code symbol indexing
compilers · medium: Extract definitions and references per file, give each symbol a stable string id, and store symbol to occurrences, re-indexing only files whose content hash changed.
›
Lossless syntax trees
compilers · medium: Concrete syntax trees with trivia, green/red node split (Roslyn, rowan) for tools that rewrite code
Parser error recovery
compilers · medium: Recover from syntax errors via synchronization tokens and error nodes to keep a full tree
Trigram index
search-sort · medium: Trigram posting lists accelerating regex and substring search
Hypothesis testing
statistics · medium: Pick a test statistic, compare it to its distribution under "no effect", read off a p-value, and control error rates across one test or many.
›
Sequential hypothesis testing
statistics · medium: Group-sequential and alpha-spending designs for interim analyses without inflating error
Linear-time regular expression matc…
search-sort · medium: Match a regular expression in O(pattern x text) worst case by tracking all live parse positions at once, then cache the state sets as a lazily built DFA for speed.
›
Sequential probability ratio test
statistics · medium: Add up the log-likelihood ratio per observation and stop when it crosses a fixed bound; the least data of any test with the same error rates (for two point hypotheses).
›
Inverted index
storage · medium: Map each term to a sorted, compressed list of the documents containing it; the core of full-text search and other fast set queries.
›
Token sampling
ml-training · low: Turn a vector of scores into one chosen item with temperature, top-k, top-p, min-p, repetition penalties and a seed.
›
LRU and CLOCK
systems · low: LRU evicts the least recently used entry via hash map plus linked list; CLOCK approximates it with a reference bit, so hits take no lock.
›
Code context retrieval
agentic-ai · unknown:
Constrained decoding
agentic-ai · low: Force LLM output to match a regex, JSON schema or grammar by masking, at every step, the tokens that would leave the language.
›
Prompt caching
agentic-ai · low: Reuse the attention keys/values of a repeated prompt prefix, so an agent's growing context is prefilled once and later turns pay only for the new tokens.
›
Continuous batching
ml-training · low: Schedule LLM inference per iteration, not per request; finished sequences leave and new ones join the running batch every step, which keeps the GPU full.
›
Fill in the middle
ml-training · low: FIM training and prompting: prefix/suffix/middle reordering with sentinel tokens for code infill
Paged KV cache
ml-training · low: Store each sequence's attention keys/values in fixed-size blocks mapped through a block table, like virtual memory pages; near-zero fragmentation and copy-on-write sharing.
›
Speculative decoding
ml-training · low: A cheap draft proposes γ tokens, the big model verifies them in one pass, and rejection sampling keeps the target's exact distribution at 2-3x speed.
›
Token healing
ml-training · low: Backing up the last prompt token and constraining generation to avoid tokenization artifacts at boundaries
Weight quantization
ml-training · low: Store weights as 8/4-bit integers with per-group scales; decode is memory-bound, so 4-bit weights give about 3-4x tokens/s and fit 4x larger models.
›
Debouncing throttling
systems · low: Coalescing rapid events: debounce (wait for quiet), throttle (max rate), and cancelling stale in-flight requests
Recursive-descent parsing
compilers · low: Hand-written top-down parser with one function per grammar rule; simple, fast, with full control over error messages and recovery.
›
Rope
graphics · low: A balanced tree of small string chunks with cached byte and newline counts, so seek, insert, delete and snapshot cost O(log n) instead of O(n).
›
Data augmentation
machine-learning · low: Generate label-preserving variants of each training example on the fly to enlarge the effective dataset and bake in invariances.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Bootstrap confidence intervals
statistics · low: Resampling-based confidence intervals for arbitrary statistics
CUPED (variance reduction with pre-…
statistics · low: Subtract the part of each unit's outcome that a pre-experiment covariate predicts; the effect estimate stays unbiased and its variance shrinks by 1 - rho^2.
›
Experiment layering (overlapping ex…
statistics · low: Partition the tunable parameters into layers, hash each unit with a per-layer salt so layers are independent, and give experiments inside a layer disjoint bucket ranges.
›
Deterministic bucketing
systems · low: Hash a stable unit id with a per-purpose salt into a bucket; "in" means bucket below the percentage, so assignment is stateless, repeatable and only grows on ramp-up.
›
Name resolution
compilers · low: Bind every use of a name to its declaration with a scope stack, handling shadowing, hoisting and closure capture in one AST walk.
›
Protobuf wire format
networking · low: Protocol Buffers wire format: tags, varints, length-delimited fields and schema evolution
Text position encoding
compilers · low: Store byte offsets, convert to (line, column) on demand with a line-start table and binary search, and count columns in the unit the peer negotiated.
›
Hash consing
data-structures · low: Intern structurally equal immutable terms so equality is pointer comparison and sharing is maximal
String interning
data-structures · low: Store each distinct string once and refer to it by id
Edit distance
search-sort · low: Levenshtein/Damerau distance by DP, bit-parallel Myers variant
Sorted-list intersection
search-sort · low: Intersect sorted id lists by merging, galloping from the short list into the long one, and skipping whole compressed blocks, shortest list first.
›
Wilson score interval
statistics · low: Wilson lower bound for ranking by up/down votes and proportions with few samples
Bayesian rate estimation
statistics · low: Estimate a conversion or event rate from few observations with a Beta-Binomial or Gamma-Poisson update: two counters, an honest interval, and P(B beats A) directly.
›
Aho-Corasick multi-pattern matching
search-sort · low: Find all occurrences of many fixed patterns in one pass over the text, in time independent of the pattern count, using a trie with failure links.
›
SIMD substring search
search-sort · low: Find bytes or substrings by comparing 16-64 haystack positions per instruction against the needle's first and last byte, then verifying the rare candidates.
›
Loglikelihood scoring
machine-learning · low: Scoring choices by summed token log-probabilities, normalization by length, perplexity computation
Bit packing and PFOR
compression · low: Store blocks of integers with exactly b bits each and patch rare outliers separately (PFOR); the fastest integer codec family, billions of values per second.
›
Delta encoding
compression · low: Store differences between consecutive values instead of the values, turning sorted IDs, timestamps and counters into small numbers that pack into few bits.
›
Varint (LEB128)
compression · low: Variable-length integers in 7-bit groups with a continuation bit; small numbers take 1 byte, the wire format of Protobuf, WebAssembly and DWARF.
›
BM25 ranking
search-sort · low: The standard lexical relevance score: IDF-weighted term frequency with saturation (k1) and length normalization (b); default in Lucene, Elasticsearch and SQLite FTS5.
›
Reference counting
compilers · low: Keep a count of owners in each object, free it when the count hits zero; deterministic and local, but pays on every copy and leaks cycles.
›
d-ary heap
data-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.
›
Fast hash functions
data-structures · low: Non-cryptographic hashes built from 64×64→128-bit multiplies and xor-shifts; pick by key size, trust level and whether the hash is persisted.
›
Radix trie
data-structures · low: Compressed prefix tree for path routing, domain matching and prefix lookups
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
Byte-pair encoding (BPE) tokenizati…
machine-learning · low: Learn a subword vocabulary by repeatedly merging the most frequent adjacent pair; byte-level BPE encodes any input losslessly with 32k-200k tokens.
›
Embeddings
machine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.
›
Text normalization
machine-learning · low: Rewrite messy text into one canonical form for matchers and models, or into spoken words for speech output (and back), keeping a map to the original offsets.
›
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Rejection sampling
ml-training · low: Best-of-n and filtered sampling: generate many candidates, keep those passing a verifier or reward threshold
Reverse-mode automatic differentiat…
ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
Low precision number formats
numerics · low: Bf16, fp16, fp8 (E4M3, E5M2), MX block formats and int4: ranges, rounding, scaling
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Finite state transducer
search-sort · low: Minimal acyclic FST mapping sorted keys to values; compact term dictionaries with prefix/fuzzy search
Hybrid search and rank fusion
search-sort · low: Merge a keyword (BM25) result list and a vector (ANN) list into one ranking, by ranks (RRF) or by normalized scores (convex combination).
›
Top-k selection
search-sort · low: Keep the k best of n items without sorting all n - bounded min-heap for streams, quickselect for arrays, threshold pruning to skip most items.
›
Unicode text tokenization for search
search-sort · low: Turn multilingual text into index terms: segment words (UAX 29 style, CJK bigrams), normalize, case-fold and accent-fold, identically at index and query time.
›
JSON Schema validation
security · low: Compile a JSON Schema once, then check every untrusted document against it, within hard size and depth limits, and report failures as JSON Pointers.
›
Copy for AI
Similar systems
Share the most building blocks with this one.
llm-inference
26 %
synthetic-data
22 %