mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
Agentic AI
structured-extraction
LLM structured extraction pipeline
document-to-JSON
System
Techniques · 11
Primitives · 57
structured-extraction
Layout analysis
machine-learning · medium: Detecting page regions (text blocks, tables, figures) and reading order in documents
LLM-as-judge
machine-learning · medium: Grade open-ended outputs with a language model under a rubric, then correct its known biases by order swapping, voting and calibration against human labels.
›
Named entity recognition
machine-learning · medium: Find the character spans of entities in text and give each a type, by labelling every token and decoding a valid label sequence.
›
Optical character recognition (text…
machine-learning · medium: Find text lines in an image, read each crop with a CTC or attention model, then correct with a lexicon and score confidence for review routing.
›
Table structure recognition
machine-learning · medium: Recovering rows, columns and cell spans of tables from document images
Vision transformer
ml-training · medium: ViT: patch embeddings, position encodings, encoder blocks; vision-language connectors
Entity resolution
storage · medium: Decide which records describe the same real-world thing: block to cut the pair space, score pairs with a probabilistic model, cluster the matches transitively.
›
Constrained decoding
agentic-ai · low: Force LLM output to match a regex, JSON schema or grammar by masking, at every step, the tokens that would leave the language.
›
Prompt caching
agentic-ai · low: Reuse the attention keys/values of a repeated prompt prefix, so an agent's growing context is prefilled once and later turns pay only for the new tokens.
›
Tool calling
agentic-ai · low: The model emits a structured call (id, name, JSON arguments) instead of prose; your code validates, runs it and returns a result tagged with the same id.
›
Calibration
machine-learning · low: Make predicted probabilities match observed frequencies: measure with ECE, fix post hoc with temperature, Platt or isotonic scaling on held-out data.
›
Evaluation metrics
machine-learning · low: Precision, recall, ROC AUC, average precision, log loss, NDCG and pass@k computed correctly, plus paired confidence intervals for comparing models.
›
PDF text extraction
machine-learning · low: Rebuild words, lines, columns and reading order from the glyph positions a born-digital PDF stores, and route pages without usable text to OCR.
›
Text chunking
machine-learning · low: Cut documents into token-bounded pieces along structure boundaries, keeping offsets and section paths, so each piece embeds well and can be cited.
›
Continuous batching
ml-training · low: Schedule LLM inference per iteration, not per request; finished sequences leave and new ones join the running batch every step, which keeps the GPU full.
›
Self consistency voting
ml-training · low: Sample several answers and pick by majority or execution agreement to raise reliability
Edit distance
search-sort · low: Levenshtein/Damerau distance by DP, bit-parallel Myers variant
JSON Schema validation
security · low: Compile a JSON Schema once, then check every untrusted document against it, within hard size and depth limits, and report failures as JSON Pointers.
›
Json streaming parser
storage · low: Streaming/zero-copy JSON parsing and in-place field rewriting
Pdf object model parsing
storage · low: PDF xref tables, object streams, content stream operators and incremental updates
Topological sort
graphs · low: Order the vertices of a DAG so every edge points forward; the core of build systems, schedulers and dependency resolution, with cycle detection for free.
›
Document image preprocessing
machine-learning · low: Turn a scan or phone photo into an upright, evenly lit, binarised or normalised grey page so layout analysis and OCR read it reliably.
›
Token sampling
ml-training · low: Turn a vector of scores into one chosen item with temperature, top-k, top-p, min-p, repetition penalties and a seed.
›
+ 4 more techniques
+ 41 more primitives
System
Techniques · 11
Primitives · 57
structured-extraction
Layout analysis
machine-learning · medium: Detecting page regions (text blocks, tables, figures) and reading order in documents
LLM-as-judge
machine-learning · medium: Grade open-ended outputs with a language model under a rubric, then correct its known biases by order swapping, voting and calibration against human labels.
›
Named entity recognition
machine-learning · medium: Find the character spans of entities in text and give each a type, by labelling every token and decoding a valid label sequence.
›
Optical character recognition (text…
machine-learning · medium: Find text lines in an image, read each crop with a CTC or attention model, then correct with a lexicon and score confidence for review routing.
›
Table structure recognition
machine-learning · medium: Recovering rows, columns and cell spans of tables from document images
Vision transformer
ml-training · medium: ViT: patch embeddings, position encodings, encoder blocks; vision-language connectors
Entity resolution
storage · medium: Decide which records describe the same real-world thing: block to cut the pair space, score pairs with a probabilistic model, cluster the matches transitively.
›
Object detection
machine-learning · medium: A network turns an image into scored, classified boxes in one pass; anchor-based, anchor-free and set-prediction heads differ in how they assign labels and remove duplicates.
›
Record linkage
search-sort · medium: Probabilistic record matching (Fellegi-Sunter) with blocking for patient or entity indexes
Inverted index
storage · medium: Map each term to a sorted, compressed list of the documents containing it; the core of full-text search and other fast set queries.
›
Near-duplicate detection with MinHa…
machine-learning · medium: Estimate set Jaccard similarity from k min-hashes and find similar pairs with banded LSH buckets; the standard dedup for web crawls, training corpora and sequence collections.
›
Constrained decoding
agentic-ai · low: Force LLM output to match a regex, JSON schema or grammar by masking, at every step, the tokens that would leave the language.
›
Prompt caching
agentic-ai · low: Reuse the attention keys/values of a repeated prompt prefix, so an agent's growing context is prefilled once and later turns pay only for the new tokens.
›
Tool calling
agentic-ai · low: The model emits a structured call (id, name, JSON arguments) instead of prose; your code validates, runs it and returns a result tagged with the same id.
›
Calibration
machine-learning · low: Make predicted probabilities match observed frequencies: measure with ECE, fix post hoc with temperature, Platt or isotonic scaling on held-out data.
›
Evaluation metrics
machine-learning · low: Precision, recall, ROC AUC, average precision, log loss, NDCG and pass@k computed correctly, plus paired confidence intervals for comparing models.
›
PDF text extraction
machine-learning · low: Rebuild words, lines, columns and reading order from the glyph positions a born-digital PDF stores, and route pages without usable text to OCR.
›
Text chunking
machine-learning · low: Cut documents into token-bounded pieces along structure boundaries, keeping offsets and section paths, so each piece embeds well and can be cited.
›
Continuous batching
ml-training · low: Schedule LLM inference per iteration, not per request; finished sequences leave and new ones join the running batch every step, which keeps the GPU full.
›
Self consistency voting
ml-training · low: Sample several answers and pick by majority or execution agreement to raise reliability
Edit distance
search-sort · low: Levenshtein/Damerau distance by DP, bit-parallel Myers variant
JSON Schema validation
security · low: Compile a JSON Schema once, then check every untrusted document against it, within hard size and depth limits, and report failures as JSON Pointers.
›
Json streaming parser
storage · low: Streaming/zero-copy JSON parsing and in-place field rewriting
Pdf object model parsing
storage · low: PDF xref tables, object streams, content stream operators and incremental updates
Topological sort
graphs · low: Order the vertices of a DAG so every edge points forward; the core of build systems, schedulers and dependency resolution, with cycle detection for free.
›
Document image preprocessing
machine-learning · low: Turn a scan or phone photo into an upright, evenly lit, binarised or normalised grey page so layout analysis and OCR read it reliably.
›
Token sampling
ml-training · low: Turn a vector of scores into one chosen item with temperature, top-k, top-p, min-p, repetition penalties and a seed.
›
Annotator agreement
statistics · low: Inter-annotator agreement measures: Cohen and Fleiss kappa, Krippendorff alpha, gold-question accuracy
Bradley terry model
statistics · low: Pairwise comparison model P(i beats j)=s_i/(s_i+s_j); MLE fit, link to Elo and reward models
Byte-pair encoding (BPE) tokenizati…
machine-learning · low: Learn a subword vocabulary by repeatedly merging the most frequent adjacent pair; byte-level BPE encodes any input losslessly with 32k-200k tokens.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
Convolutional codes and the Viterbi…
signal-processing · low: Streaming error-correcting code: each output bit is a parity of the last K input bits; the Viterbi decoder finds the most likely path through a 2^(K-1)-state trellis.
›
Connectionist temporal classificati…
machine-learning · low: Alignment-free sequence loss that sums the probability of every frame-to-label alignment with a blank symbol, using a dynamic program.
›
Convolutional networks
ml-training · low: Layers that slide a small shared kernel over a grid, giving translation equivariance and few parameters; still the most compute-efficient choice for many vision tasks.
›
Attention variants
machine-learning · low: Multi-head, multi-query, grouped-query, sliding-window and latent attention; same softmax(QKᵀ/√d)V core, different trade-offs between quality and KV-cache size.
›
Data augmentation
machine-learning · low: Generate label-preserving variants of each training example on the fly to enlarge the effective dataset and bake in invariances.
›
Embeddings
machine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.
›
Union-find
graphs · low: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Non-maximum suppression
machine-learning · low: Keep the best-scoring candidate in each neighbourhood and drop the rest; greedy NMS, Soft-NMS and 1-D peak picking share one loop.
›
Linear and logistic regression
machine-learning · low: Fit a weighted sum of features to a number (least squares) or a probability (logistic); the baseline every model must beat.
›
Fuzzy string matching
search-sort · low: Edit distance, Jaro-Winkler and trigram indexes for approximate name and address matching
Bit packing and PFOR
compression · low: Store blocks of integers with exactly b bits each and patch rare outliers separately (PFOR); the fastest integer codec family, billions of values per second.
›
Delta encoding
compression · low: Store differences between consecutive values instead of the values, turning sorted IDs, timestamps and counters into small numbers that pack into few bits.
›
Varint (LEB128)
compression · low: Variable-length integers in 7-bit groups with a continuation bit; small numbers take 1 byte, the wire format of Protobuf, WebAssembly and DWARF.
›
BM25 ranking
search-sort · low: The standard lexical relevance score: IDF-weighted term frequency with saturation (k1) and length normalization (b); default in Lucene, Elasticsearch and SQLite FTS5.
›
Sorted-list intersection
search-sort · low: Intersect sorted id lists by merging, galloping from the short list into the long one, and skipping whole compressed blocks, shortest list first.
›
Fast hash functions
data-structures · low: Non-cryptographic hashes built from 64×64→128-bit multiplies and xor-shifts; pick by key size, trust level and whether the hash is persisted.
›
Rolling hash
data-structures · low: Rabin-Karp polynomial rolling hashes and winnowing for substring fingerprints and document matching
Swiss table
data-structures · low: Open-addressing hash table that filters 8-16 slots at once with 1-byte hash tags and SIMD, giving fast lookups at 87.5 % load.
›
Reference counting
compilers · low: Keep a count of owners in each object, free it when the count hits zero; deterministic and local, but pays on every copy and leaks cycles.
›
d-ary heap
data-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.
›
Radix trie
data-structures · low: Compressed prefix tree for path routing, domain matching and prefix lookups
BFS and DFS
graphs · low: The two basic graph traversals; BFS gives hop distances and levels, DFS gives discovery/finish order for cycles, components and topological order.
›
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
Text normalization
machine-learning · low: Rewrite messy text into one canonical form for matchers and models, or into spoken words for speech output (and back), keeping a map to the original offsets.
›
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Paged KV cache
ml-training · low: Store each sequence's attention keys/values in fixed-size blocks mapped through a block table, like virtual memory pages; near-zero fragmentation and copy-on-write sharing.
›
Reverse-mode automatic differentiat…
ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.
›
Json rpc
networking · low: JSON-RPC 2.0 requests, notifications and batching
L-BFGS
numerics · low: Quasi-Newton minimizer that approximates the inverse Hessian from the last m gradient differences via the two-loop recursion; O(m·n) memory and work per step.
›
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Sparse matrices (CSR)
numerics · low: Compressed sparse row stores only nonzeros as row pointers, column indices and values; SpMV streams them once, so it is memory-bandwidth bound.
›
Finite state transducer
search-sort · low: Minimal acyclic FST mapping sorted keys to values; compact term dictionaries with prefix/fuzzy search
Top-k selection
search-sort · low: Keep the k best of n items without sorting all n - bounded min-heap for streams, quickselect for arrays, threshold pruning to skip most items.
›
Unicode text tokenization for search
search-sort · low: Turn multilingual text into index terms: segment words (UAX 29 style, CJK bigrams), normalize, case-fold and accent-fold, identically at index and query time.
›
Bootstrap confidence intervals
statistics · low: Resampling-based confidence intervals for arbitrary statistics
LRU and CLOCK
systems · low: LRU evicts the least recently used entry via hash map plus linked list; CLOCK approximates it with a reference bit, so hits take no lock.
›
Copy for AI
Similar systems
Share the most building blocks with this one.
document-ocr
38 %
eval-harness
18 %
llm-guardrails
17 %
loan-origination
16 %
ai-tutor
16 %