mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
AI and ML infrastructure
hyperparameter-search
hyperparameter search
Optuna, Vizier
System
Techniques · 8
Primitives · 34
hyperparameter-search
Bayesian optimization
ml-training · medium: Surrogate model plus acquisition function (EI, UCB) to choose the next expensive evaluation
Experiment tracking
ml-training · medium: Logging params, metrics, artifacts and code versions per run with an append-only metric store and comparison
Population based training
ml-training · medium: Parallel training population with periodic exploit (copy best weights) and explore (perturb hyperparameters)
Successive halving hyperband
ml-training · medium: Early stopping of weak trials by successive halving; Hyperband and ASHA scheduling
Tree parzen estimator
ml-training · medium: TPE: model good vs bad hyperparameter densities and sample where their ratio is high
Cma es
numerics · medium: Covariance matrix adaptation evolution strategy for derivative-free continuous optimization
Durable job queue
distributed · medium: Persisted tasks that workers lease for a visibility timeout, ack on success and retry with backoff; at-least-once execution that survives worker and queue crashes.
›
Gaussian processes
machine-learning · low: GP regression with kernels: posterior mean/variance via Cholesky, hyperparameters by marginal likelihood
L-BFGS
numerics · low: Quasi-Newton minimizer that approximates the inverse Hessian from the last m gradient differences via the two-loop recursion; O(m·n) memory and work per step.
›
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Leases
distributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.
›
Low discrepancy sequences
numerics · low: Sobol, Halton and blue-noise sequences with scrambling for quasi-Monte Carlo
Optimistic concurrency control
storage · low: Run transactions without locks, record what they read, and validate at commit that nothing changed; fastest when conflicts are rare.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Segmented log
distributed · low: Split an append-only log into files named by their first offset, write only the newest, and find any record by binary search plus a sparse index.
›
Lttb downsampling
statistics · low: Largest-Triangle-Three-Buckets: reduce a series to N points while keeping its visual shape
Content-addressed storage
storage · low: Store each blob under the hash of its bytes: immutable, deduplicated by construction, self-verifying on every read, and composable into Merkle trees.
›
Top-k selection
search-sort · low: Keep the k best of n items without sorting all n - bounded min-heap for streams, quickselect for arrays, threshold pruning to skip most items.
›
Constrained parameter transforms
numerics · low: Map constrained parameters to unconstrained space (log, logit, simplex, Cholesky) with Jacobian corrections
PCA and SVD
machine-learning · low: Find orthogonal directions of maximum variance by an SVD of the centred data; randomized SVD gets the top k components in a few passes.
›
Dead-letter queue
distributed · low: A side queue for messages that keep failing: after N delivery attempts or a permanent error, park them with the reason so the main queue keeps moving.
›
Idempotency keys
finance · low: Client-chosen unique key per logical operation, stored with a request fingerprint and the final response, so retries replay the first result instead of acting twice.
›
+ 1 more techniques
+ 18 more primitives
System
Techniques · 8
Primitives · 34
hyperparameter-search
Bayesian optimization
ml-training · medium: Surrogate model plus acquisition function (EI, UCB) to choose the next expensive evaluation
Experiment tracking
ml-training · medium: Logging params, metrics, artifacts and code versions per run with an append-only metric store and comparison
Population based training
ml-training · medium: Parallel training population with periodic exploit (copy best weights) and explore (perturb hyperparameters)
Successive halving hyperband
ml-training · medium: Early stopping of weak trials by successive halving; Hyperband and ASHA scheduling
Tree parzen estimator
ml-training · medium: TPE: model good vs bad hyperparameter densities and sample where their ratio is high
Cma es
numerics · medium: Covariance matrix adaptation evolution strategy for derivative-free continuous optimization
Durable job queue
distributed · medium: Persisted tasks that workers lease for a visibility timeout, ack on success and retry with backoff; at-least-once execution that survives worker and queue crashes.
›
Distributed checkpointing
ml-training · medium: Sharded, asynchronous checkpoint save/load of model and optimizer state, with resharding to new world sizes
Gaussian processes
machine-learning · low: GP regression with kernels: posterior mean/variance via Cholesky, hyperparameters by marginal likelihood
L-BFGS
numerics · low: Quasi-Newton minimizer that approximates the inverse Hessian from the last m gradient differences via the two-loop recursion; O(m·n) memory and work per step.
›
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Leases
distributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.
›
Low discrepancy sequences
numerics · low: Sobol, Halton and blue-noise sequences with scrambling for quasi-Monte Carlo
Optimistic concurrency control
storage · low: Run transactions without locks, record what they read, and validate at commit that nothing changed; fastest when conflicts are rare.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Segmented log
distributed · low: Split an append-only log into files named by their first offset, write only the newest, and find any record by binary search plus a sparse index.
›
Lttb downsampling
statistics · low: Largest-Triangle-Three-Buckets: reduce a series to N points while keeping its visual shape
Content-addressed storage
storage · low: Store each blob under the hash of its bytes: immutable, deduplicated by construction, self-verifying on every read, and composable into Merkle trees.
›
Top-k selection
search-sort · low: Keep the k best of n items without sorting all n - bounded min-heap for streams, quickselect for arrays, threshold pruning to skip most items.
›
Constrained parameter transforms
numerics · low: Map constrained parameters to unconstrained space (log, logit, simplex, Cholesky) with Jacobian corrections
PCA and SVD
machine-learning · low: Find orthogonal directions of maximum variance by an SVD of the centred data; randomized SVD gets the top k components in a few passes.
›
Dead-letter queue
distributed · low: A side queue for messages that keep failing: after N delivery attempts or a permanent error, park them with the reason so the main queue keeps moving.
›
Idempotency keys
finance · low: Client-chosen unique key per logical operation, stored with a request fingerprint and the final response, so retries replay the first result instead of acting twice.
›
Write-ahead log
storage · low: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.
›
Exponential backoff with jitter
systems · low: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.
›
Atomic file replacement
hardware-os · low: Write temp file, fsync, rename over target, fsync directory: crash-safe whole-file updates
Collective communication
ml-training · low: The family of group exchanges (broadcast, reduce, all-gather, reduce-scatter, all-to-all) and the latency-plus-bytes cost model that picks ring, tree or direct algorithms by message size and topology.
›
Tensor layout transforms
ml-training · low: Choosing and converting tensor layouts (NCHW/NHWC, blocked, tiled) for kernel efficiency
ZeRO and FSDP sharding
ml-training · low: Data parallelism that shards optimizer state, gradients and weights across ranks and gathers weights just in time; per-GPU model memory drops from 16Ψ to 16Ψ/N.
›
d-ary heap
data-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.
›
Merkle dag
data-structures · low: Content-addressed DAG of hashed nodes (git trees/commits) for dedupe and verifiable history
Fencing tokens
distributed · low: A number that grows with every lock grant and is checked by the protected resource, so a paused ex-holder's late write is rejected.
›
CRC32C checksums
hardware-os · low: The Castagnoli CRC-32 with dedicated instructions on x86 (SSE4.2) and ARMv8. It is the standard fast integrity check for storage blocks, logs and network frames.
›
fsync, O_DSYNC and O_DIRECT
hardware-os · low: What fsync, fdatasync, O_DSYNC, O_DIRECT and F_FULLFSYNC really guarantee on Linux and macOS, and the exact call sequences for crash-safe files.
›
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Mixed-precision training
ml-training · low: Run matmuls in bf16/fp16/fp8 on tensor cores while keeping master weights, optimizer state and reductions in fp32; about 2x speed, half the activation memory.
›
Ring all-reduce
ml-training · low: Sum a buffer across N workers via reduce-scatter then all-gather around a ring; each link carries 2(N−1)/N of the buffer, which is bandwidth-optimal.
›
Floating-point error
numerics · low: How IEEE 754 rounding error accumulates and how to contain it: compensated (Kahan/Neumaier) and pairwise summation, error-free transformations with FMA, ULP comparisons.
›
Low precision number formats
numerics · low: Bf16, fp16, fp8 (E4M3, E5M2), MX block formats and int4: ranges, rounding, scaling
Cryptographic hashes (SHA-2, BLAKE3)
security · low: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.
›
Copy for AI
Similar systems
Share the most building blocks with this one.
uncertainty-quantification
17 %