mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
AI and ML infrastructure
self-play-training
self-play RL training loop
AlphaZero, Leela
System
Techniques · 8
Primitives · 37
self-play-training
Batched MCTS inference
game-ai · medium: Feed an accelerator from tree search driven by a learned evaluator: virtual loss gathers leaves within a tree, parallel searches fill batches, an async queue overlaps CPU and GPU.
›
Monte Carlo tree search
game-ai · medium: Grows a search tree one simulation at a time, picking visits by a bandit rule (UCT or PUCT) and backing up averages; the search behind AlphaZero.
›
AlphaZero self-play
machine-learning · medium: A policy-value net guides PUCT search; search visit counts and game results become its training targets, so it learns from its own improved play.
›
Data loading pipelines
machine-learning · medium: Turn files on disk into shuffled, sharded, resumable GPU batches fast enough that the accelerator never waits.
›
Distributed checkpointing
ml-training · medium: Sharded, asynchronous checkpoint save/load of model and optimizer state, with resharding to new world sizes
Sequential probability ratio test
statistics · medium: Add up the log-likelihood ratio per observation and stop when it crosses a fixed bound; the least data of any test with the same error rates (for two point hypotheses).
›
Multi-armed bandits
machine-learning · medium: Choose among K options repeatedly while learning their payoffs: epsilon-greedy, UCB1 and Thompson sampling keep total regret low where a fixed split or a fixed exploration rate pays forever.
›
Transposition table
game-ai · low: A fixed-size, lossy hash table of search results keyed by a state hash; gives ordering and cutoffs for repeated states, shared lock-free between threads.
›
Experience replay
machine-learning · low: Ring-buffer replay memory with uniform or prioritized sampling for off-policy and self-play training
Convolutional networks
ml-training · low: Layers that slide a small shared kernel over a grid, giving translation equivariance and few parameters; still the most compute-efficient choice for many vision tasks.
›
SGD with momentum
ml-training · low: Gradient descent plus a decaying velocity buffer that averages minibatch noise and speeds up progress along shallow, consistent directions.
›
Compact state encoding
data-structures · low: Pack a game position into a few fixed bytes (24 for chess: occupancy bitboard plus nibbles) for training data and dedupe
Bitboards
game-ai · low: One 64-bit word per set of up to 64 items, so set algebra becomes shifts, ANDs and table lookups; magic bitboards give ray queries in one multiply.
›
Zobrist hashing
game-ai · low: A state's hash is the XOR of one random 64-bit key per (item, slot) fact that holds in it, updated in a few XORs per change.
›
Elo rating
game-networking · low: One number per competitor, moved by K × (actual − expected score) after each pairwise result; expected score is logistic in the rating difference.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Mixed-precision training
ml-training · low: Run matmuls in bf16/fp16/fp8 on tensor cores while keeping master weights, optimizer state and reductions in fp32; about 2x speed, half the activation memory.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
Weight quantization
ml-training · low: Store weights as 8/4-bit integers with per-group scales; decode is memory-bound, so 4-bit weights give about 3-4x tokens/s and fit 4x larger models.
›
Weight synchronization
ml-training · low: Pushing updated policy weights from trainers to inference/actor workers (NCCL broadcast, sharded resharding)
MPMC bounded queue
concurrency · low: Vyukov's array queue for many producers and consumers: each slot carries a sequence number, so one CAS per operation claims a slot without locks.
›
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Symmetry data augmentation
machine-learning · low: Augment board-game training data with the board's symmetry group (8 dihedral transforms)
+ 1 more techniques
+ 21 more primitives
System
Techniques · 8
Primitives · 37
self-play-training
Batched MCTS inference
game-ai · medium: Feed an accelerator from tree search driven by a learned evaluator: virtual loss gathers leaves within a tree, parallel searches fill batches, an async queue overlaps CPU and GPU.
›
Monte Carlo tree search
game-ai · medium: Grows a search tree one simulation at a time, picking visits by a bandit rule (UCT or PUCT) and backing up averages; the search behind AlphaZero.
›
AlphaZero self-play
machine-learning · medium: A policy-value net guides PUCT search; search visit counts and game results become its training targets, so it learns from its own improved play.
›
Data loading pipelines
machine-learning · medium: Turn files on disk into shuffled, sharded, resumable GPU batches fast enough that the accelerator never waits.
›
Distributed checkpointing
ml-training · medium: Sharded, asynchronous checkpoint save/load of model and optimizer state, with resharding to new world sizes
Sequential probability ratio test
statistics · medium: Add up the log-likelihood ratio per observation and stop when it crosses a fixed bound; the least data of any test with the same error rates (for two point hypotheses).
›
Multi-armed bandits
machine-learning · medium: Choose among K options repeatedly while learning their payoffs: epsilon-greedy, UCB1 and Thompson sampling keep total regret low where a fixed split or a fixed exploration rate pays forever.
›
Dynamic batching
ml-training · medium: Queue independent requests and dispatch them as one batch when it is full or the oldest has waited max_delay; bounded extra latency buys throughput.
›
Transposition table
game-ai · low: A fixed-size, lossy hash table of search results keyed by a state hash; gives ordering and cutoffs for repeated states, shared lock-free between threads.
›
Experience replay
machine-learning · low: Ring-buffer replay memory with uniform or prioritized sampling for off-policy and self-play training
Convolutional networks
ml-training · low: Layers that slide a small shared kernel over a grid, giving translation equivariance and few parameters; still the most compute-efficient choice for many vision tasks.
›
SGD with momentum
ml-training · low: Gradient descent plus a decaying velocity buffer that averages minibatch noise and speeds up progress along shallow, consistent directions.
›
Compact state encoding
data-structures · low: Pack a game position into a few fixed bytes (24 for chess: occupancy bitboard plus nibbles) for training data and dedupe
Bitboards
game-ai · low: One 64-bit word per set of up to 64 items, so set algebra becomes shifts, ANDs and table lookups; magic bitboards give ray queries in one multiply.
›
Zobrist hashing
game-ai · low: A state's hash is the XOR of one random 64-bit key per (item, slot) fact that holds in it, updated in a few XORs per change.
›
Elo rating
game-networking · low: One number per competitor, moved by K × (actual − expected score) after each pairwise result; expected score is logistic in the rating difference.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Mixed-precision training
ml-training · low: Run matmuls in bf16/fp16/fp8 on tensor cores while keeping master weights, optimizer state and reductions in fp32; about 2x speed, half the activation memory.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
Weight quantization
ml-training · low: Store weights as 8/4-bit integers with per-group scales; decode is memory-bound, so 4-bit weights give about 3-4x tokens/s and fit 4x larger models.
›
Weight synchronization
ml-training · low: Pushing updated policy weights from trainers to inference/actor workers (NCCL broadcast, sharded resharding)
MPMC bounded queue
concurrency · low: Vyukov's array queue for many producers and consumers: each slot carries a sequence number, so one CAS per operation claims a slot without locks.
›
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Symmetry data augmentation
machine-learning · low: Augment board-game training data with the board's symmetry group (8 dihedral transforms)
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
mmap and the page cache
hardware-os · low: How read() and mmap reach file data through the kernel page cache, what readahead and madvise do, and why databases use their own buffer pool instead.
›
Sequence packing
ml-training · low: Packing variable-length sequences into fixed-size batches with attention masks to avoid padding waste
Atomic file replacement
hardware-os · low: Write temp file, fsync, rename over target, fsync directory: crash-safe whole-file updates
Collective communication
ml-training · low: The family of group exchanges (broadcast, reduce, all-gather, reduce-scatter, all-to-all) and the latency-plus-bytes cost model that picks ring, tree or direct algorithms by message size and topology.
›
Tensor layout transforms
ml-training · low: Choosing and converting tensor layouts (NCHW/NHWC, blocked, tiled) for kernel efficiency
ZeRO and FSDP sharding
ml-training · low: Data parallelism that shards optimizer state, gradients and weights across ranks and gathers weights just in time; per-GPU model memory drops from 16Ψ to 16Ψ/N.
›
Loglikelihood scoring
machine-learning · low: Scoring choices by summed token log-probabilities, normalization by length, perplexity computation
Bayesian rate estimation
statistics · low: Estimate a conversion or event rate from few observations with a Beta-Binomial or Gamma-Poisson update: two counters, an honest interval, and P(B beats A) directly.
›
Backpressure
systems · low: Bound every queue and make a full queue push back on the producer by blocking, dropping, rejecting or withholding credit.
›
Hierarchical timing wheel
systems · low: Timers bucketed by expiry into wheels of slots, coarser wheels for far deadlines: O(1) insert and cancel for millions of timeouts.
›
False sharing
concurrency · low: Independent variables written by different cores on one cache line force the line to bounce between cores; pad or regroup hot data to fix it.
›
Memory ordering
concurrency · low: What Relaxed, Acquire, Release and SeqCst atomics actually guarantee, what they cost on x86 and ARM, and which one each pattern needs.
›
CPU caches and the memory hierarchy
hardware-os · low: Sizes, line widths and latencies of L1/L2/L3/DRAM on modern x86 and Apple M-series, and the data-layout rules that follow from them.
›
Virtual memory and the TLB
hardware-os · low: How page tables and TLBs translate every address, what TLB reach means, and when 2 MiB or 1 GiB huge pages pay off on Linux.
›
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Ring all-reduce
ml-training · low: Sum a buffer across N workers via reduce-scatter then all-gather around a ring; each link carries 2(N−1)/N of the buffer, which is bandwidth-optimal.
›
Floating-point error
numerics · low: How IEEE 754 rounding error accumulates and how to contain it: compensated (Kahan/Neumaier) and pairwise summation, error-free transformations with FMA, ULP comparisons.
›
Low precision number formats
numerics · low: Bf16, fp16, fp8 (E4M3, E5M2), MX block formats and int4: ranges, rounding, scaling
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Slab allocator
systems · low: Pre-carve pages into equal-size slots per size class with an in-place free list: O(1) alloc and free, no external fragmentation, cache-warm reuse.
›
Copy for AI
Similar systems
Share the most building blocks with this one.
go-engine
38 %
nnue-training
26 %
fair-play
21 %