mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
Frontier AI research
robot-policy
robot foundation policy
RT-2, π0
System
Techniques · 9
Primitives · 43
robot-policy
Data loading pipelines
machine-learning · medium: Turn files on disk into shuffled, sharded, resumable GPU batches fast enough that the accelerator never waits.
›
Domain randomization
machine-learning · medium: Randomize simulator physics and visuals so policies trained in sim transfer to real robots
Imitation learning
machine-learning · medium: Learning policies from demonstrations: behavior cloning, DAgger, covariate shift
Flow matching
ml-training · medium: Training a velocity field along straight (rectified) paths from noise to data; sample by integrating an ODE
LLM fine-tuning
ml-training · medium: Adapting a pretrained LLM with supervised fine-tuning, LoRA/QLoRA adapters and preference optimization (DPO), and evaluating that it improved without forgetting.
›
Vision transformer
ml-training · medium: ViT: patch embeddings, position encodings, encoder blocks; vision-language connectors
Impedance control
robotics · medium: Make a robot behave like a spring-damper at the contact point for safe interaction
Action chunking
machine-learning · low: Policy predicts a chunk of future actions per step, executed with temporal ensembling for smooth control
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Learning-rate warmup and cosine dec…
ml-training · low: Ramp the learning rate up linearly for the first ~1% of steps, then decay it along a half cosine to ~10% of peak.
›
Weight quantization
ml-training · low: Store weights as 8/4-bit integers with per-group scales; decode is memory-bound, so 4-bit weights give about 3-4x tokens/s and fit 4x larger models.
›
Attention variants
machine-learning · low: Multi-head, multi-query, grouped-query, sliding-window and latent attention; same softmax(QKᵀ/√d)V core, different trade-offs between quality and KV-cache size.
›
Embeddings
machine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
Fixed-rate control loop
embedded · low: Run a controller at an exact period using absolute deadlines and a fixed dt, so jitter never accumulates and the math stays valid.
›
Action tokenization
machine-learning · low: Discretize or compress continuous robot actions into tokens a sequence model can predict
Rotary position embedding (RoPE)
machine-learning · low: Encode position by rotating pairs of query and key dimensions by position-dependent angles, so attention scores depend only on relative distance; the LLM default.
›
Diffusion models
ml-training · low: Train a network to undo Gaussian noise at every noise level, then generate by integrating from pure noise to data; the standard for images, video and audio.
›
Mixed-precision training
ml-training · low: Run matmuls in bf16/fp16/fp8 on tensor cores while keeping master weights, optimizer state and reductions in fp32; about 2x speed, half the activation memory.
›
ZeRO and FSDP sharding
ml-training · low: Data parallelism that shards optimizer state, gradients and weights across ranks and gathers weights just in time; per-GPU model memory drops from 16Ψ to 16Ψ/N.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
mmap and the page cache
hardware-os · low: How read() and mmap reach file data through the kernel page cache, what readahead and madvise do, and why databases use their own buffer pool instead.
›
Sequence packing
ml-training · low: Packing variable-length sequences into fixed-size batches with attention masks to avoid padding waste
+ 2 more techniques
+ 27 more primitives
System
Techniques · 9
Primitives · 43
robot-policy
Data loading pipelines
machine-learning · medium: Turn files on disk into shuffled, sharded, resumable GPU batches fast enough that the accelerator never waits.
›
Domain randomization
machine-learning · medium: Randomize simulator physics and visuals so policies trained in sim transfer to real robots
Imitation learning
machine-learning · medium: Learning policies from demonstrations: behavior cloning, DAgger, covariate shift
Flow matching
ml-training · medium: Training a velocity field along straight (rectified) paths from noise to data; sample by integrating an ODE
LLM fine-tuning
ml-training · medium: Adapting a pretrained LLM with supervised fine-tuning, LoRA/QLoRA adapters and preference optimization (DPO), and evaluating that it improved without forgetting.
›
Vision transformer
ml-training · medium: ViT: patch embeddings, position encodings, encoder blocks; vision-language connectors
Impedance control
robotics · medium: Make a robot behave like a spring-damper at the contact point for safe interaction
Direct preference optimization
ml-training · medium: DPO and variants: optimize a policy directly on preference pairs with a reference-model log-ratio loss
Rigid body dynamics algorithms
physics · medium: Featherstone RNEA/ABA/CRBA for articulated-body inverse and forward dynamics
Action chunking
machine-learning · low: Policy predicts a chunk of future actions per step, executed with temporal ensembling for smooth control
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Learning-rate warmup and cosine dec…
ml-training · low: Ramp the learning rate up linearly for the first ~1% of steps, then decay it along a half cosine to ~10% of peak.
›
Weight quantization
ml-training · low: Store weights as 8/4-bit integers with per-group scales; decode is memory-bound, so 4-bit weights give about 3-4x tokens/s and fit 4x larger models.
›
Attention variants
machine-learning · low: Multi-head, multi-query, grouped-query, sliding-window and latent attention; same softmax(QKᵀ/√d)V core, different trade-offs between quality and KV-cache size.
›
Embeddings
machine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
Fixed-rate control loop
embedded · low: Run a controller at an exact period using absolute deadlines and a fixed dt, so jitter never accumulates and the math stays valid.
›
Action tokenization
machine-learning · low: Discretize or compress continuous robot actions into tokens a sequence model can predict
Rotary position embedding (RoPE)
machine-learning · low: Encode position by rotating pairs of query and key dimensions by position-dependent angles, so attention scores depend only on relative distance; the LLM default.
›
Diffusion models
ml-training · low: Train a network to undo Gaussian noise at every noise level, then generate by integrating from pure noise to data; the standard for images, video and audio.
›
Mixed-precision training
ml-training · low: Run matmuls in bf16/fp16/fp8 on tensor cores while keeping master weights, optimizer state and reductions in fp32; about 2x speed, half the activation memory.
›
ZeRO and FSDP sharding
ml-training · low: Data parallelism that shards optimizer state, gradients and weights across ranks and gathers weights just in time; per-GPU model memory drops from 16Ψ to 16Ψ/N.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
mmap and the page cache
hardware-os · low: How read() and mmap reach file data through the kernel page cache, what readahead and madvise do, and why databases use their own buffer pool instead.
›
Sequence packing
ml-training · low: Packing variable-length sequences into fixed-size batches with attention masks to avoid padding waste
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Data augmentation
machine-learning · low: Generate label-preserving variants of each training example on the fly to enlarge the effective dataset and bake in invariances.
›
Rigid-body dynamics
physics · low: Advance one 3D rigid body per step from force and torque: semi-implicit Euler for position, an inertia tensor in the body frame, and quaternion orientation updated by the exponential map.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Adaptive Runge-Kutta (Dormand-Princ…
numerics · low: An embedded 5(4) Runge-Kutta pair estimates each step's error for free and resizes the step to meet a tolerance; the default non-stiff solver.
›
Explicit Euler
numerics · low: Step an ODE forward by its current slope, y += h·f(t, y); first-order, one evaluation per step, the baseline every other integrator is measured against.
›
Gradient checkpointing
ml-training · low: Save only some activations in the forward pass and recompute the rest during backward, trading about 33% extra compute for O(√n) activation memory.
›
Lora adapters
ml-training · low: Low-rank adapter matrices for fine-tuning; QLoRA; serving many adapters on one base model
PID controller
robotics · low: Drive a measured value to a setpoint with three terms on the error: proportional, integral, derivative. The default feedback loop for motors, heaters, drones and gimbals.
›
Bradley terry model
statistics · low: Pairwise comparison model P(i beats j)=s_i/(s_i+s_j); MLE fit, link to Elo and reward models
Quaternion rotations
geometry · low: Unit quaternions as 3D rotations: 4 floats, cheap composition, no gimbal lock, plus the conversions, interpolation choices and numerical care needed in practice.
›
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Rigid-body transforms (SE(3) poses)
robotics · low: Represent, compose, invert, integrate and interpolate 3D poses as unit quaternion plus translation, with exp/log maps for small motions and uncertainty.
›
Dma transfers
embedded · low: DMA controllers: descriptors, circular buffers, cache coherence for zero-CPU data movement
PWM generation
embedded · low: A timer's counter, period and compare registers draw the pulse train; preload, dead time and DMA make duty changes glitch-free, shoot-through-free and cheap.
›
Homogeneous transforms
geometry · low: 4x4 affine and projective matrices, homogeneous coordinates, view and projection, normal matrices
Fixed timestep loop
graphics · low: Fixed simulation dt with accumulator, render interpolation, spiral-of-death guard
Virtual memory and the TLB
hardware-os · low: How page tables and TLBs translate every address, what TLB reach means, and when 2 MiB or 1 GiB huge pages pay off on Linux.
›
Normalization layers
machine-learning · low: Standardize activations per batch, token or group with a learned scale so deep nets train fast and stably; RMSNorm in LLMs, BatchNorm in convnets.
›
Collective communication
ml-training · low: The family of group exchanges (broadcast, reduce, all-gather, reduce-scatter, all-to-all) and the latency-plus-bytes cost model that picks ring, tree or direct algorithms by message size and topology.
›
Convolutional networks
ml-training · low: Layers that slide a small shared kernel over a grid, giving translation equivariance and few parameters; still the most compute-efficient choice for many vision tasks.
›
Reverse-mode automatic differentiat…
ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.
›
Ring all-reduce
ml-training · low: Sum a buffer across N workers via reduce-scatter then all-gather around a ring; each link carries 2(N−1)/N of the buffer, which is bandwidth-optimal.
›
Floating-point error
numerics · low: How IEEE 754 rounding error accumulates and how to contain it: compensated (Kahan/Neumaier) and pairwise summation, error-free transformations with FMA, ULP comparisons.
›
Low precision number formats
numerics · low: Bf16, fp16, fp8 (E4M3, E5M2), MX block formats and int4: ranges, rounding, scaling
Runge-Kutta 4
numerics · low: The classic four-stage, fourth-order ODE step; error drops 16× when the step halves, with no derivatives or past steps needed.
›
Semi-implicit Euler
numerics · low: Update velocity from forces first, then position with the new velocity: explicit Euler cost, but symplectic and stable; the default game-physics integrator.
›
Copy for AI
Similar systems
Share the most building blocks with this one.
world-model
28 %
ml-weather-model
25 %
humanoid-robot
22 %
llm-pretraining
21 %
video-generation
20 %