mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
Frontier AI research
world-model
interactive generative world models
Genie 3, Cosmos
System
Techniques · 6
Primitives · 44
world-model
Data loading pipelines
machine-learning · medium: Turn files on disk into shuffled, sharded, resumable GPU batches fast enough that the accelerator never waits.
›
Near-duplicate detection with MinHa…
machine-learning · medium: Estimate set Jaccard similarity from k min-hashes and find similar pairs with banded LSH buckets; the standard dedup for web crawls, training corpora and sequence collections.
›
Flow matching
ml-training · medium: Training a velocity field along straight (rectified) paths from noise to data; sample by integrating an ODE
Knowledge distillation
ml-training · medium: Train a small student to match a large teacher's softened outputs (plus the hard labels) so it keeps most of the teacher's accuracy at a fraction of the cost.
›
Variational autoencoders
ml-training · medium: VAEs and latent autoencoders: ELBO, KL weighting, latent compression for diffusion models
Vision transformer
ml-training · medium: ViT: patch embeddings, position encodings, encoder blocks; vision-language connectors
Vector quantization
compression · low: Codebook and pyramid vector quantization of coefficient vectors
Attention variants
machine-learning · low: Multi-head, multi-query, grouped-query, sliding-window and latent attention; same softmax(QKᵀ/√d)V core, different trade-offs between quality and KV-cache size.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
Action tokenization
machine-learning · low: Discretize or compress continuous robot actions into tokens a sequence model can predict
Perceptual quality metrics (PSNR, S…
media · low: Full-reference scores that say how bad a compressed picture looks: PSNR (pixel error), SSIM (structure), VMAF (learned fusion), and when each one lies.
›
Scene cut detection
media · low: Detect shot changes from frame-difference statistics
Video codecs
media · low: H.264/HEVC/AV1 basics: frames, GOPs, hardware decode/encode APIs and latency settings
Classifier free guidance
ml-training · low: Mixing conditional and unconditional diffusion predictions with a guidance scale; condition dropout in training
Context parallelism
ml-training · low: Splitting the sequence dimension across GPUs (ring attention, Ulysses all-to-all) for long-context training
Diffusion models
ml-training · low: Train a network to undo Gaussian noise at every noise level, then generate by integrating from pure noise to data; the standard for images, video and audio.
›
Diffusion samplers
ml-training · low: DDPM/DDIM and ODE solvers (DPM-Solver, Euler, Heun) for few-step sampling from diffusion and flow models
FlashAttention
ml-training · low: Exact attention computed tile by tile in on-chip SRAM with online softmax, so the N×N score matrix never touches HBM; 2-4x faster, O(N) memory.
›
Mixed-precision training
ml-training · low: Run matmuls in bf16/fp16/fp8 on tensor cores while keeping master weights, optimizer state and reductions in fp32; about 2x speed, half the activation memory.
›
Paged KV cache
ml-training · low: Store each sequence's attention keys/values in fixed-size blocks mapped through a block table, like virtual memory pages; near-zero fragmentation and copy-on-write sharing.
›
ZeRO and FSDP sharding
ml-training · low: Data parallelism that shards optimizer state, gradients and weights across ranks and gathers weights just in time; per-GPU model memory drops from 16Ψ to 16Ψ/N.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
+ 28 more primitives
System
Techniques · 6
Primitives · 44
world-model
Data loading pipelines
machine-learning · medium: Turn files on disk into shuffled, sharded, resumable GPU batches fast enough that the accelerator never waits.
›
Near-duplicate detection with MinHa…
machine-learning · medium: Estimate set Jaccard similarity from k min-hashes and find similar pairs with banded LSH buckets; the standard dedup for web crawls, training corpora and sequence collections.
›
Flow matching
ml-training · medium: Training a velocity field along straight (rectified) paths from noise to data; sample by integrating an ODE
Knowledge distillation
ml-training · medium: Train a small student to match a large teacher's softened outputs (plus the hard labels) so it keeps most of the teacher's accuracy at a fraction of the cost.
›
Variational autoencoders
ml-training · medium: VAEs and latent autoencoders: ELBO, KL weighting, latent compression for diffusion models
Vision transformer
ml-training · medium: ViT: patch embeddings, position encodings, encoder blocks; vision-language connectors
Vector quantization
compression · low: Codebook and pyramid vector quantization of coefficient vectors
Attention variants
machine-learning · low: Multi-head, multi-query, grouped-query, sliding-window and latent attention; same softmax(QKᵀ/√d)V core, different trade-offs between quality and KV-cache size.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
Action tokenization
machine-learning · low: Discretize or compress continuous robot actions into tokens a sequence model can predict
Perceptual quality metrics (PSNR, S…
media · low: Full-reference scores that say how bad a compressed picture looks: PSNR (pixel error), SSIM (structure), VMAF (learned fusion), and when each one lies.
›
Scene cut detection
media · low: Detect shot changes from frame-difference statistics
Video codecs
media · low: H.264/HEVC/AV1 basics: frames, GOPs, hardware decode/encode APIs and latency settings
Classifier free guidance
ml-training · low: Mixing conditional and unconditional diffusion predictions with a guidance scale; condition dropout in training
Context parallelism
ml-training · low: Splitting the sequence dimension across GPUs (ring attention, Ulysses all-to-all) for long-context training
Diffusion models
ml-training · low: Train a network to undo Gaussian noise at every noise level, then generate by integrating from pure noise to data; the standard for images, video and audio.
›
Diffusion samplers
ml-training · low: DDPM/DDIM and ODE solvers (DPM-Solver, Euler, Heun) for few-step sampling from diffusion and flow models
FlashAttention
ml-training · low: Exact attention computed tile by tile in on-chip SRAM with online softmax, so the N×N score matrix never touches HBM; 2-4x faster, O(N) memory.
›
Mixed-precision training
ml-training · low: Run matmuls in bf16/fp16/fp8 on tensor cores while keeping master weights, optimizer state and reductions in fp32; about 2x speed, half the activation memory.
›
Paged KV cache
ml-training · low: Store each sequence's attention keys/values in fixed-size blocks mapped through a block table, like virtual memory pages; near-zero fragmentation and copy-on-write sharing.
›
ZeRO and FSDP sharding
ml-training · low: Data parallelism that shards optimizer state, gradients and weights across ranks and gathers weights just in time; per-GPU model memory drops from 16Ψ to 16Ψ/N.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
mmap and the page cache
hardware-os · low: How read() and mmap reach file data through the kernel page cache, what readahead and madvise do, and why databases use their own buffer pool instead.
›
Sequence packing
ml-training · low: Packing variable-length sequences into fixed-size batches with attention masks to avoid padding waste
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Fast hash functions
data-structures · low: Non-cryptographic hashes built from 64×64→128-bit multiplies and xor-shifts; pick by key size, trust level and whether the hash is persisted.
›
Rolling hash
data-structures · low: Rabin-Karp polynomial rolling hashes and winnowing for substring fingerprints and document matching
Swiss table
data-structures · low: Open-addressing hash table that filters 8-16 slots at once with 1-byte hash tags and SIMD, giving fast lookups at 87.5 % load.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Adaptive Runge-Kutta (Dormand-Princ…
numerics · low: An embedded 5(4) Runge-Kutta pair estimates each step's error for free and resizes the step to meet a tolerance; the default non-stiff solver.
›
Explicit Euler
numerics · low: Step an ODE forward by its current slope, y += h·f(t, y); first-order, one evaluation per step, the baseline every other integrator is measured against.
›
Data augmentation
machine-learning · low: Generate label-preserving variants of each training example on the fly to enlarge the effective dataset and bake in invariances.
›
Token sampling
ml-training · low: Turn a vector of scores into one chosen item with temperature, top-k, top-p, min-p, repetition penalties and a seed.
›
Convolutional networks
ml-training · low: Layers that slide a small shared kernel over a grid, giving translation equivariance and few parameters; still the most compute-efficient choice for many vision tasks.
›
Reverse-mode automatic differentiat…
ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.
›
Embeddings
machine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.
›
Reference counting
compilers · low: Keep a count of owners in each object, free it when the count hits zero; deterministic and local, but pays on every copy and leaks cycles.
›
Gpu memory coalescing banking
hardware-os · low: Coalesced global loads, shared memory bank conflicts, L2 and HBM bandwidth on GPUs
Virtual memory and the TLB
hardware-os · low: How page tables and TLBs translate every address, what TLB reach means, and when 2 MiB or 1 GiB huge pages pay off on Linux.
›
CUDA and GPU languages
languages · low: Programming GPUs: thousands of lockstep threads, bound by memory bandwidth; libraries first, Triton for fused kernels, CUDA C++ for peak speed.
›
Normalization layers
machine-learning · low: Standardize activations per batch, token or group with a learned scale so deep nets train fast and stably; RMSNorm in LLMs, BatchNorm in convnets.
›
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Collective communication
ml-training · low: The family of group exchanges (broadcast, reduce, all-gather, reduce-scatter, all-to-all) and the latency-plus-bytes cost model that picks ring, tree or direct algorithms by message size and topology.
›
Cuda graphs
ml-training · low: Capturing a sequence of GPU kernel launches once and replaying it to remove CPU launch overhead
Kernel fusion
ml-training · low: Merge chains of memory-bound GPU ops into one kernel, so intermediates stay in registers instead of round-tripping through HBM.
›
Ring all-reduce
ml-training · low: Sum a buffer across N workers via reduce-scatter then all-gather around a ring; each link carries 2(N−1)/N of the buffer, which is bandwidth-optimal.
›
Tiled matrix multiplication
ml-training · low: Block GEMM into tiles held in shared memory and registers, so each byte loaded from DRAM feeds many FMAs; the core of fast GEMM.
›
Floating-point error
numerics · low: How IEEE 754 rounding error accumulates and how to contain it: compensated (Kahan/Neumaier) and pairwise summation, error-free transformations with FMA, ULP comparisons.
›
Low precision number formats
numerics · low: Bf16, fp16, fp8 (E4M3, E5M2), MX block formats and int4: ranges, rounding, scaling
Runge-Kutta 4
numerics · low: The classic four-stage, fourth-order ODE step; error drops 16× when the step halves, with no derivatives or past steps needed.
›
Copy for AI
Similar systems
Share the most building blocks with this one.
video-generation
48 %
image-generation
36 %
robot-policy
28 %
ml-weather-model
24 %
protein-structure
20 %