mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
AI applications
video-generation
video generation
Veo, Sora
System
Techniques · 5
Primitives · 36
video-generation
Flow matching
ml-training · medium: Training a velocity field along straight (rectified) paths from noise to data; sample by integrating an ODE
Image model training
ml-training · medium: Proven setups for training image classifiers and diffusion models: optimizers and LRs, augmentation, label smoothing, EMA, resolution schedules and aspect buckets.
›
Knowledge distillation
ml-training · medium: Train a small student to match a large teacher's softened outputs (plus the hard labels) so it keeps most of the teacher's accuracy at a fraction of the cost.
›
Variational autoencoders
ml-training · medium: VAEs and latent autoencoders: ELBO, KL weighting, latent compression for diffusion models
Invisible watermarking
security · medium: Embedding robust imperceptible marks in images/audio/video or token sampling for later detection
Diffusion models
ml-training · low: Train a network to undo Gaussian noise at every noise level, then generate by integrating from pure noise to data; the standard for images, video and audio.
›
Attention variants
machine-learning · low: Multi-head, multi-query, grouped-query, sliding-window and latent attention; same softmax(QKᵀ/√d)V core, different trade-offs between quality and KV-cache size.
›
Rotary position embedding (RoPE)
machine-learning · low: Encode position by rotating pairs of query and key dimensions by position-dependent angles, so attention scores depend only on relative distance; the LLM default.
›
Video codecs
media · low: H.264/HEVC/AV1 basics: frames, GOPs, hardware decode/encode APIs and latency settings
Classifier free guidance
ml-training · low: Mixing conditional and unconditional diffusion predictions with a guidance scale; condition dropout in training
Context parallelism
ml-training · low: Splitting the sequence dimension across GPUs (ring attention, Ulysses all-to-all) for long-context training
Diffusion samplers
ml-training · low: DDPM/DDIM and ODE solvers (DPM-Solver, Euler, Heun) for few-step sampling from diffusion and flow models
FlashAttention
ml-training · low: Exact attention computed tile by tile in on-chip SRAM with online softmax, so the N×N score matrix never touches HBM; 2-4x faster, O(N) memory.
›
Tensor parallelism
ml-training · low: Split each layer's weight matrices across GPUs (Megatron-style column then row splits) so one layer's math runs on t devices with one all-reduce per block.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
ZeRO and FSDP sharding
ml-training · low: Data parallelism that shards optimizer state, gradients and weights across ranks and gathers weights just in time; per-GPU model memory drops from 16Ψ to 16Ψ/N.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Adaptive Runge-Kutta (Dormand-Princ…
numerics · low: An embedded 5(4) Runge-Kutta pair estimates each step's error for free and resizes the step to meet a tolerance; the default non-stiff solver.
›
Explicit Euler
numerics · low: Step an ODE forward by its current slope, y += h·f(t, y); first-order, one evaluation per step, the baseline every other integrator is measured against.
›
Data augmentation
machine-learning · low: Generate label-preserving variants of each training example on the fly to enlarge the effective dataset and bake in invariances.
›
Normalization layers
machine-learning · low: Standardize activations per batch, token or group with a learned scale so deep nets train fast and stably; RMSNorm in LLMs, BatchNorm in convnets.
›
+ 20 more primitives
System
Techniques · 5
Primitives · 36
video-generation
Flow matching
ml-training · medium: Training a velocity field along straight (rectified) paths from noise to data; sample by integrating an ODE
Image model training
ml-training · medium: Proven setups for training image classifiers and diffusion models: optimizers and LRs, augmentation, label smoothing, EMA, resolution schedules and aspect buckets.
›
Knowledge distillation
ml-training · medium: Train a small student to match a large teacher's softened outputs (plus the hard labels) so it keeps most of the teacher's accuracy at a fraction of the cost.
›
Variational autoencoders
ml-training · medium: VAEs and latent autoencoders: ELBO, KL weighting, latent compression for diffusion models
Invisible watermarking
security · medium: Embedding robust imperceptible marks in images/audio/video or token sampling for later detection
Diffusion models
ml-training · low: Train a network to undo Gaussian noise at every noise level, then generate by integrating from pure noise to data; the standard for images, video and audio.
›
Attention variants
machine-learning · low: Multi-head, multi-query, grouped-query, sliding-window and latent attention; same softmax(QKᵀ/√d)V core, different trade-offs between quality and KV-cache size.
›
Rotary position embedding (RoPE)
machine-learning · low: Encode position by rotating pairs of query and key dimensions by position-dependent angles, so attention scores depend only on relative distance; the LLM default.
›
Video codecs
media · low: H.264/HEVC/AV1 basics: frames, GOPs, hardware decode/encode APIs and latency settings
Classifier free guidance
ml-training · low: Mixing conditional and unconditional diffusion predictions with a guidance scale; condition dropout in training
Context parallelism
ml-training · low: Splitting the sequence dimension across GPUs (ring attention, Ulysses all-to-all) for long-context training
Diffusion samplers
ml-training · low: DDPM/DDIM and ODE solvers (DPM-Solver, Euler, Heun) for few-step sampling from diffusion and flow models
FlashAttention
ml-training · low: Exact attention computed tile by tile in on-chip SRAM with online softmax, so the N×N score matrix never touches HBM; 2-4x faster, O(N) memory.
›
Tensor parallelism
ml-training · low: Split each layer's weight matrices across GPUs (Megatron-style column then row splits) so one layer's math runs on t devices with one all-reduce per block.
›
Transformer block
ml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.
›
ZeRO and FSDP sharding
ml-training · low: Data parallelism that shards optimizer state, gradients and weights across ranks and gathers weights just in time; per-GPU model memory drops from 16Ψ to 16Ψ/N.
›
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Adaptive Runge-Kutta (Dormand-Princ…
numerics · low: An embedded 5(4) Runge-Kutta pair estimates each step's error for free and resizes the step to meet a tolerance; the default non-stiff solver.
›
Explicit Euler
numerics · low: Step an ODE forward by its current slope, y += h·f(t, y); first-order, one evaluation per step, the baseline every other integrator is measured against.
›
Data augmentation
machine-learning · low: Generate label-preserving variants of each training example on the fly to enlarge the effective dataset and bake in invariances.
›
Normalization layers
machine-learning · low: Standardize activations per batch, token or group with a learned scale so deep nets train fast and stably; RMSNorm in LLMs, BatchNorm in convnets.
›
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Convolutional networks
ml-training · low: Layers that slide a small shared kernel over a grid, giving translation equivariance and few parameters; still the most compute-efficient choice for many vision tasks.
›
Learning-rate warmup and cosine dec…
ml-training · low: Ramp the learning rate up linearly for the first ~1% of steps, then decay it along a half cosine to ~10% of peak.
›
Mixed-precision training
ml-training · low: Run matmuls in bf16/fp16/fp8 on tensor cores while keeping master weights, optimizer state and reductions in fp32; about 2x speed, half the activation memory.
›
Token sampling
ml-training · low: Turn a vector of scores into one chosen item with temperature, top-k, top-p, min-p, repetition penalties and a seed.
›
Vector quantization
compression · low: Codebook and pyramid vector quantization of coefficient vectors
Reverse-mode automatic differentiat…
ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.
›
Discrete cosine transform
numerics · low: Fast integer DCT/DST block transforms used by image and video codecs
PRNGs (PCG, xoshiro)
numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.
›
Forward error correction
signal-processing · low: Add redundancy (LDPC, Reed-Solomon, XOR parity) so receivers repair loss without retransmit
Gpu memory coalescing banking
hardware-os · low: Coalesced global loads, shared memory bank conflicts, L2 and HBM bandwidth on GPUs
CUDA and GPU languages
languages · low: Programming GPUs: thousands of lockstep threads, bound by memory bandwidth; libraries first, Triton for fused kernels, CUDA C++ for peak speed.
›
Collective communication
ml-training · low: The family of group exchanges (broadcast, reduce, all-gather, reduce-scatter, all-to-all) and the latency-plus-bytes cost model that picks ring, tree or direct algorithms by message size and topology.
›
Cuda graphs
ml-training · low: Capturing a sequence of GPU kernel launches once and replaying it to remove CPU launch overhead
Kernel fusion
ml-training · low: Merge chains of memory-bound GPU ops into one kernel, so intermediates stay in registers instead of round-tripping through HBM.
›
Ring all-reduce
ml-training · low: Sum a buffer across N workers via reduce-scatter then all-gather around a ring; each link carries 2(N−1)/N of the buffer, which is bandwidth-optimal.
›
Tiled matrix multiplication
ml-training · low: Block GEMM into tiles held in shared memory and registers, so each byte loaded from DRAM feeds many FMAs; the core of fast GEMM.
›
Floating-point error
numerics · low: How IEEE 754 rounding error accumulates and how to contain it: compensated (Kahan/Neumaier) and pairwise summation, error-free transformations with FMA, ULP comparisons.
›
Low precision number formats
numerics · low: Bf16, fp16, fp8 (E4M3, E5M2), MX block formats and int4: ranges, rounding, scaling
Runge-Kutta 4
numerics · low: The classic four-stage, fourth-order ODE step; error drops 16× when the step halves, with no derivatives or past steps needed.
›
Copy for AI
Similar systems
Share the most building blocks with this one.
image-generation
50 %
world-model
48 %
ml-weather-model
24 %
robot-policy
20 %
llm-pretraining
19 %