mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
AI and ML infrastructure
model-compression
quantization and distillation pipeline
GPTQ, AWQ
System
Techniques · 2
Primitives · 18
model-compression
Knowledge distillation
ml-training · medium: Train a small student to match a large teacher's softened outputs (plus the hard labels) so it keeps most of the teacher's accuracy at a fraction of the cost.
›
Network pruning
ml-training · medium: Removing weights/heads/channels (magnitude, structured, 2:4 sparsity) with retraining
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Bit packing and PFOR
compression · low: Store blocks of integers with exactly b bits each and patch rare outliers separately (PFOR); the fastest integer codec family, billions of values per second.
›
Evaluation metrics
machine-learning · low: Precision, recall, ROC AUC, average precision, log loss, NDCG and pass@k computed correctly, plus paired confidence intervals for comparing models.
›
PCA and SVD
machine-learning · low: Find orthogonal directions of maximum variance by an SVD of the centred data; randomized SVD gets the top k components in a few passes.
›
Lora adapters
ml-training · low: Low-rank adapter matrices for fine-tuning; QLoRA; serving many adapters on one base model
Mixed-precision training
ml-training · low: Run matmuls in bf16/fp16/fp8 on tensor cores while keeping master weights, optimizer state and reductions in fp32; about 2x speed, half the activation memory.
›
Quantization aware training
ml-training · low: Fake-quantize weights/activations in training with straight-through gradients so int8/int4 models keep accuracy
Weight quantization
ml-training · low: Store weights as 8/4-bit integers with per-group scales; decode is memory-bound, so 4-bit weights give about 3-4x tokens/s and fit 4x larger models.
›
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Data augmentation
machine-learning · low: Generate label-preserving variants of each training example on the fly to enlarge the effective dataset and bake in invariances.
›
Token sampling
ml-training · low: Turn a vector of scores into one chosen item with temperature, top-k, top-p, min-p, repetition penalties and a seed.
›
Regularization
machine-learning · low: Techniques that trade training fit for generalization: weight decay, dropout, early stopping, label smoothing and data mixing, with the exact update rules and defaults.
›
Structured sparsity
ml-training · low: 2:4 and block sparsity patterns that hardware can accelerate; pruning to fit them
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Floating-point error
numerics · low: How IEEE 754 rounding error accumulates and how to contain it: compensated (Kahan/Neumaier) and pairwise summation, error-free transformations with FMA, ULP comparisons.
›
+ 2 more primitives
System
Techniques · 2
Primitives · 18
model-compression
Knowledge distillation
ml-training · medium: Train a small student to match a large teacher's softened outputs (plus the hard labels) so it keeps most of the teacher's accuracy at a fraction of the cost.
›
Network pruning
ml-training · medium: Removing weights/heads/channels (magnitude, structured, 2:4 sparsity) with retraining
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Bit packing and PFOR
compression · low: Store blocks of integers with exactly b bits each and patch rare outliers separately (PFOR); the fastest integer codec family, billions of values per second.
›
Evaluation metrics
machine-learning · low: Precision, recall, ROC AUC, average precision, log loss, NDCG and pass@k computed correctly, plus paired confidence intervals for comparing models.
›
PCA and SVD
machine-learning · low: Find orthogonal directions of maximum variance by an SVD of the centred data; randomized SVD gets the top k components in a few passes.
›
Lora adapters
ml-training · low: Low-rank adapter matrices for fine-tuning; QLoRA; serving many adapters on one base model
Mixed-precision training
ml-training · low: Run matmuls in bf16/fp16/fp8 on tensor cores while keeping master weights, optimizer state and reductions in fp32; about 2x speed, half the activation memory.
›
Quantization aware training
ml-training · low: Fake-quantize weights/activations in training with straight-through gradients so int8/int4 models keep accuracy
Weight quantization
ml-training · low: Store weights as 8/4-bit integers with per-group scales; decode is memory-bound, so 4-bit weights give about 3-4x tokens/s and fit 4x larger models.
›
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Data augmentation
machine-learning · low: Generate label-preserving variants of each training example on the fly to enlarge the effective dataset and bake in invariances.
›
Token sampling
ml-training · low: Turn a vector of scores into one chosen item with temperature, top-k, top-p, min-p, repetition penalties and a seed.
›
Regularization
machine-learning · low: Techniques that trade training fit for generalization: weight decay, dropout, early stopping, label smoothing and data mixing, with the exact update rules and defaults.
›
Structured sparsity
ml-training · low: 2:4 and block sparsity patterns that hardware can accelerate; pruning to fit them
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
AdamW
ml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.
›
Floating-point error
numerics · low: How IEEE 754 rounding error accumulates and how to contain it: compensated (Kahan/Neumaier) and pairwise summation, error-free transformations with FMA, ULP comparisons.
›
Low precision number formats
numerics · low: Bf16, fp16, fp8 (E4M3, E5M2), MX block formats and int4: ranges, rounding, scaling
Bootstrap confidence intervals
statistics · low: Resampling-based confidence intervals for arbitrary statistics
Copy for AI
Similar systems
Share the most building blocks with this one.
nnue-training
18 %
image-generation
17 %
safety-classifiers
16 %