mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
Numerical and mathematical computing
autodiff-framework
automatic differentiation
JAX, Enzyme
System
Techniques · 14
Primitives · 40
autodiff-framework
Alias analysis
compilers · medium: Decide whether two pointers may refer to the same memory (Andersen, Steensgaard, type-based) to enable optimizations
Dataflow analysis
compilers · medium: Compute a fact per program point (live variables, reaching definitions, available expressions) by iterating transfer functions over the CFG to a fixed point.
›
Function inlining
compilers · medium: Inline call sites by cost model and call-graph order; enables downstream optimizations
Instruction selection
compilers · medium: Cover the IR with target instruction patterns at least cost; greedy maximal munch is simple, dynamic programming over a tree or DAG is optimal.
›
JIT compilation
compilers · medium: Run cold code in a cheap tier, count executions, and compile only what gets hot, trading warm-up time and memory for peak speed.
›
Kernel autotuning
ml-training · medium: Searching tile sizes and launch parameters per shape/hardware, caching the best configs
Tensor compiler scheduling
ml-training · medium: Tile, fuse and lay out tensor ops for a target (XLA, TVM, Triton schedules), plus autotuning
SSA form
compilers · low: Give every variable exactly one definition, merging at joins with phi functions; def-use becomes explicit and optimizations turn sparse and cheap.
›
Kernel fusion
ml-training · low: Merge chains of memory-bound GPU ops into one kernel, so intermediates stay in registers instead of round-tripping through HBM.
›
Tensor layout transforms
ml-training · low: Choosing and converting tensor layouts (NCHW/NHWC, blocked, tiled) for kernel efficiency
Tensor memory planning
ml-training · low: Static buffer reuse from tensor lifetimes so a graph runs in a fixed arena with minimal peak memory
Shape inference
compilers · low: Propagating tensor shapes and dtypes through an operator graph, with symbolic dimensions
Python
languages · low: The fastest language to write and the home of ML and data science; slow as an interpreter, fast when C/CUDA libraries do the work.
›
Collective communication
ml-training · low: The family of group exchanges (broadcast, reduce, all-gather, reduce-scatter, all-to-all) and the latency-plus-bytes cost model that picks ring, tree or direct algorithms by message size and topology.
›
Cuda graphs
ml-training · low: Capturing a sequence of GPU kernel launches once and replaying it to remove CPU launch overhead
Gradient checkpointing
ml-training · low: Save only some activations in the forward pass and recompute the rest during backward, trading about 33% extra compute for O(√n) activation memory.
›
Reverse-mode automatic differentiat…
ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.
›
Tensor parallelism
ml-training · low: Split each layer's weight matrices across GPUs (Megatron-style column then row splits) so one layer's math runs on t devices with one all-reduce per block.
›
Union-find
graphs · low: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.
›
Control-flow graph
compilers · low: Split a linear instruction list into basic blocks and connect them by jumps; the graph every optimizer, SSA builder and dataflow pass walks.
›
Topological sort
graphs · low: Order the vertices of a DAG so every edge points forward; the core of build systems, schedulers and dependency resolution, with cycle detection for free.
›
Strongly connected components
graphs · low: Tarjan/Kosaraju SCCs for cycle detection in dependency graphs
Machine code emission
compilers · low: Encode instructions to bytes for x86-64 and AArch64, branch relaxation, constant pools
+ 7 more techniques
+ 24 more primitives
System
Techniques · 14
Primitives · 40
autodiff-framework
Alias analysis
compilers · medium: Decide whether two pointers may refer to the same memory (Andersen, Steensgaard, type-based) to enable optimizations
Dataflow analysis
compilers · medium: Compute a fact per program point (live variables, reaching definitions, available expressions) by iterating transfer functions over the CFG to a fixed point.
›
Function inlining
compilers · medium: Inline call sites by cost model and call-graph order; enables downstream optimizations
Instruction selection
compilers · medium: Cover the IR with target instruction patterns at least cost; greedy maximal munch is simple, dynamic programming over a tree or DAG is optimal.
›
JIT compilation
compilers · medium: Run cold code in a cheap tier, count executions, and compile only what gets hot, trading warm-up time and memory for peak speed.
›
Kernel autotuning
ml-training · medium: Searching tile sizes and launch parameters per shape/hardware, caching the best configs
Tensor compiler scheduling
ml-training · medium: Tile, fuse and lay out tensor ops for a target (XLA, TVM, Triton schedules), plus autotuning
Call graph construction
compilers · medium: Building call graphs with dynamic dispatch resolution (CHA, RTA)
Deoptimization
compilers · medium: Speculate with guards, and on failure rebuild the slow-tier frames from compile-time metadata and resume there, so optimized code may assume the common case.
›
Inline caches
compilers · medium: Per-call-site cache of "this object shape, that slot or target", turning dynamic property and method lookup into a compare and a load.
›
Register allocation
compilers · medium: Map unlimited virtual registers onto a few machine registers, spilling the rest to the stack; linear scan is fast, graph coloring and splitting allocators give better code.
›
Stack unwinding
compilers · medium: Frame-pointer, DWARF CFI and compact unwinding of call stacks
Graph coloring
graphs · medium: Greedy/DSatur colouring for conflict-free assignment such as radio channels
Debug info (DWARF)
compilers · medium: DWARF stores a tree of typed entries (DIEs), a line-number bytecode, unwind tables and location expressions that map machine state back to source; pick it for ELF/Mach-O debug builds.
›
SSA form
compilers · low: Give every variable exactly one definition, merging at joins with phi functions; def-use becomes explicit and optimizations turn sparse and cheap.
›
Kernel fusion
ml-training · low: Merge chains of memory-bound GPU ops into one kernel, so intermediates stay in registers instead of round-tripping through HBM.
›
Tensor layout transforms
ml-training · low: Choosing and converting tensor layouts (NCHW/NHWC, blocked, tiled) for kernel efficiency
Tensor memory planning
ml-training · low: Static buffer reuse from tensor lifetimes so a graph runs in a fixed arena with minimal peak memory
Shape inference
compilers · low: Propagating tensor shapes and dtypes through an operator graph, with symbolic dimensions
Python
languages · low: The fastest language to write and the home of ML and data science; slow as an interpreter, fast when C/CUDA libraries do the work.
›
Collective communication
ml-training · low: The family of group exchanges (broadcast, reduce, all-gather, reduce-scatter, all-to-all) and the latency-plus-bytes cost model that picks ring, tree or direct algorithms by message size and topology.
›
Cuda graphs
ml-training · low: Capturing a sequence of GPU kernel launches once and replaying it to remove CPU launch overhead
Gradient checkpointing
ml-training · low: Save only some activations in the forward pass and recompute the rest during backward, trading about 33% extra compute for O(√n) activation memory.
›
Reverse-mode automatic differentiat…
ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.
›
Tensor parallelism
ml-training · low: Split each layer's weight matrices across GPUs (Megatron-style column then row splits) so one layer's math runs on t devices with one all-reduce per block.
›
Union-find
graphs · low: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.
›
Control-flow graph
compilers · low: Split a linear instruction list into basic blocks and connect them by jumps; the graph every optimizer, SSA builder and dataflow pass walks.
›
Topological sort
graphs · low: Order the vertices of a DAG so every edge points forward; the core of build systems, schedulers and dependency resolution, with cycle detection for free.
›
Strongly connected components
graphs · low: Tarjan/Kosaraju SCCs for cycle detection in dependency graphs
Machine code emission
compilers · low: Encode instructions to bytes for x86-64 and AArch64, branch relaxation, constant pools
Tree pattern matching
compilers · low: Matching AST patterns with metavariables
Bytecode interpreter
compilers · low: Register/stack VM dispatch loop for executing compiled plans or rules (SQLite VDBE, nftables)
Executable memory under W^X
compilers · low: How JITs and code generators emit machine code under W^X: RW to RX flips, dual mappings, per-thread write toggles, and the instruction-cache flush.
›
Roofline model
hardware-os · low: Bound a kernel's speed by min(peak FLOP/s, arithmetic intensity × memory bandwidth) to know whether to optimize compute or data movement.
›
GPU compute shaders
ml-training · low: The vendor-neutral GPU launch model of Metal, Vulkan and WebGPU: a grid of workgroups, shared memory, barriers and subgroups, driven by recorded command buffers.
›
Tiled matrix multiplication
ml-training · low: Block GEMM into tiles held in shared memory and registers, so each byte loaded from DRAM feeds many FMAs; the core of fast GEMM.
›
Robust statistics
statistics · low: Location and scale estimates (median, MAD, trimmed mean, Huber M-estimator) that stay correct when a minority of the data is garbage.
›
Name resolution
compilers · low: Bind every use of a name to its declaration with a scope stack, handling shadowing, hoisting and closure capture in one AST walk.
›
Hidden classes (shapes)
compilers · low: Objects that gained the same properties in the same order share one layout descriptor, so a property is an index and the layout is one comparable id.
›
Interval tree
data-structures · low: Index of ranges that answers "which intervals overlap this point or range?" in O(log n + k); static sorted-array form is the compact, cache-friendly choice.
›
Calling conventions abi
compilers · low: System V, Windows x64 and AAPCS64 ABIs: argument registers, stack frames, struct passing, FFI
Executable formats elf pe
compilers · low: ELF, PE and Mach-O: sections, segments, symbol tables, relocations, dynamic loading
d-ary heap
data-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.
›
BFS and DFS
graphs · low: The two basic graph traversals; BFS gives hop distances and levels, DFS gives discovery/finish order for cycles, components and topological order.
›
Varint (LEB128)
compression · low: Variable-length integers in 7-bit groups with a continuation bit; small numbers take 1 byte, the wire format of Protobuf, WebAssembly and DWARF.
›
Dominator tree
compilers · low: Dominators via Lengauer-Tarjan or Cooper-Harvey-Kennedy; dominance frontiers for SSA
String interning
data-structures · low: Store each distinct string once and refer to it by id
Swiss table
data-structures · low: Open-addressing hash table that filters 8-16 slots at once with 1-byte hash tags and SIMD, giving fast lookups at 87.5 % load.
›
CPU caches and the memory hierarchy
hardware-os · low: Sizes, line widths and latencies of L1/L2/L3/DRAM on modern x86 and Apple M-series, and the data-layout rules that follow from them.
›
Gpu memory coalescing banking
hardware-os · low: Coalesced global loads, shared memory bank conflicts, L2 and HBM bandwidth on GPUs
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
CUDA and GPU languages
languages · low: Programming GPUs: thousands of lockstep threads, bound by memory bandwidth; libraries first, Triton for fused kernels, CUDA C++ for peak speed.
›
Ring all-reduce
ml-training · low: Sum a buffer across N workers via reduce-scatter then all-gather around a ring; each link carries 2(N−1)/N of the buffer, which is bandwidth-optimal.
›
Sparse matrices (CSR)
numerics · low: Compressed sparse row stores only nonzeros as row pointers, column indices and values; SpMV streams them once, so it is memory-bandwidth bound.
›
Copy for AI
Similar systems
Share the most building blocks with this one.
ml-compiler
35 %
tensor-runtime
21 %
cuda-platform
18 %
compiler
15 %