← Back
Home › Systems › AI and ML infrastructure

ml-compiler

ML compiler

XLA, TVM, Triton

SystemTechniques · 14Primitives · 34 ml-compiler Dataflow analysiscompilers · medium: Compute a fact per program point (live variables, reaching definitions, available expressions) by iterating transfer functions over the CFG to a fixed point.›Instruction selectioncompilers · medium: Cover the IR with target instruction patterns at least cost; greedy maximal munch is simple, dynamic programming over a tree or DAG is optimal.›JIT compilationcompilers · medium: Run cold code in a cheap tier, count executions, and compile only what gets hot, trading warm-up time and memory for peak speed.›Loop optimizationscompilers · medium: LICM, induction variables, strength reduction, unrolling, loop interchange and tilingRegister allocationcompilers · medium: Map unlimited virtual registers onto a few machine registers, spilling the rest to the stack; linear scan is fast, graph coloring and splitting allocators give better code.›Kernel autotuningml-training · medium: Searching tile sizes and launch parameters per shape/hardware, caching the best configsOperator graph partitioningml-training · medium: Splitting a model graph across CPU/GPU/NPU backends by supported ops and transfer costSSA formcompilers · low: Give every variable exactly one definition, merging at joins with phi functions; def-use becomes explicit and optimizations turn sparse and cheap.›Roofline modelhardware-os · low: Bound a kernel's speed by min(peak FLOP/s, arithmetic intensity × memory bandwidth) to know whether to optimize compute or data movement.›Tiled matrix multiplicationml-training · low: Block GEMM into tiles held in shared memory and registers, so each byte loaded from DRAM feeds many FMAs; the core of fast GEMM.›Shape inferencecompilers · low: Propagating tensor shapes and dtypes through an operator graph, with symbolic dimensionsTensor memory planningml-training · low: Static buffer reuse from tensor lifetimes so a graph runs in a fixed arena with minimal peak memoryE graphscompilers · low: Equality saturation: e-graphs compactly hold many equivalent programs, then extract the cheapestSIMD programminghardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.›CUDA and GPU languageslanguages · low: Programming GPUs: thousands of lockstep threads, bound by memory bandwidth; libraries first, Triton for fused kernels, CUDA C++ for peak speed.›Kernel fusionml-training · low: Merge chains of memory-bound GPU ops into one kernel, so intermediates stay in registers instead of round-tripping through HBM.›Tensor layout transformsml-training · low: Choosing and converting tensor layouts (NCHW/NHWC, blocked, tiled) for kernel efficiencyControl-flow graphcompilers · low: Split a linear instruction list into basic blocks and connect them by jumps; the graph every optimizer, SSA builder and dataflow pass walks.›Topological sortgraphs · low: Order the vertices of a DAG so every edge points forward; the core of build systems, schedulers and dependency resolution, with cycle detection for free.›Machine code emissioncompilers · low: Encode instructions to bytes for x86-64 and AArch64, branch relaxation, constant poolsTree pattern matchingcompilers · low: Matching AST patterns with metavariablesBytecode interpretercompilers · low: Register/stack VM dispatch loop for executing compiled plans or rules (SQLite VDBE, nftables)Executable memory under W^Xcompilers · low: How JITs and code generators emit machine code under W^X: RW to RX flips, dual mappings, per-thread write toggles, and the instruction-cache flush.›+ 7 more techniques+ 18 more primitives

Similar systems

Share the most building blocks with this one.

autodiff-framework 35 %tensor-runtime 31 %on-device-inference 22 %fft-library 19 %cuda-platform 19 %