← Back
Home › Systems › Hardware and datacenter

rack-scale-system

rack-scale AI system

NVIDIA GB200 NVL72, AMD Instinct racks, AWS Trn2 UltraServer

SystemTechniques · 20Primitives · 55 rack-scale-system Datacenter coolingcloud · medium: Hot/cold aisle containment, free cooling, direct liquid cooling and PUE mathPower oversubscription cappingcloud · medium: Oversubscribe datacenter power with telemetry and fast capping (RAPL, priority-based throttling)Ups power distributioncloud · medium: UPS, PDUs, busways and redundancy schemes (N+1, 2N) for datacenter powerPipe network hydraulicsembedded · medium: Solve pipe-network flows and heads (Hazen-Williams, gradient method) for water systemsCache coherence protocolshardware-os · medium: MESI and MOESI, snooping vs directories: how cores keep caches consistent and what that costsMemory semantic interconnecthardware-os · medium: Load/store fabrics (CXL, NVLink-style) giving hosts cache-coherent or pooled remote memory with address windows and device-managed coherenceGpu health checksml-training · medium: Detecting bad GPUs/links before and during jobs: ECC/XID errors, NCCL tests, throughput probes, drainingPID controllerrobotics · low: Drive a measured value to a setpoint with three terms on the error: proportional, integral, derivative. The default feedback loop for motors, heaters, drones and gimbals.›Power management dvfshardware-os · low: DVFS governors, P- and C-states, race-to-idle, thermal throttlingCollective communicationml-training · low: The family of group exchanges (broadcast, reduce, all-gather, reduce-scatter, all-to-all) and the latency-plus-bytes cost model that picks ring, tree or direct algorithms by message size and topology.›Network topologiesnetworking · low: Fat-tree and Clos, dragonfly, torus and rail-optimized topologies: bisection bandwidth and diameterBmc out of band managementhardware-os · low: IPMI/Redfish BMC control: power, boot device, sensors, serial-over-LANDMA and IOMMUhardware-os · low: How devices move data by direct memory access, and how an IOMMU gives each device its own address space, faults on stray accesses and lets user space or guests own a device safely.›Tensor parallelismml-training · low: Split each layer's weight matrices across GPUs (Megatron-style column then row splits) so one layer's math runs on t devices with one all-reduce per block.›Credit based flow controlnetworking · low: Receiver-granted credits/windows bound in-flight data (HTTP/2 windows, AMQP prefetch, backpressure)Rdmanetworking · low: RDMA verbs over InfiniBand/RoCE: queue pairs, memory registration, one-sided opsForward error correctionsignal-processing · low: Add redundancy (LDPC, Reed-Solomon, XOR parity) so receivers repair loss without retransmitBacnet protocolembedded · low: BACnet objects, properties and services over IP/MSTP for building automationModbus protocolembedded · low: Modbus RTU/TCP registers and function codes with polling and CRCFixed-rate control loopembedded · low: Run a controller at an exact period using absolute deadlines and a fixed dt, so jitter never accumulates and the math stays valid.›Newton root findingnumerics · low: Linearize at the guess and jump to the tangent root; quadratic convergence near a simple root, made robust by brackets (1D) or line search.›Sparse matrices (CSR)numerics · low: Compressed sparse row stores only nonzeros as row pointers, column indices and values; SpMV streams them once, so it is memory-bandwidth bound.›Memory orderingconcurrency · low: What Relaxed, Acquire, Release and SeqCst atomics actually guarantee, what they cost on x86 and ARM, and which one each pattern needs.›+ 13 more techniques+ 39 more primitives

Similar systems

Share the most building blocks with this one.

scale-up-fabric 42 %gigawatt-ai-campus 37 %training-fabric 23 %datacenter-facility 21 %multi-datacenter-training 19 %