← Back
Home › Systems › AI and ML infrastructure

gpu-job-scheduler

GPU cluster job scheduler

Kueue, Slurm

SystemTechniques · 9Primitives · 38 gpu-job-scheduler Backfill schedulingdistributed · medium: Run small jobs in gaps ahead of reserved large jobs without delaying themGang schedulingdistributed · medium: All-or-nothing co-scheduling of a job's tasksRaftdistributed · medium: Leader-based consensus on a replicated log; one leader per term, majority commit; the default for a strongly consistent replicated state machine.›Distributed checkpointingml-training · medium: Sharded, asynchronous checkpoint save/load of model and optimizer state, with resharding to new world sizesGpu health checksml-training · medium: Detecting bad GPUs/links before and during jobs: ECC/XID errors, NCCL tests, throughput probes, drainingFair share schedulingsystems · medium: Dominant resource fairness and hierarchical fair-share queuesTopology aware placementsystems · medium: Place work by NUMA, rack, zone and GPU interconnect topologyLeasesdistributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.›d-ary heapdata-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.›Bin packingoptimization · low: Pack items of given sizes into the fewest fixed-capacity bins; sort descending, then first-fit, gives at most 11/9 of optimal and runs in O(n log n).›Containers and Kubernetescloud · low: Containers package a service with its dependencies; Kubernetes schedules them across machines. Use the smallest orchestrator that does the job.›Phi accrual failure detectordistributed · low: Instead of a fixed heartbeat timeout, output a continuous suspicion level φ from the observed heartbeat interval distribution; adapts to each link's jitter.›Cgroups and namespaceshardware-os · low: Linux namespaces limit what a process can see; cgroup v2 limits and accounts what it may use. Containers, sandboxes and job schedulers are built from the two.›Interval treedata-structures · low: Index of ranges that answers "which intervals overlap this point or range?" in O(log n + k); static sorted-array form is the compact, cache-friendly choice.›Distributed barrierdistributed · low: Rendezvous/barrier for a dynamic worker group: join, agree on membership and rank assignment, restart on changeWrite-ahead logstorage · low: Append each change to a checksummed sequential log and fsync it before touching data pages; the basis of crash durability in nearly every database.›Exponential backoff with jittersystems · low: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.›Object storagecloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.›Atomic file replacementhardware-os · low: Write temp file, fsync, rename over target, fsync directory: crash-safe whole-file updatesCollective communicationml-training · low: The family of group exchanges (broadcast, reduce, all-gather, reduce-scatter, all-to-all) and the latency-plus-bytes cost model that picks ring, tree or direct algorithms by message size and topology.›Tensor layout transformsml-training · low: Choosing and converting tensor layouts (NCHW/NHWC, blocked, tiled) for kernel efficiencyZeRO and FSDP shardingml-training · low: Data parallelism that shards optimizer state, gradients and weights across ranks and gathers weights just in time; per-GPU model memory drops from 16Ψ to 16Ψ/N.›Edac memory scrubbingembedded · low: ECC/EDAC memory with periodic scrubbing to correct radiation-induced bit flips+ 2 more techniques+ 22 more primitives

Similar systems

Share the most building blocks with this one.

cluster-scheduler 59 %elastic-training 25 %gpu-cloud 23 %

In-house systems

Named systems of this kind at large companies, with their paper or write-up.

SingularityMicrosoft