← Back
Home › Systems › Numerical and mathematical computing

surrogate-modeling

neural surrogates and PINNs for simulation

NVIDIA PhysicsNeMo

SystemTechniques · 5Primitives · 42 surrogate-modeling Active learningmachine-learning · medium: Choosing which unlabeled examples to label next: uncertainty, margin, diversity and committee samplingData loading pipelinesmachine-learning · medium: Turn files on disk into shuffled, sharded, resumable GPU batches fast enough that the accelerator never waits.›Graph neural networksmachine-learning · medium: Message-passing neural networks over graphs (GCN, GAT, equivariant) for molecules and relationsExperiment trackingml-training · medium: Logging params, metrics, artifacts and code versions per run with an append-only metric store and comparisonChunked array storagestorage · medium: Cut an N-dimensional array into a regular grid of independently compressed blocks, so a slice read touches only the overlapping blocks and writers to different blocks run in parallel.›Normalization layersmachine-learning · low: Standardize activations per batch, token or group with a learned scale so deep nets train fast and stably; RMSNorm in LLMs, BatchNorm in convnets.›Gaussian processesmachine-learning · low: GP regression with kernels: posterior mean/variance via Cholesky, hyperparameters by marginal likelihoodLoss functionsmachine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.›AdamWml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.›Learning-rate warmup and cosine dec…ml-training · low: Ramp the learning rate up linearly for the first ~1% of steps, then decay it along a half cosine to ~10% of peak.›Mixed-precision trainingml-training · low: Run matmuls in bf16/fp16/fp8 on tensor cores while keeping master weights, optimizer state and reductions in fp32; about 2x speed, half the activation memory.›Reverse-mode automatic differentiat…ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.›Ring all-reduceml-training · low: Sum a buffer across N workers via reduce-scatter then all-gather around a ring; each link carries 2(N−1)/N of the buffer, which is bandwidth-optimal.›Transformer blockml-training · low: The decoder-only pre-norm block (attention plus MLP on a residual stream) behind every modern LLM, with exact shapes, parameter, FLOP and KV-cache counts.›Fast Fourier transformnumerics · low: Computes the DFT in O(n log n) by recursively splitting into even and odd halves; the basis of fast convolution and spectral methods.›Finite differencesnumerics · low: Replace derivatives with weighted differences of grid values; simple stencils plus a stability condition (CFL) turn PDEs like heat, advection and waves into array loops.›L-BFGSnumerics · low: Quasi-Newton minimizer that approximates the inverse Hessian from the last m gradient differences via the two-loop recursion; O(m·n) memory and work per step.›Low discrepancy sequencesnumerics · low: Sobol, Halton and blue-noise sequences with scrambling for quasi-Monte CarloUnstructured mesh representationnumerics · low: Cell-face-node connectivity in CSR form for FEM and FVM meshes; partition-friendly orderingCalibrationmachine-learning · low: Make predicted probabilities match observed frequencies: measure with ECE, fix post hoc with temperature, Platt or isotonic scaling on held-out data.›Embeddingsmachine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.›+ 26 more primitives

Similar systems

Share the most building blocks with this one.

ml-weather-model 30 %interpretability-platform 25 %llm-pretraining 21 %