← Back
Home › Systems › Numerical and mathematical computing

probabilistic-programming

Bayesian inference engine

Stan, PyMC

SystemTechniques · 4Primitives · 13 probabilistic-programmi… Hamiltonian monte carlo nutsnumerics · medium: HMC with leapfrog integration and the No-U-Turn sampler; step size and mass-matrix adaptationMarkov chain monte carlonumerics · medium: Metropolis-Hastings and Gibbs sampling: proposals, burn-in, thinning, convergenceSequential monte carlonumerics · medium: Particle filters and SMC samplers with resampling for sequential Bayesian inferenceVariational inferencenumerics · medium: ADVI and mean-field VI: optimize the ELBO with reparameterization gradientsReverse-mode automatic differentiat…ml-training · low: Record the forward computation on a tape, then sweep it backwards once to get the gradient of a scalar loss for all inputs.›Constrained parameter transformsnumerics · low: Map constrained parameters to unconstrained space (log, logit, simplex, Cholesky) with Jacobian correctionsMcmc convergence diagnosticsnumerics · low: R-hat, effective sample size and divergence checks for MCMC runsPRNGs (PCG, xoshiro)numerics · low: Small, fast, statistically strong non-cryptographic generators: PCG32 and xoshiro256**, with seeding, parallel streams and unbiased ranges.›Floating-point errornumerics · low: How IEEE 754 rounding error accumulates and how to contain it: compensated (Kahan/Neumaier) and pairwise summation, error-free transformations with FMA, ULP comparisons.›L-BFGSnumerics · low: Quasi-Newton minimizer that approximates the inverse Hessian from the last m gradient differences via the two-loop recursion; O(m·n) memory and work per step.›LU and Cholesky factorizationnumerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).›Verlet / leapfrognumerics · low: Half-kick, drift, half-kick: a second-order, time-reversible, symplectic integrator with one force evaluation per step whose energy error stays bounded for millions of steps.›Alias method samplingnumerics · low: Vose alias tables: O(1) sampling from a discrete distribution after O(n) setupImportance samplingnumerics · low: Sample from a proposal distribution and reweight to cut Monte Carlo varianceAdamWml-training · low: Adam's per-parameter adaptive steps with weight decay applied directly to the weights; the default optimizer for transformers and LLM pretraining.›Monte Carlo integrationnumerics · low: Estimate an expectation by averaging random samples; error falls as σ/√N in any dimension, and variance reduction (antithetic, control variates, importance sampling) shrinks σ.›Rejection samplingml-training · low: Best-of-n and filtered sampling: generate many candidates, keep those passing a verifier or reward threshold

Similar systems

Share the most building blocks with this one.

nonlinear-optimizer 22 %derivatives-pricing 19 %uncertainty-quantification 18 %nonlinear-solver 17 %statistics-environment 17 %