← Back
Home › Systems › Frontend and UI

ar-runtime

AR headset runtime

Vision Pro, ARKit

SystemTechniques · 8Primitives · 27 ar-runtime Foveated renderinggraphics · medium: Lower shading rate in the periphery via variable rate shading or multi-resolution, driven by eye trackingLate latching reprojectiongraphics · medium: Sample the newest head pose right before scanout and reproject the frame (ATW, ASW, spacewarp)Levenberg marquardtnumerics · medium: Damped Gauss-Newton nonlinear least squares for model calibrationBundle adjustmentrobotics · medium: Jointly refine camera poses and 3D points by sparse nonlinear least squares on reprojection errorCamera calibrationrobotics · medium: Intrinsic/extrinsic calibration (Zhang's method, distortion models, bundle adjustment)Tsdf volume fusionrobotics · medium: Fuse depth frames into truncated signed distance volumes and extract meshesVisual inertial odometryrobotics · medium: Fuse camera features with IMU preintegration in a filter or sliding-window optimizer for 6-DoF poseKalman filterrobotics · low: Optimal recursive estimator for a linear system with Gaussian noise: fuses a motion model with noisy measurements into a state estimate plus its covariance.›Homogeneous transformsgeometry · low: 4x4 affine and projective matrices, homogeneous coordinates, view and projection, normal matricesQuaternion rotationsgeometry · low: Unit quaternions as 3D rotations: 4 floats, cheap composition, no gimbal lock, plus the conversions, interpolation choices and numerical care needed in practice.›Vsync frame pacinggraphics · low: Vsync, swapchain presentation modes, frame pacing, latency and variable refreshLens distortion correctiongraphics · low: Pre-warp rendered frames for headset optics: barrel distortion mesh and chromatic aberrationFeature detection orbrobotics · low: FAST corners with oriented BRIEF descriptors and Hamming matching for tracking and relocalizationReal time linux schedulingembedded · low: PREEMPT_RT, SCHED_FIFO/DEADLINE, CPU isolation and mlockall for low-jitter loopsRansacrobotics · low: Random sample consensus for robust model fitting with outliers (homographies, planes, poses)GPU command buffersgraphics · low: Record GPU work as a flat list of commands on the CPU, submit it in batches, and reuse its memory through a ring of frames in flight guarded by fences.›Image resampling filtersgraphics · low: Resize an image by convolving with a kernel (box, triangle, cubic, Lanczos) whose width scales with the shrink factor, done as two separable 1-D passes with precomputed weights.›Image warping remapgraphics · low: Resampling with lookup maps: resize, lens undistortion, homographies, bilinear/bicubicFinite differencesnumerics · low: Replace derivatives with weighted differences of grid values; simple stencils plus a stability condition (CFL) turn PDEs like heat, advection and waves into array loops.›LU and Cholesky factorizationnumerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).›Sparse matrices (CSR)numerics · low: Compressed sparse row stores only nonzeros as row pointers, column indices and values; SpMV streams them once, so it is memory-bandwidth bound.›Rigid-body transforms (SE(3) poses)robotics · low: Represent, compose, invert, integrate and interpolate 3D poses as unit quaternion plus translation, with exp/log maps for small motions and uncertainty.›Robust statisticsstatistics · low: Location and scale estimates (median, MAD, trimmed mean, Huber M-estimator) that stay correct when a minority of the data is garbage.›+ 1 more techniques+ 11 more primitives

Similar systems

Share the most building blocks with this one.