mhtirogla
Home
Systems
API
Pricing
About
Log in
← Back
Home
›
Systems
›
Simulation and scientific computing
astronomy-pipeline
sky survey alert pipeline
Rubin LSST
System
Techniques · 8
Primitives · 29
astronomy-pipeline
Pub/sub fan-out
distributed · medium: Deliver one published message to every subscriber of a topic by sharing one encoded payload and giving each connection a bounded queue with an explicit slow-consumer policy.
›
Gradient-boosted decision trees (GB…
machine-learning · medium: Add small regression trees one at a time, each fitted to the loss gradient; histogram splits over 256 bins make it the strongest tabular default.
›
Astrometric plate solving
media · medium: Match detected star patterns (quad hashes) to a catalog to solve the image WCS
Image differencing psf matching
media · medium: Subtract a reference image after PSF-matching convolution (Alard-Lupton, ZOGY) to find transients
Source extraction
media · medium: Detect and measure sources in images: background estimation, thresholding, deblending, photometry
Fft convolution
signal-processing · medium: Overlap-add and uniformly partitioned convolution for long FIR filters and reverb
Columnar storage
storage · medium: Store each column contiguously so analytic scans read only the columns they need, compress well and run as tight SIMD loops.
›
Healpix sky pixelization
geometry · low: HEALPix and HTM equal-area hierarchical sphere pixelization for sky indexing and cross-matching
k-d tree
geometry · low: A binary tree that splits k-dimensional points by one coordinate per level, giving fast nearest-neighbour and box queries on static low-dimensional data.
›
Robust statistics
statistics · low: Location and scale estimates (median, MAD, trimmed mean, Huber M-estimator) that stay correct when a minority of the data is garbage.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Image resampling filters
graphics · low: Resize an image by convolving with a kernel (box, triangle, cubic, Lanczos) whose width scales with the shrink factor, done as two separable 1-D passes with precomputed weights.
›
SPSC ring buffer
data-structures · low: Bounded wait-free queue for exactly one producer and one consumer thread, built from an array and two indices with acquire/release ordering.
›
Topic matching trie
data-structures · low: Trie over topic levels with + and # wildcards for subscription matching
Backpressure
systems · low: Bound every queue and make a full queue push back on the producer by blocking, dropping, rejecting or withholding credit.
›
Quantile sketches
data-structures · low: Mergeable quantile sketches: t-digest, DDSketch, HDR histograms
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Regularization
machine-learning · low: Techniques that trade training fit for generalization: weight decay, dropout, early stopping, label smoothing and data mixing, with the exact update rules and defaults.
›
Map projections
geometry · low: Web Mercator, UTM and equal-area projections, datums and coordinate transforms
Ransac
robotics · low: Random sample consensus for robust model fitting with outliers (homographies, planes, poses)
Image warping remap
graphics · low: Resampling with lookup maps: resize, lens undistortion, homographies, bilinear/bicubic
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Convolution image filters
graphics · low: Separable 2D convolution kernels for blur, sharpen (unsharp mask) and edges with border handling
+ 1 more techniques
+ 13 more primitives
System
Techniques · 8
Primitives · 29
astronomy-pipeline
Pub/sub fan-out
distributed · medium: Deliver one published message to every subscriber of a topic by sharing one encoded payload and giving each connection a bounded queue with an explicit slow-consumer policy.
›
Gradient-boosted decision trees (GB…
machine-learning · medium: Add small regression trees one at a time, each fitted to the loss gradient; histogram splits over 256 bins make it the strongest tabular default.
›
Astrometric plate solving
media · medium: Match detected star patterns (quad hashes) to a catalog to solve the image WCS
Image differencing psf matching
media · medium: Subtract a reference image after PSF-matching convolution (Alard-Lupton, ZOGY) to find transients
Source extraction
media · medium: Detect and measure sources in images: background estimation, thresholding, deblending, photometry
Fft convolution
signal-processing · medium: Overlap-add and uniformly partitioned convolution for long FIR filters and reverb
Columnar storage
storage · medium: Store each column contiguously so analytic scans read only the columns they need, compress well and run as tight SIMD loops.
›
Vectorized execution
storage · medium: Query operators pass batches of about 1-2K values per column instead of single tuples, amortizing interpretation overhead and enabling tight SIMD loops.
›
Healpix sky pixelization
geometry · low: HEALPix and HTM equal-area hierarchical sphere pixelization for sky indexing and cross-matching
k-d tree
geometry · low: A binary tree that splits k-dimensional points by one coordinate per level, giving fast nearest-neighbour and box queries on static low-dimensional data.
›
Robust statistics
statistics · low: Location and scale estimates (median, MAD, trimmed mean, Huber M-estimator) that stay correct when a minority of the data is garbage.
›
Object storage
cloud · low: S3-compatible blob storage at cents per GB-month with 11 nines durability; pick by egress price, request price and latency, not by storage price alone.
›
Image resampling filters
graphics · low: Resize an image by convolving with a kernel (box, triangle, cubic, Lanczos) whose width scales with the shrink factor, done as two separable 1-D passes with precomputed weights.
›
SPSC ring buffer
data-structures · low: Bounded wait-free queue for exactly one producer and one consumer thread, built from an array and two indices with acquire/release ordering.
›
Topic matching trie
data-structures · low: Trie over topic levels with + and # wildcards for subscription matching
Backpressure
systems · low: Bound every queue and make a full queue push back on the producer by blocking, dropping, rejecting or withholding credit.
›
Quantile sketches
data-structures · low: Mergeable quantile sketches: t-digest, DDSketch, HDR histograms
Loss functions
machine-learning · low: The scalar a model minimizes: cross-entropy for classes and tokens, MSE or Huber for values, each with a stable form and simple gradient.
›
Regularization
machine-learning · low: Techniques that trade training fit for generalization: weight decay, dropout, early stopping, label smoothing and data mixing, with the exact update rules and defaults.
›
Map projections
geometry · low: Web Mercator, UTM and equal-area projections, datums and coordinate transforms
Ransac
robotics · low: Random sample consensus for robust model fitting with outliers (homographies, planes, poses)
Image warping remap
graphics · low: Resampling with lookup maps: resize, lens undistortion, homographies, bilinear/bicubic
LU and Cholesky factorization
numerics · low: Factor A = PLU (any square matrix, partial pivoting) or A = LLᵀ (SPD) once in O(n³), then solve each right-hand side in O(n²).
›
Convolution image filters
graphics · low: Separable 2D convolution kernels for blur, sharpen (unsharp mask) and edges with border handling
Union-find
graphs · low: Disjoint-set forest with union by size and path compression; merges sets and answers same-set queries in effectively constant time.
›
Spline interpolation
numerics · low: Linear, bilinear/trilinear and cubic spline interpolation on grids and tables
Peak detection
signal-processing · low: Robust peak and event detection in noisy signals with adaptive thresholds and refractory periods
SIMD programming
hardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.
›
Fast Fourier transform
numerics · low: Computes the DFT in O(n log n) by recursively splitting into even and odd halves; the basis of fast convolution and spectral methods.
›
Bit packing and PFOR
compression · low: Store blocks of integers with exactly b bits each and patch rare outliers separately (PFOR); the fastest integer codec family, billions of values per second.
›
Delta encoding
compression · low: Store differences between consecutive values instead of the values, turning sorted IDs, timestamps and counters into small numbers that pack into few bits.
›
Dictionary encoding
compression · low: Map values to small integer codes via a dictionary for compact, comparable columns
Zone maps
storage · low: Per-block min/max (and sparse primary) indexes to skip data during scans
Varint (LEB128)
compression · low: Variable-length integers in 7-bit groups with a continuation bit; small numbers take 1 byte, the wire format of Protobuf, WebAssembly and DWARF.
›
False sharing
concurrency · low: Independent variables written by different cores on one cache line force the line to bounce between cores; pad or regroup hot data to fix it.
›
Memory ordering
concurrency · low: What Relaxed, Acquire, Release and SeqCst atomics actually guarantee, what they cost on x86 and ARM, and which one each pattern needs.
›
CPU caches and the memory hierarchy
hardware-os · low: Sizes, line widths and latencies of L1/L2/L3/DRAM on modern x86 and Apple M-series, and the data-layout rules that follow from them.
›
Copy for AI