← Back
Home › Systems › Simulation and scientific computing

flood-forecasting

river flood and hydrology forecasting

GloFAS, Google Flood Hub

SystemTechniques · 9Primitives · 50 flood-forecasting DAG workflow executiondistributed · medium: Run a graph of dependent tasks by releasing each one when its inputs are done, longest chain first, and skip any task whose input hashes are unchanged.›Parallel file system stripingdistributed · medium: Striping files across object storage targets (Lustre, GPFS); collective I/O and metadata serversConservative regriddingnumerics · medium: Remap fields between grids preserving integrals, using overlap areas (first and second order)Ensemble kalman filternumerics · medium: Kalman update with covariance estimated from an ensemble; localization and inflation for huge statesFinite volume methodnumerics · medium: Conservative cell-averaged discretization with numerical fluxes on structured and unstructured meshesChunked array storagestorage · medium: Cut an N-dimensional array into a regular grid of independently compressed blocks, so a slice read touches only the overlapping blocks and writers to different blocks run in parallel.›Durable job queuedistributed · medium: Persisted tasks that workers lease for a visibility timeout, ack on success and retry with backoff; at-least-once execution that survives worker and queue crashes.›Topological sortgraphs · low: Order the vertices of a DAG so every edge points forward; the core of build systems, schedulers and dependency resolution, with cycle detection for free.›Map projectionsgeometry · low: Web Mercator, UTM and equal-area projections, datums and coordinate transformsMap tile pyramidgeometry · low: Z/x/y tile pyramids, quadkeys and overviews for multi-resolution maps and imageryVector tile encodinggeometry · low: Mapbox Vector Tile protobuf: zigzag-delta geometry commands, layers and feature attributesCalibrationmachine-learning · low: Make predicted probabilities match observed frequencies: measure with ECE, fix post hoc with temperature, Platt or isotonic scaling on held-out data.›Quantile regressionmachine-learning · low: Predicting quantiles with the pinball loss for probabilistic forecasts and prediction intervalsRecurrent networksmachine-learning · low: GRU and LSTM cells for streaming sequence modelsCommon alerting protocolnetworking · low: CAP alert message format: areas as polygons/circles, severity, urgency, multilingual info blocksMonte Carlo integrationnumerics · low: Estimate an expectation by averaging random samples; error falls as σ/√N in any dimension, and variance reduction (antithetic, control variates, importance sampling) shrinks σ.›Content-addressed storagestorage · low: Store each blob under the hash of its bytes: immutable, deduplicated by construction, self-verifying on every read, and composable into Merkle trees.›Exponential backoff with jittersystems · low: Retry after delays that grow geometrically and are randomized, so failed clients spread out instead of hammering a recovering service in synchronized waves.›Leasesdistributed · low: A lock that expires on its own: bounded-time ownership that survives crashes and partitions, given bounded clock drift and fencing tokens on writes.›Range partitioningdistributed · low: Cut the sorted key space into contiguous half-open ranges kept in a boundary map; split hot or big ranges, merge small ones, move ranges between nodes.›Rdmanetworking · low: RDMA verbs over InfiniBand/RoCE: queue pairs, memory registration, one-sided opsR-treegeometry · low: A balanced tree of nested bounding boxes that indexes rectangles and points, answering window and nearest-neighbour queries by pruning whole subtrees.›Sparse matrices (CSR)numerics · low: Compressed sparse row stores only nonzeros as row pointers, column indices and values; SpMV streams them once, so it is memory-bandwidth bound.›+ 2 more techniques+ 34 more primitives

Similar systems

Share the most building blocks with this one.

air-quality-model 28 %nowcasting 24 %wildfire-spread 19 %storm-surge 19 %ocean-model 15 %