← Back
Home › Systems › Finance and trading

market-data

market data feed handler

ITCH, OPRA

SystemTechniques · 3Primitives · 29 market-data Clock synchronizationdistributed · medium: Estimate a clock's offset from four timestamps, filter out queueing noise, and steer the clock smoothly; NTP gets about a millisecond, PTP with hardware timestamps under a microsecond.›Columnar storagestorage · medium: Store each column contiguously so analytic scans read only the columns they need, compress well and run as tight SIMD loops.›Vectorized executionstorage · medium: Query operators pass batches of about 1-2K values per column instead of single tuples, amortizing interpretation overhead and enabling tight SIMD loops.›Delta encodingcompression · low: Store differences between consecutive values instead of the values, turning sorted IDs, timestamps and counters into small numbers that pack into few bits.›Memory orderingconcurrency · low: What Relaxed, Acquire, Release and SeqCst atomics actually guarantee, what they cost on x86 and ARM, and which one each pattern needs.›Seqlockconcurrency · low: Readers copy shared data optimistically and retry if a sequence counter changed; zero reader writes, so reads scale perfectly with a single fast writer.›SPSC ring bufferdata-structures · low: Bounded wait-free queue for exactly one producer and one consumer thread, built from an array and two indices with acquire/release ordering.›Limit order bookfinance · low: Per-symbol store of resting orders as FIFO queues per price level; array-indexed ticks plus intrusive lists give O(1) add, cancel and best-price lookup.›Low-latency hot pathfinance · low: Rules for microsecond trading paths: one pinned busy-polling thread per stage, no allocation, syscalls, locks or page faults after startup, kernel-bypass I/O, and measured tails.›Buffer pools for zero-allocation I/Ohardware-os · low: Preallocate a fixed set of aligned I/O buffers and recycle them through a free list, so the hot path never mallocs, faults or unmaps. The bounded count doubles as back pressure.›SIMD programminghardware-os · low: Process 4-16 lanes per instruction with AVX2, AVX-512 or NEON via Rust std::arch, with runtime dispatch and a portable fallback.›IP multicastnetworking · low: Send each datagram once to a class D group address; the network copies it to every joined receiver, and sequence numbers let receivers find loss.›Kernel-bypass networkingnetworking · low: User space (DPDK, AF_XDP) or an in-driver eBPF hook (XDP) handles packets straight from NIC rings, skipping sockets, for 10-40 Mpps per core.›Packet capture pcapnetworking · low: Lossless packet capture with hardware timestamps, pcap/pcapng format, replaySequenced multicast feednetworking · low: Exchange order-by-order data as fixed binary messages in sequenced UDP multicast packetsZero-copy serializationnetworking · low: Wire formats laid out like in-memory structs (FlatBuffers, Cap'n Proto, SBE, rkyv) so readers access fields in place with no decode pass or allocation.›PID controllerrobotics · low: Drive a measured value to a setpoint with three terms on the error: proportional, integral, derivative. The default feedback loop for motors, heaters, drones and gimbals.›Phase-locked loop (digital)signal-processing · low: Feedback loop that locks a local oscillator to the phase and frequency of an input tone; the core of carrier recovery, clock recovery and GNSS tracking.›Robust statisticsstatistics · low: Location and scale estimates (median, MAD, trimmed mean, Huber M-estimator) that stay correct when a minority of the data is garbage.›+ 13 more primitives

Similar systems

Share the most building blocks with this one.

hft 40 %exchange 32 %smart-order-router 22 %physics-daq 20 %