← Back
Home › Systems › Observability and operations

continuous-profiling

continuous profiling

Parca, Pyroscope

SystemTechniques · 7Primitives · 20 continuous-profiling Stack unwindingcompilers · medium: Frame-pointer, DWARF CFI and compact unwinding of call stacksSymbolicationcompilers · medium: Map addresses to functions and lines via DWARF, PDB and source mapsSampling profiling pmuhardware-os · medium: Timer/PMU-driven stack sampling with low overheadColumnar storagestorage · medium: Store each column contiguously so analytic scans read only the columns they need, compress well and run as tight SIMD loops.›Telemetry samplingsystems · medium: Head, tail and dynamic sampling of traces, events and logs to a budgetDebug info (DWARF)compilers · medium: DWARF stores a tree of typed entries (DIEs), a line-number bytecode, unwind tables and location expressions that map machine state back to source; pick it for ELF/Mach-O debug builds.›Vectorized executionstorage · medium: Query operators pass batches of about 1-2K values per column instead of single tuples, amortizing interpretation overhead and enabling tight SIMD loops.›Hardware performance countershardware-os · low: PMU counters via perf_event: instructions, cycles, cache missesDelta encodingcompression · low: Store differences between consecutive values instead of the values, turning sorted IDs, timestamps and counters into small numbers that pack into few bits.›Zstandard (zstd)compression · low: LZ77 plus Huffman and FSE entropy coding, with levels from LZ4-like speed to xz-like ratio, all decoding at about 1-2 GB/s.›String interningdata-structures · low: Store each distinct string once and refer to it by idEbpfhardware-os · low: Verified in-kernel programs on hooks (kprobes, XDP, tc) with mapsFlame graphshardware-os · low: Aggregate sampled stacks into a prefix tree and render flame or icicle graphsTime partitioningstorage · low: Cut a table into fixed-interval time blocks so queries read only overlapping blocks and retention drops a whole block as one metadata change.›Calling conventions abicompilers · low: System V, Windows x64 and AAPCS64 ABIs: argument registers, stack frames, struct passing, FFIExecutable formats elf pecompilers · low: ELF, PE and Mach-O: sections, segments, symbol tables, relocations, dynamic loadingSource mapscompilers · low: Source map v3: VLQ-encoded mappings from generated to original positionsInterval treedata-structures · low: Index of ranges that answers "which intervals overlap this point or range?" in O(log n + k); static sorted-array form is the compact, cache-friendly choice.›Interrupt handlinghardware-os · low: How a device signal reaches a CPU handler through an interrupt controller and vector table, and how top/bottom halves, polling switch-over and coalescing keep load bounded.›Bit packing and PFORcompression · low: Store blocks of integers with exactly b bits each and patch rare outliers separately (PFOR); the fastest integer codec family, billions of values per second.›Dictionary encodingcompression · low: Map values to small integer codes via a dictionary for compact, comparable columnsZone mapsstorage · low: Per-block min/max (and sparse primary) indexes to skip data during scansHash based samplingdata-structures · low: Deterministic sampling by hashing an entity id so all its events are kept or dropped together+ 4 more primitives

Similar systems

Share the most building blocks with this one.