Debug info (DWARF)

Deep dive

How it works

Parts:

• .debug_info + .debug_abbrev — tree of DIEs per compilation unit; abbreviation tables give each DIE its tag and attribute/form list.
• .debug_line — line-number program per unit: header (file tables, line_base, line_range, opcode_base) plus bytecode that builds the address-to-line table.
• .debug_frame / .eh_frame — call-frame information: CIEs (shared defaults) and FDEs (one per function range) holding DW_CFA_* ops that recover the caller's registers.
• Location expressions and lists — DW_OP_* stack programs; .debug_loc (v4) or .debug_loclists (v5) pick one per address range.
• String and range tables — .debug_str, .debug_line_str, .debug_ranges or .debug_rnglists, .debug_aranges, .debug_addr, .debug_str_offsets.

1. Read a unit header. Unit length, version (5 current), unit type (DW_UT_compile, DW_UT_type, DW_UT_skeleton, DW_UT_split_compile, ...), address size, offset into .debug_abbrev.
2. Decode a DIE. Read a ULEB abbreviation code; 0 ends a sibling list. The abbreviation gives the tag (DW_TAG_subprogram, DW_TAG_variable, DW_TAG_inlined_subroutine), whether children follow, and (attribute, form) pairs. The form fixes the encoding of each value (DW_FORM_data4, DW_FORM_strp, DW_FORM_ref4, DW_FORM_implicit_const keeps a constant in the abbreviation itself).
3. Find the function for a pc. A subprogram has DW_AT_low_pc + DW_AT_high_pc, or DW_AT_ranges for split hot/cold code. .debug_aranges indexes this so a tool skips most units.
4. Find the line. Run the unit's line program (see Code) to rows (address, file, line, column, is_stmt, end_sequence); the answer for a pc is the last row at or below it inside its sequence. is_stmt marks good breakpoint places.
5. Expand inlining. Inside the subprogram, nested DW_TAG_inlined_subroutine DIEs carry DW_AT_abstract_origin (the real function), DW_AT_call_file/call_line, and their own pc ranges. Walking the nesting for one pc yields the virtual frames, innermost first. See function inlining.
6. Locate a variable. DW_AT_location is either one expression or a location list: entries DW_LLE_offset_pair (range relative to the unit base) + expression. Typical expressions: DW_OP_fbreg -20 (offset from DW_AT_frame_base), DW_OP_reg3 (in a register), DW_OP_piece (split across places), DW_OP_entry_value (the register's value at function entry). The type is DW_AT_type, a reference to a base, pointer, struct or typedef DIE.
7. Unwind a frame. Find the FDE whose range holds the pc; run its DW_CFA_* program (advance_loc, def_cfa, def_cfa_offset, offset) up to the pc to get a row: how to compute the CFA, and where each callee-saved register and the return address were saved. Caller pc = saved return address; repeat. See stack unwinding and symbolication.

.debug_info  CU ─ subprogram main ─ variable x (loclist) ─ inlined_subroutine f (range, call_line)
     │ abbrev code ─► .debug_abbrev (tag, children?, attr/form pairs)
     │ DW_AT_stmt_list ─► .debug_line (header + opcode program ─► row table)
.eh_frame    CIE ◄── FDE (pc range, CFA ops) ─► unwind row at pc

DWARF 5 changes: forms strx*/addrx* index .debug_str_offsets and .debug_addr (bases DW_AT_str_offsets_base, DW_AT_addr_base), so strings and addresses become small indices; .debug_rnglists/.debug_loclists replace .debug_ranges/.debug_loc (DW_RLE_*, DW_LLE_*); the line header uses entry formats (DW_LNCT_path, ...) with paths in .debug_line_str; .debug_names adds a lookup index.

Split DWARF moves the bulk into per-object .dwo files and leaves a small skeleton unit in the object (DW_UT_skeleton, DW_AT_dwo_name, DW_AT_dwo_id); the linker never reads the bulk. A .dwp packs many .dwo files and indexes them with .debug_cu_index/.debug_tu_index.

Code

LEB128 and the line-number state machine, with line_base = -5, line_range = 14, opcode_base = 13 (the usual values), run on a hand-encoded program.

fn uleb(b: &[u8], i: &mut usize) -> u64 {
    let (mut v, mut sh) = (0u64, 0);
    loop {
        let x = b[*i]; *i += 1;
        if sh < 64 { v |= u64::from(x & 0x7f) << sh; }
        sh += 7;
        if x & 0x80 == 0 { return v; }
    }
}
fn sleb(b: &[u8], i: &mut usize) -> i64 {
    let (mut v, mut sh) = (0i64, 0);
    loop {
        let x = b[*i]; *i += 1;
        if sh < 64 { v |= i64::from(x & 0x7f) << sh; }
        sh += 7;
        if x & 0x80 == 0 {
            if sh < 64 && x & 0x40 != 0 { v |= -1i64 << sh; } // sign-extend
            return v;
        }
    }
}

struct Hdr { min_inst: u64, line_base: i64, line_range: u64, opcode_base: u8, std_lens: Vec<u8> }
#[derive(Clone, Copy)]
struct Row { addr: u64, file: u64, line: i64, col: u64, is_stmt: bool, end_seq: bool }
const START: Row = Row { addr: 0, file: 1, line: 1, col: 0, is_stmt: true, end_seq: false };

fn run(h: &Hdr, p: &[u8]) -> Vec<Row> {
    let (mut rows, mut r, mut i) = (vec![], START, 0);
    while i < p.len() {
        let op = p[i]; i += 1;
        if op >= h.opcode_base { // special: advance address and line, emit a row
            let adj = u64::from(op - h.opcode_base);
            r.addr += (adj / h.line_range) * h.min_inst;
            r.line += h.line_base + (adj % h.line_range) as i64;
            rows.push(r);
        } else if op == 0 { // extended: length, sub-opcode, operands
            let end = uleb(p, &mut i) as usize + i;
            match p[i] {
                1 => { r.end_seq = true; rows.push(r); r = START; }          // end_sequence
                2 => r.addr = u64::from_le_bytes(p[i + 1..i + 9].try_into().unwrap()), // set_address
                _ => {} // define_file, set_discriminator, vendor: skip by length
            }
            i = end;
        } else {
            match op {
                1 => rows.push(r),                                   // copy
                2 => r.addr += uleb(p, &mut i) * h.min_inst,         // advance_pc
                3 => r.line += sleb(p, &mut i),                      // advance_line
                4 => r.file = uleb(p, &mut i),                       // set_file
                5 => r.col = uleb(p, &mut i),                        // set_column
                6 => r.is_stmt = !r.is_stmt,                         // negate_stmt
                8 => r.addr += ((255 - h.opcode_base) as u64 / h.line_range) * h.min_inst, // const_add_pc
                9 => { r.addr += u16::from_le_bytes([p[i], p[i + 1]]) as u64; i += 2; } // fixed_advance_pc: u16, not ULEB
                _ => for _ in 0..h.std_lens[op as usize - 1] { uleb(p, &mut i); } // 7, 10-12, unknown
            }
        }
    }
    rows
}

/// pc -> (file, line): last row with addr <= pc, unless it ends a sequence.
fn lookup(rows: &[Row], pc: u64) -> Option<(u64, i64)> {
    let r = rows.get(rows.partition_point(|r| r.addr <= pc).checked_sub(1)?)?;
    if r.end_seq { None } else { Some((r.file, r.line)) }
}

fn main() {
    let h = Hdr { min_inst: 1, line_base: -5, line_range: 14, opcode_base: 13,
                  std_lens: vec![0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1] };
    let mut p = vec![0, 9, 2];                    // DW_LNE_set_address 0x1000
    p.extend(0x1000u64.to_le_bytes());
    p.extend([5, 3, 3, 9, 1]);                    // column 3; line += 9; copy -> 0x1000:10
    p.push(13 + (4 + 5) + 14 * 4);                // special: addr +4, line +4 -> 0x1004:14
    p.extend([3, 0x7b, 2, 0x90, 0x01, 1]);        // line -= 5; addr += 144; copy -> 0x1094:9
    p.extend([2, 6, 0, 1, 1]);                    // addr += 6; end_sequence at 0x109a
    let rows = run(&h, &p);
    assert_eq!(rows.len(), 4);
    assert_eq!((rows[1].addr, rows[1].line), (0x1004, 14));
    assert_eq!((rows[2].addr, rows[2].line), (0x1094, 9));
    assert_eq!(lookup(&rows, 0x1002), Some((1, 10)));
    assert_eq!(lookup(&rows, 0x1050), Some((1, 14)));
    assert_eq!(lookup(&rows, 0x109a), None);
    assert_eq!(uleb(&[0xe5, 0x8e, 0x26], &mut 0), 624485);
    assert_eq!(sleb(&[0x80, 0x7f], &mut 0), -128);
}

Complexity and constants

Decoding is linear in bytes, with a data-dependent branch per LEB byte. A pc lookup is a binary search over the rows after decoding that unit's program once. Cost drivers: bytes read from disk (cold page cache), strings (.debug_str is often the largest section) and link-time relocations. Make it cheap by decoding lazily per unit and caching the row table.

Efficiency

• Cost: measured here (Rust stable, macOS arm64, debug build of the code above; dsymutil output): executable 557 KB, DWARF in the dSYM 2.27 MB of sections, of which .debug_str 1.04 MB, .debug_info 0.46 MB, .debug_ranges 0.24 MB, .debug_line 0.17 MB. Gzip level 6 on the whole 2.42 MB dSYM file gave 0.60 MB. Debug info is several times the code because it carries std's types. Reading is bound by I/O and lookups, not arithmetic.
• Levers:
  • Compressed sections — roughly 4x on the file above (self-measured with gzip, not the exact ELF codec) — SHF_COMPRESSED with ELFCOMPRESS_ZLIB or ELFCOMPRESS_ZSTD — tools must decompress a whole section before use.
  • Split DWARF — linker reads a skeleton, not the bulk (GCC DebugFission wiki, qualitative) — large C++ links where debug info dominates link time.
  • Lazy parsing — read .debug_aranges/.debug_names first and decode only the unit holding the pc (the design of the gimli crate) — symbolizers touching few addresses.
  • DWARF 5 string and address tables — fewer relocations and repeated strings — many objects, many repeated names.
  • Line tables only (-g1) — drops variables and types, keeps pc-to-line — release builds that need backtraces.
• Naive → standard → state of the art: parse every DIE into a tree up front · memory several times the section size. Index by .debug_aranges, parse one unit on demand · bytes touched scale with units hit. Split DWARF, compressed sections and lazy per-unit parsing · small link input, work only for addresses asked.

Implementation notes

• Initial length 0xffffffff means 64-bit DWARF (8-byte offsets); check the unit's address size too.
• Apply relocations before reading a .o or .dwo; linked executables are final.
• Cache per unit: abbreviation table (keyed by offset) and sorted line rows.
• Debug sections are not SHF_ALLOC; separate them with objcopy --only-keep-debug plus a build-id or .gnu_debuglink.
• Keep .eh_frame and .eh_frame_hdr (binary-search table) in release builds if anything must unwind.

Pitfalls

• Optimized code: a variable may have no location at some pcs, or only via DW_OP_entry_value; show "optimized out", do not guess.
• Several rows can share one address; pick by your tool's rule.
• DW_AT_high_pc is an offset when its form is a constant, an address when it is an address form.
• v4 .debug_ranges/.debug_loc and v5 *lists differ in encoding and base-address rules.
• max_ops_per_inst > 1 (VLIW) changes the advance formulas.

Variants and alternatives

In production

• Native tooling: GDB, LLDB, profilers such as perf, addr2line, sanitizer and Valgrind symbolization, Rust backtraces (via gimli), crash-reporter symbol files.
• Browsers: Chrome DevTools debugs C/C++ compiled to WebAssembly from DWARF (blog).
• JITs and runtimes: GDB's JIT interface lets a JIT register in-memory objects with debug info (manual).
• Operating systems and eBPF: the Linux kernel's BTF is produced from DWARF by pahole, a dwarf2btf converter (docs).

Implementations

Sources

• DWARF Debugging Information Format, Version 5, 2017. standard
• gimli source (src/constants.rs names checked against this page). code
• GCC wiki, DebugFission. page
• Linux Standard Base, Exception Frames (.eh_frame). page
• Linux kernel docs, BPF Type Format (BTF). page
• Go standard library, debug/gosym. page