Repository navigation
fix(dwarf): derive DWARF from where each instruction was emitted - #340
Conversation
There was a problem hiding this comment.
This PR changes how walrus moves DWARF onto the emitted code. It used to map addresses through instruction_map, which keeps only one output place per input location. Now every line row, range, location list entry and code address is built from where each input instruction was actually emitted (CodeMap over function_instructions). It also fixes DW_OP_addr, which was being remapped as a code address, drops the dangling macro attributes, and gives delegate a location. I take it to be the last step of the DWARF series (#338, #339), aimed at keeping debug info correct after optimizing passes such as wasm-shrink's. The design is sound and well tested; the main concerns are an unannounced semver break on CodeTransform and exprloc locals that are only remapped for their own function once code is copied into another one. A smaller limitation, not a blocker: when a pass copies code from one compile unit into a function owned by another, the copied instructions get no line rows. The owning unit's sequences don't cover the source addresses, so the sequence just ends there, and no other unit claims the function (convert_line_program, the owns check). It's worth noting in the description or fixing later.
Not merge-ready:
CodeTransformgains a private field, which breaks building it with a struct expression outside walrus, and the CHANGELOG doesn't say so.- An
exprloclocal or frame base is remapped only for its original function, so in code a pass copied into another function it names the wrong local; no test covers this.
cfb789c to
7ad25f2
Compare
c34f4ed to
d9c692d
Compare
7ad25f2 to
252b570
Compare
d9c692d to
1a49041
Compare
Emitting a function now records its instructions in order with their locations, and DWARF is converted from that: line rows wherever the input row covering an emitted instruction changes, range and location list entries as runs of emitted code, and single addresses at the first instruction still emitted at or after them. Code that passes insert, copy or remove is handled, a low_pc/high_pc pair whose code is now in several places becomes DW_AT_ranges, and a local a pass replaced is found through the instructions that used it. Also: - DW_OP_addr is a linear memory address in Wasm and is kept; it was treated as a code address, which replaced static variables' locations. - `delegate` records its location as the end of the try body. - Line sequences end where the input's coverage ends, and rows from the function before don't describe the next one's start. - DW_AT_macros/DW_AT_macro_info/DW_AT_GNU_macros are removed, since the sections they point into aren't emitted. dwarf_wasm_locations put low_pc at the body's size byte; LLVM's low_pc is the body start.
An `exprloc` was remapped only for its own function's locals, so in code a pass copied into another function it named an unrelated local there. Each entry now carries the emitted code of its scope; where a local's index differs between the functions that code is in, the location becomes a location list with the right index in each, for attributes that can be one. Line rows for code copied from another compile unit were lost: only the unit that owned an output function described it, and its sequences don't cover the copy. Every unit whose sequences cover code emitted in a function now describes that code. Keep `CodeTransform` as it was, so it can still be built outside walrus, by moving the emitted instruction lists to `EmitContext`.
252b570 to
96913bb
Compare
An entry's local indices came from the function its `DW_AT_low_pc` is in, or else its parent's. An entry that gives its code as `DW_AT_ranges` instead, as subprograms and lexical blocks can, got its parent's function (often none, as a unit's `DW_AT_low_pc` of 0 is a tombstone), and its parent's code as its scope. Its locals were then left unchanged, naming locals walrus removed or renumbered, or remapped through another function and dropped. The fuzzer found this on 16 of the 127 `round_trip` and `valid` modules. An entry's scope is now its own code, from `DW_AT_ranges` or low/high pc, split into pieces by the input function each came from, as local indices are that function's. An expression whose pieces are now in functions that number its locals differently becomes a location list with the right local in each, as #340 already did for code a pass copied. A location list entry is remapped per piece, rather than with the function its start is in. A `DW_LLE_default_location` can't be split, so it's dropped where its locals differ. Expressions that name no locals, such as a static variable's `DW_OP_addr` in a unit covering every function, are remapped once. Also, DWARF that can't be read is dropped with a warning instead of making `emit_wasm` panic, including two line program inputs gimli panics on instead of returning an error: a DWARF 4 `DW_LNE_define_file` with an empty name, and a `DW_LNS_advance_line` of `i64::MIN`. They're rejected before gimli sees them.
Add `walrus_tests_utils::dwarf`, which compares what a debugger learns at each instruction of the input and of walrus's output: the source line, the scopes covering it and where each variable is. It pairs instructions by walking both modules' function bodies side by side, so it doesn't rely on walrus's own record of where it put each instruction. `synthesize` gives a module DWARF chosen by an `arbitrary` input, varying forms (DWARF 4 and 5, low/high pc, ranges, location lists, exprlocs) and ranges. It runs in: - `dwarf_synthetic`: every `round_trip` and `valid` module, with default DWARF and 32 fixed seeds, with and without gc; - `dwarf_fixtures`: rustc and clang output at several optimization levels and DWARF versions, rebuilt by `dwarf-fixtures/build.sh`, and through `llvm-dwarfdump --verify` when it's on `$PATH`; - the `dwarf` fuzz target, over wasm-smith modules with synthesized DWARF; - the `dwarf-raw` fuzz target, which mutates the fixtures and checks that walrus doesn't panic, emits readable DWARF with code ranges inside functions, and emits the same bytes twice. `dwarf_mapping` has regression tests for what the fuzzers found on main that #340 already fixes: ranges and line rows starting in removed code or the byte before an instruction, `DW_OP_addr`, low/high pc spanning functions, rows at a function's end and VLIW line tables. Some `dwarf_synthetic` seeds fail until the next commit, which fixes the locals of entries that give their code as `DW_AT_ranges`.
An entry's local indices came from the function its `DW_AT_low_pc` is in, or else its parent's. An entry that gives its code as `DW_AT_ranges` instead, as subprograms and lexical blocks can, got its parent's function (often none, as a unit's `DW_AT_low_pc` of 0 is a tombstone), and its parent's code as its scope. Its locals were then left unchanged, naming locals walrus removed or renumbered, or remapped through another function and dropped. The fuzzer found this on 16 of the 127 `round_trip` and `valid` modules. An entry's scope is now its own code, from `DW_AT_ranges` or low/high pc, split into pieces by the input function each came from, as local indices are that function's. An expression whose pieces are now in functions that number its locals differently becomes a location list with the right local in each, as #340 already did for code a pass copied. A location list entry is remapped per piece, rather than with the function its start is in. A `DW_LLE_default_location` can't be split, so it's dropped where its locals differ. Expressions that name no locals, such as a static variable's `DW_OP_addr` in a unit covering every function, are remapped once. Also, DWARF that can't be read is dropped with a warning instead of making `emit_wasm` panic, including two line program inputs gimli panics on instead of returning an error: a DWARF 4 `DW_LNE_define_file` with an empty name, and a `DW_LNS_advance_line` of `i64::MIN`. They're rejected before gimli sees them.
Stacked on #339. With #338 and #339, walrus writes correct DWARF for a parse and emit, but not after passes change the code, and a few input shapes still broke. This PR converts DWARF from where each input instruction was emitted, and fixes the remaining cases.
What was still wrong
DW_OP_addrwas treated as a code address. In Wasm it's a linear memory address of static data, so the location of every static or global variable got replaced, usually with the removed-code tombstone. This bug is already in 0.28.CodeTransform::instruction_mapmaps each input location to one output offset, so code that a pass copied (inlining, duplication) kept only one of its places. Locals that a pass replaced (coalescing) lost their locations.delegatehad no location, so its line row was lost.DW_AT_macros/DW_AT_macro_infopointed into.debug_macro/.debug_macinfo, which walrus never emitted because gimli can't write them.How it works now
EmitContext::function_instructions, which is crate-private.CodeTransformis unchanged.CodeMapmaps every input instruction to all of its emissions.low_pc/high_pcpair whose code is now in several places becomes aDW_AT_rangeslist.low_pcwithout a size,DW_AT_call_return_pc, …) move to the first instruction at or after them that's still emitted.DW_OP_WASM_locationlocals are looked up byLocalId. If a pass replaced the local, the local the input instructions now use is taken instead, by majority vote. That's per output function: anexprlocwhose scope's code is now in several functions with different local numbering becomes a location list over that code, with the right local in each. This is done for the attributes that can be location lists, such asDW_AT_locationandDW_AT_frame_base.DW_OP_addris left unchanged.expression.rs, the code-address mapping for expressions, is gone.Verification
I checked the output against ground truth from walrus's instruction map. The corpus is 20 inputs:
Each input ran through a plain parse and emit, through
gc, and through wasm-shrink's pass pipeline (inlining, local coalescing, DCE, …).llvm-dwarfdump --verifyThe same checker on #339 finds:
New tests:
dwarf_passes.rs: a plain emit (static address kept), a pass inserting code, a pass copying code (→DW_AT_ranges), a pass copying code that uses a local into another function under a new local (→ a location list for the variable), and a pass coalescing locals.dwarf_lines.rs: unreachable code,delegate, overlapping sequences, and code copied from another unit's function.Six of these fail on #339. I also fixed
dwarf_wasm_locations.rs, which putlow_pcat the body's size byte. LLVM'sDW_AT_low_pcis the body start, after the size.Cost
Emit time with
generate_dwarf:Without
generate_dwarf, emit time doesn't change.Not covered here
-fdebug-types-section). gimli's writer has no type-unit support: it writes DWARF 5 type units as compile units and drops.debug_types, soDW_FORM_ref_sig8references dangle. That needs a gimli change. With a patched gimli, type units verify clean in all modes..debug_aranges,.debug_pubnames,.debug_names) are dropped, as before. They're optional, and consumers build their own index.