Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
152 commits
Select commit Hold shift + click to select a range
d7b7378
Add the Rust rewrite roadmap and the release-only build guards
Koukyosyumei Sep 13, 2026
8a707f8
Add test coverage targets to the roadmap
Koukyosyumei Sep 13, 2026
3002728
Add DecBench to the roadmap as the external decompiler benchmark
Koukyosyumei Sep 13, 2026
0c61ecc
M0: replace the C++ tree with a Rust workspace and r12e-core
Koukyosyumei Sep 13, 2026
236ff4d
M1: ELF loader with provenance-tagged function hints
Koukyosyumei Sep 13, 2026
064d534
M2: AArch64 decoder, at objdump parity over 1.4M instructions
Koukyosyumei Sep 13, 2026
62057e0
M3 and M8: analysis engine and the r12e command line
Koukyosyumei Sep 13, 2026
97dc7c4
M7: content anchors and the git-mergeable annotation log
Koukyosyumei Sep 13, 2026
bd30912
Record what is built in the roadmap
Koukyosyumei Sep 13, 2026
bd6df25
M2: x86-64 decoder, at llvm-objdump parity over 4,760 instructions
Koukyosyumei Sep 13, 2026
02f0be0
M3: jump table recovery
Koukyosyumei Sep 13, 2026
cbff467
M1: PE and COFF loader
Koukyosyumei Sep 13, 2026
5801d8f
M5: demanglers for Itanium, Rust and MSVC
Koukyosyumei Sep 13, 2026
ee311cf
M3: no-return propagation, and the PLT naming bug it exposed
Koukyosyumei Sep 13, 2026
a1fe597
M2: Advanced SIMD, and the exception-decode bug it turned up
Koukyosyumei Sep 13, 2026
a86277b
M9: MCP server, so an agent can drive the analysis
Koukyosyumei Sep 13, 2026
1001bd7
M10: binary diff, ranked so the answer is readable
Koukyosyumei Sep 13, 2026
5ef32f1
M11: a scorecard, including the numbers that go against us
Koukyosyumei Sep 13, 2026
4080c3c
M1: Mach-O loader, thin and fat
Koukyosyumei Sep 13, 2026
1c06dc1
M12: fuzzing in the ordinary test run, and the panic it found immedia…
Koukyosyumei Sep 13, 2026
1be7b62
Bring the README in line with what is built
Koukyosyumei Sep 13, 2026
8ca0878
M4: the IR, the AArch64 lifter, and the interpreter that proves it
Koukyosyumei Sep 13, 2026
5a55f8d
M4: SSA and the dataflow passes
Koukyosyumei Sep 13, 2026
9b9399a
M6: structuring and the pseudo-C emitter
Koukyosyumei Sep 13, 2026
5c960fe
M4: the x86-64 lifter, and an oracle that is a processor
Koukyosyumei Sep 13, 2026
9526514
M4: floating point in the IR, and the SIMD both architectures use
Koukyosyumei Sep 13, 2026
fc15e3c
M6: the decompiler, gated on whether a compiler accepts its output
Koukyosyumei Sep 13, 2026
e9721c2
M4: stack slots become variables, and the SSA bug that found
Koukyosyumei Sep 13, 2026
d87cb3f
Bring the roadmap and the scorecard in line with what is built
Koukyosyumei Sep 13, 2026
b814f58
M5: DWARF, a C type model, and output typed from what the compiler knew
Koukyosyumei Sep 13, 2026
4bc72ca
M5: structures recovered from how a pointer is used
Koukyosyumei Sep 13, 2026
c3bb1cf
M10: signature matching, and the coin flips it refuses to make
Koukyosyumei Sep 13, 2026
2abaa22
Raise the shape floor in the signature tests to match the code
Koukyosyumei Sep 13, 2026
ee0dae3
M10: a query language over the program model
Koukyosyumei Sep 13, 2026
766fcfa
M1 and M10: object files get their relocations, and a function can be…
Koukyosyumei Sep 13, 2026
2339986
Mark what the last few milestones actually finished
Koukyosyumei Sep 13, 2026
5671529
M5: C++ virtual tables, and the unbounded loops the fuzzer found
Koukyosyumei Sep 13, 2026
d87d495
Correct three roadmap items that were done but still unchecked
Koukyosyumei Sep 13, 2026
2706f50
M4: prototype recovery, and the argument setup that was being deleted
Koukyosyumei Sep 13, 2026
41723dc
Reserve the module slots for the PDB and runtime-metadata readers
Koukyosyumei Sep 13, 2026
c9a6fab
M5: the rest of clang's DWARF 5, and a cast that bound to the wrong t…
Koukyosyumei Sep 13, 2026
8b13225
M4: feedback edges, so the callers settle what a function cannot
Koukyosyumei Sep 13, 2026
23cb8db
M5: recovered structures reach the output
Koukyosyumei Sep 13, 2026
68294c6
M8 and M12: completions, a man page, and a manual
Koukyosyumei Sep 13, 2026
62ef8d1
M5: a PDB reader, producing the same shape DWARF does
Koukyosyumei Sep 13, 2026
fafa8d9
Add the two items that were missing from the roadmap
Koukyosyumei Sep 13, 2026
e7f2c95
Add the i386 decoder as its own roadmap item
Koukyosyumei Sep 13, 2026
cb1901f
M5: Go, Rust and Objective-C runtime metadata
Koukyosyumei Sep 13, 2026
61d20c2
Give a metadata key a space before its value
Koukyosyumei Sep 13, 2026
92d11e2
M4: range analysis, and the two things it found in the jump tables
Koukyosyumei Sep 13, 2026
20b09ed
M9: batch mode, one analysis and one document
Koukyosyumei Sep 13, 2026
3f31998
M0: CONTRIBUTING.md, with the clean-room rule and what a change has t…
Koukyosyumei Sep 13, 2026
a8e24d9
M1: an ARM32 and Thumb-2 decoder, checked against llvm-objdump
Koukyosyumei Sep 13, 2026
26c7bcb
M1: the system register table, from the assembler rather than by hand
Koukyosyumei Sep 13, 2026
7fc54f5
M6: ar archives, overlays and entropy, with an r12e archive and r12e …
Koukyosyumei Sep 13, 2026
aedcfdb
M8: patch sets and project files, with r12e patch and r12e project
Koukyosyumei Sep 13, 2026
447e4e8
M7: a port of Ghidra's decompiler datatests, and the first bug it found
Koukyosyumei Sep 13, 2026
8a94811
M7: run the decompiled C and compare it against the machine
Koukyosyumei Sep 13, 2026
a034984
Document the archive, overlay, patch and project commands, and the nu…
Koukyosyumei Sep 13, 2026
bd1bff9
M0: a design document per subsystem
Koukyosyumei Sep 13, 2026
8c7703f
M7: the annotation store's design document, which the roadmap asks fo…
Koukyosyumei Sep 13, 2026
4375022
Point the roadmap at the design documents that now exist
Koukyosyumei Sep 13, 2026
990c9f7
M0: the tutorial, which ends at the step the incumbents cannot do
Koukyosyumei Sep 13, 2026
ea61006
Make the tutorial's transcripts the real output, and its numbers meas…
Koukyosyumei Sep 13, 2026
b9f9d47
Bring the roadmap's checkboxes up to what is actually built
Koukyosyumei Sep 13, 2026
523515a
CI installs the oracles, and prints which ones it found
Koukyosyumei Sep 13, 2026
b640614
M8: paging, colour, and a listing that fits the terminal without losi…
Koukyosyumei Sep 13, 2026
d344ca5
Document the colour and pager options
Koukyosyumei Sep 13, 2026
0c699e4
M3: x86-64 jump tables, and a bound that was silently wrong
Koukyosyumei Sep 13, 2026
fd5f6bf
M4: a comparison keeps the width the machine compared at
Koukyosyumei Sep 13, 2026
a87655e
M2: the .sla format, worked out from the files, and a reader for it
Koukyosyumei Sep 13, 2026
18b1885
M1: Windows entry points, and a stride that was wrong on ARM64
Koukyosyumei Sep 13, 2026
44b2362
Roadmap: Windows entry points done, .sla loading done and decoding not
Koukyosyumei Sep 13, 2026
0edd06e
M12: the resource limit policy, and the justification for the one uns…
Koukyosyumei Sep 13, 2026
30043ce
M9: a budget, so a long job gives back what it finished
Koukyosyumei Sep 13, 2026
93a3900
M12: the first DecBench run, which we lose
Koukyosyumei Sep 13, 2026
af62244
M2: i386, the 32-bit mode of the x86 decoder
Koukyosyumei Sep 13, 2026
6c075c9
M4: a byte register is part of its register, not storage of its own
Koukyosyumei Sep 13, 2026
b0ee8ab
M5: DWARF inlined frames, call sites and location lists, and a Swift …
Koukyosyumei Sep 13, 2026
1cadc87
M5: read a C declaration, and let an asserted one outrank the recovery
Koukyosyumei Sep 14, 2026
2a5bfc6
M4: a sign extension reads its source as signed
Koukyosyumei Sep 14, 2026
badddb4
M10: query over dataflow facts
Koukyosyumei Sep 14, 2026
314ce92
M2: an assembler for x86-64 and AArch64, gated by round trip
Koukyosyumei Sep 14, 2026
30a6ced
M10: r12e patch record --asm, so a patch is written as an instruction
Koukyosyumei Sep 14, 2026
23ed79a
M2: the SLEIGH front end, over every language Ghidra ships
Koukyosyumei Sep 14, 2026
f2b2a62
M6: nothing is dropped from the decompiled output any more
Koukyosyumei Sep 14, 2026
cec3e24
Make the roundtrip gate a ratchet, and stop the determinism gate abor…
Koukyosyumei Sep 14, 2026
396c670
M11: a lazy session, an on-disk cache, and the memory measurement
Koukyosyumei Sep 14, 2026
4e5c2e2
Roadmap: tick the nine items that landed and were measured but never …
Koukyosyumei Sep 14, 2026
58e5a96
M4: fold a compiler's reciprocal division back into a division
Koukyosyumei Sep 14, 2026
a178480
M5: an asserted declaration reaches the decompiled output
Koukyosyumei Sep 14, 2026
58f91cd
M8: wire the annotation log's type assertions into decompilation
Koukyosyumei Sep 14, 2026
56f5036
M1: i386 ELF relocations, and a confident zero that was wrong
Koukyosyumei Sep 14, 2026
02a014e
M6: place a block once, and copy a shared tail rather than jumping to it
Koukyosyumei Sep 14, 2026
df20a7c
M6: one function decompiles the same alone as in a whole program
Koukyosyumei Sep 14, 2026
a00a0fd
M2: the SLEIGH runtime, and RISC-V decoded at zero wrong
Koukyosyumei Sep 14, 2026
ec00690
Scorecard: the numbers from the last day of work
Koukyosyumei Sep 14, 2026
cc6d857
Roadmap: the SLEIGH runtime, i386 relocations and structure feedback
Koukyosyumei Sep 14, 2026
05fdd61
M2: a SLEIGH compiler, as far as the format is established
Koukyosyumei Sep 14, 2026
47d310f
M5: C++ classes from RTTI, and a rule that was naming the wrong class
Koukyosyumei Sep 14, 2026
4fbb385
Correct what the model says about operand numbering in a compiled file
Koukyosyumei Sep 14, 2026
05b4da5
M8: an r12e classes command, and apply a shared library's relocations
Koukyosyumei Sep 14, 2026
141b5e6
Roadmap: C++ class recovery and the SLEIGH compiler
Koukyosyumei Sep 14, 2026
77ab4c3
The dataflow query fixture, and the lock file the new crates need
Koukyosyumei Sep 14, 2026
bb536fa
M12: static musl builds, and a release workflow that has been run
Koukyosyumei Sep 14, 2026
6cac68f
M10: diff at instruction granularity, and signatures from distributio…
Koukyosyumei Sep 14, 2026
52f10ab
M10: emulate a selected path, with stubs that answer rather than act
Koukyosyumei Sep 14, 2026
c99c57d
Record the AArch64 byte-table jump form that recovery does not model
Koukyosyumei Sep 14, 2026
8b0b1ae
M5: PDB, measured against a real database rather than a synthetic one
Koukyosyumei Sep 14, 2026
99c9a54
M5: find the program database a PE names, and refuse the wrong one
Koukyosyumei Sep 14, 2026
e7b8e44
M4: detect the convention a function uses rather than assuming one
Koukyosyumei Sep 14, 2026
93f66f4
M4: lift what a growing corpus exposed, and check each one on a proce…
Koukyosyumei Sep 14, 2026
605e658
Raise both lifting floors to what is now measured
Koukyosyumei Sep 14, 2026
fefba07
M6: declare every name the emitter can write
Koukyosyumei Sep 14, 2026
4d5af8d
M11: the memory ceiling, met and measured
Koukyosyumei Sep 14, 2026
67c762b
M6: sink a shared tail instead of jumping to it
Koukyosyumei Sep 14, 2026
03d0db8
Roadmap: the memory ceiling, calling conventions and path emulation
Koukyosyumei Sep 14, 2026
25b7f4e
M8: an interactive session, over one analysis
Koukyosyumei Sep 14, 2026
0ca536e
M12: one no-panic gate over every path that reads foreign bytes, and …
Koukyosyumei Sep 14, 2026
503f6bd
M12: G4, measured against DWARF over 95,697 real functions
Koukyosyumei Sep 14, 2026
5f01434
M6: 8,898 gotos to 7,177, and the 318 of them that were wrong code
Koukyosyumei Sep 14, 2026
95d96cf
Analysis: the anchored compact jump table, a data map, and progress f…
Koukyosyumei Sep 14, 2026
3543da1
M11: measured against rizin 0.8.2, with a budget that holds the numbers
Koukyosyumei Sep 14, 2026
259f1a0
M6: a loop exit inside a switch arm is not a `break`
Koukyosyumei Sep 14, 2026
d5fb378
M6: seven more Ghidra datatests pass, and a cast C never needed
Koukyosyumei Sep 14, 2026
792df7e
CI gates recall, not speed: delete the bench workflow
Koukyosyumei Sep 14, 2026
7c4f112
Roadmap: datatests 44 passing to 56, five defects left
Koukyosyumei Sep 14, 2026
411492a
M6: a 32-bit `lea` named a value nothing wrote. Roundtrip 39 to 7.
Koukyosyumei Sep 14, 2026
7e4354e
M6: an argument is as wide as the body reads it, not as wide as the r…
Koukyosyumei Sep 14, 2026
a5a92a2
SLEIGH: 0 wrong on every architecture, and every instruction lifts co…
Koukyosyumei Sep 14, 2026
36eb9d2
M6: a `float` argument is a `float`, and reading it is reading it
Koukyosyumei Sep 14, 2026
613860d
M6: the roundtrip ratchet reaches zero
Koukyosyumei Sep 14, 2026
084b7ec
Docs: the roundtrip gate is at zero, so two claims about it are no lo…
Koukyosyumei Sep 14, 2026
efced07
IR: the reciprocal recognizer searched 2^24 states over a graph of hu…
Koukyosyumei Sep 14, 2026
16cdbf8
M6: a dereferenced argument is a pointer, and what DecBench could not…
Koukyosyumei Sep 14, 2026
1722ffc
i386 lifting: the same rules at half the pointer width
Koukyosyumei Sep 14, 2026
a5d6cdb
ARM32 lifting, checked against a processor
Koukyosyumei Sep 14, 2026
7b6c773
ARM function recovery: Thumb is an instruction set the loader now dec…
Koukyosyumei Sep 14, 2026
0068950
Thumb: a load into the program counter is a branch, and was rejected
Koukyosyumei Sep 14, 2026
d5c3dc9
ARM: a swept gate over the encoding space, and what it found
Koukyosyumei Sep 15, 2026
f856cb7
CI on main and pull requests, and install.sh instead of a formula nob…
Koukyosyumei Sep 15, 2026
f65c471
README: rewritten, and `r12e emulate` could not be run at all
Koukyosyumei Sep 15, 2026
648ae61
Remove the MCP server: one agent interface, and it is the complete one
Koukyosyumei Sep 15, 2026
13e9933
Rename the project from r12e to e5r
Koukyosyumei Sep 15, 2026
8fe6eba
CI: a fixture named .a64 has to be AArch64, and two jobs that gate no…
Koukyosyumei Sep 15, 2026
fe2d47b
AArch64: every 32-bit `orr Rd, ZR, #imm` printed as `mov`
Koukyosyumei Sep 15, 2026
a5e6d89
Three more fixtures that were not the architecture their name claimed
Koukyosyumei Sep 15, 2026
728eca6
Seven arithmetic overflows, and oracles that answered nothing to ever…
Koukyosyumei Sep 15, 2026
3fed013
Fixtures and oracles that followed the host, and a PLT form we did no…
Koukyosyumei Sep 15, 2026
ba5bb9f
movz was never lifted, .plt.sec was paired wrong, and a floor made of…
Koukyosyumei Sep 15, 2026
28183e0
.plt.sec named by what it jumps through, and a hang test that measure…
Koukyosyumei Sep 15, 2026
7c733df
A size from the file became a quarter of a billion iterations
Koukyosyumei Sep 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
*.e5r-patch merge=union
*.e5r-proj merge=union
165 changes: 165 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
name: ci

# `main` and pull requests, and nothing else.
#
# A branch name in this list is a name that will be wrong later: it was
# `full-rewrite-to-rs`, which is one branch of one period of this repository's
# life, and every other branch went unchecked while that one burned a runner on
# every push. Work in progress is checked when it is proposed, which is what a
# pull request is; a push to a branch nobody has proposed is not yet a claim
# that anything works.
on:
push:
branches: [main]
# Any base branch, so a feature branch proposed against anything is checked.
pull_request:

env:
CARGO_TERM_COLOR: always
RUSTFLAGS: -D warnings

# One in-flight run per ref. Pushing a fix ten seconds after a typo should not
# leave the first run holding a runner; `main` is exempt so a merge is never
# cancelled by the merge behind it.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}

jobs:
# Runners are disposable, so these build the dev profile on purpose. The
# release-only rule in CLAUDE.md is about the maintainer's machine.
check:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt, clippy
- uses: Swatinem/rust-cache@v2
- run: cargo fmt --all --check
- run: cargo clippy --workspace --all-targets -- -D warnings

# G2, G3, G4, G6, G7, G9: the whole suite, with the corpus built so the
# oracle-backed gates actually run. Without fixtures they skip silently.
test:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
# The AArch64 std, for the Rust fixture: `build-fixtures.sh` builds it
# for a fixed target so the counts the tests hold do not follow the host.
- uses: dtolnay/rust-toolchain@stable
with:
targets: aarch64-unknown-linux-gnu
- uses: Swatinem/rust-cache@v2
# Every gate that has no tool skips silently, which is the worst
# possible outcome: a green run that measured nothing. These are what
# the oracle-backed gates need, named so a missing one is visible.
# clang, llvm, binutils the fixture corpus and objdump parity
# llvm-objdump the ARM32 and x86-64 parity oracles
# gcc-aarch64-linux-gnu the `.a64` corpus. This runner is x86-64, so
# without it plain `gcc` writes an x86-64 binary
# into a name claiming AArch64, and the tests
# that read one measure the wrong file
# qemu-user-static both halves of the lifter's hardware oracle,
# and the recorded output of every executed
# AArch64 fixture
# golang-go the Go fixture, which found five decoder bugs
- name: Install the oracles
run: |
sudo apt-get update
sudo apt-get install -y clang llvm binutils qemu-user-static golang-go \
gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
- name: Build the fixture corpus
run: ./scripts/build-fixtures.sh
- name: Say which oracles are present
run: |
for t in clang llvm-objdump objdump readelf c++filt qemu-x86_64 go rustc; do
printf '%-16s %s\n' "$t" "$(command -v $t || echo MISSING)"
done
# `--no-fail-fast`: without it cargo stops at the first failing test
# binary, so a runner that differs from the maintainer's machine in
# several ways reports them one push at a time.
- run: cargo test --workspace --no-fail-fast -- --nocapture

# G5: the same bytes must produce the same output at any thread count.
determinism:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
# The AArch64 std, for the Rust fixture: `build-fixtures.sh` builds it
# for a fixed target so the counts the tests hold do not follow the host.
- uses: dtolnay/rust-toolchain@stable
with:
targets: aarch64-unknown-linux-gnu
- uses: Swatinem/rust-cache@v2
# The cross gcc because this runner is x86-64 and a `.a64` fixture
# has to be AArch64; qemu because several of them are executed.
- run: |
sudo apt-get update
sudo apt-get install -y clang \
gcc-aarch64-linux-gnu g++-aarch64-linux-gnu qemu-user-static
- run: ./scripts/build-fixtures.sh
- run: cargo build --release
- run: ./scripts/check-determinism.sh

# G8: what the analysis FOUND, against the floors in scripts/bench-budget.json.
#
# Not a benchmark, and deliberately not run as one. Recall against the symbol
# table and the count of entries no symbol names are the same bytes in, same
# numbers out, on any machine at any thread count, so they gate. Wall time
# and peak memory are facts about a machine, and a shared runner with a noisy
# neighbour is not the machine the ceilings were recorded on -- so timing is
# not measured here, not gated here, and not published from here. It is
# measured by hand on a known machine and written down in docs/benchmarks.md.
#
# One run per binary for the same reason: the fastest of three exists to beat
# down timing noise, and there is no timing to beat down.
recall:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
# The AArch64 std, for the Rust fixture: `build-fixtures.sh` builds it
# for a fixed target so the counts the tests hold do not follow the host.
- uses: dtolnay/rust-toolchain@stable
with:
targets: aarch64-unknown-linux-gnu
- uses: Swatinem/rust-cache@v2
# The cross gcc because this runner is x86-64 and a `.a64` fixture
# has to be AArch64; qemu because several of them are executed.
- run: |
sudo apt-get update
sudo apt-get install -y clang binutils \
gcc-aarch64-linux-gnu g++-aarch64-linux-gnu qemu-user-static
- run: ./scripts/build-fixtures.sh
# Release, like determinism above: the recall numbers have to come from
# the code path a user runs.
- run: cargo build --release
- run: RUNS=1 SKIP=rizin OUT=target/recall.json ./scripts/compare-tools.sh
- run: ./scripts/check-bench-budget.py target/recall.json --portable-only --no-timing

# `install.sh` is what people pipe into a shell, and nothing else in this
# workflow would notice it rotting: it is not built, not linked and not
# imported. Two checks, both cheap.
#
# `sh -n` is the syntax gate. `shellcheck` is the rest of it -- an unquoted
# expansion in an installer run as root is not a style note -- and it is
# pinned to POSIX `sh` because the shebang is `sh`, not bash.
installer:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- run: sh -n install.sh
- run: sudo apt-get update -q && sudo apt-get install -y shellcheck
- run: shellcheck --shell=sh install.sh
# The one thing syntax cannot see: that the script still knows where the
# release puts things. `build-release.sh` names the archive and the
# directory inside it, and the installer reaches into that directory by
# name -- so a change to either that forgot the other is a download that
# unpacks into a path nobody looks at.
- name: The installer agrees with the release layout
run: ./scripts/check-installer.sh
Loading
Loading