Skip to content

fix: enforce regmap size invariant to close OOB writes (#64, #65) - #90

Merged
SoundMatt merged 1 commit into
mainfrom
fix/n2-01-02-regmap-oob-write
Jul 30, 2026
Merged

SoundMatt merged 1 commit into
mainfrom
fix/n2-01-02-regmap-oob-write

Conversation

@SoundMatt

Copy link
Copy Markdown
Owner

Summary

Fixes two memory-safety issues in rcp::regmap::Ep0 (include/rcp/regmap.hpp):

  • [cpp-RCP-N2-01] Ep0 config writes bounds-checked against the wrong vector — heap out-of-bounds write #64 (cpp-RCP-N2-01): Ep0's per-endpoint access checks (check_read_access, check_write_access, set_endpoint_owner) all bound a target endpoint against endpoint_owner_, which is sized from RegisterMap::endpoint_count at construction. The actual writes in write_generic_config/write_functional_config land in regs_.generic_configs/regs_.functional_configs instead — two vectors never required to be the same length as endpoint_count. A RegisterMap constructed with endpoint_count larger than those vectors (including the default-constructed empty-vector case) let an in-range access check pass and the subsequent write go out of bounds.
  • [cpp-RCP-N2-02] write_whole_map accepts mismatched vector sizes and never resizes endpoint_owner_ #65 (cpp-RCP-N2-02): write_whole_map let the root client move-assign an entirely new RegisterMap in with no check that its generic_configs/functional_configs sizes matched its own endpoint_count, and never resized endpoint_owner_ to match. A smaller replacement left endpoint_owner_ stale-and-too-large (OOB write on subsequent per-endpoint write); a larger replacement left it too small (new endpoints could never get an owner — silent DoS).

Fix

Both share one root invariant: generic_configs.size() == functional_configs.size() == endpoint_count. Added a shared private size_invariant_holds() check:

  • The Ep0 constructor now fails closed — if the incoming RegisterMap violates the invariant, endpoint_owner_ is sized to 0 instead of the untrustworthy endpoint_count, so every non-EP0 access check returns INVALID_PARAMETER instead of a false in-range verdict. A new is_endpoint_table_consistent() accessor lets callers detect this degraded state.
  • write_whole_map now validates the replacement map's invariant up front and rejects an inconsistent one with INVALID_PARAMETER (leaving the live map untouched), and resizes/resets endpoint_owner_ to match the new endpoint_count on acceptance — prior per-endpoint owner assignments are discarded since they describe endpoints in the map being replaced, not necessarily the same endpoints in the new one.
  • write_generic_config/write_functional_config additionally bounds-check against the actual vector being indexed (not just check_write_access's endpoint_owner_-based verdict) as a second, independent layer of defense.

No exceptions introduced — everything still returns std::error_code/RegMapErrc::invalid_parameter, matching the rest of this class.

Sanitizer verification (per the "prove it first" requirement)

I could not add a sanitizer build to a RCP_BUILD_TESTS CMake target and get a crash out of the existing test_regmap.cpp suite as it stood pre-fix — every existing test already builds RegisterMaps with consistent vector sizes via make_map(), so the existing suite never exercised the mismatched-size path at all (that's the bug: nothing enforced it). So, per the instructions, I wrote small standalone repros compiled directly against the unfixed header with clang++ -fsanitize=address,undefined, ran them, and confirmed a real crash before touching any code, then re-ran them after the fix.

Before fix — heap-buffer-overflow (issue #64, undersized generic_configs)

about to write_generic_config(target=4) into a 1-length generic_configs vector (endpoint_count=4)
=================================================================
==63049==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000001c90 at pc 0x000104957840 bp 0x00016b4b2470 sp 0x00016b4b2468
READ of size 8 at 0x603000001c90 thread T0
    #0 ... std::vector<unsigned short>::__vdeallocate() vector.h:889
    #1 ... std::vector<unsigned short>::__move_assign(...) vector.h:1020
    #2 ... std::vector<unsigned short>::operator=(...) vector.h:303
    #3 rcp::regmap::EndpointGenericConfig::operator=(rcp::regmap::EndpointGenericConfig&&) regmap.hpp:121
    #4 rcp::regmap::Ep0::write_generic_config(unsigned long, unsigned short, rcp::regmap::EndpointGenericConfig) regmap.hpp:421
    #5 main repro_n2_01b.cpp:31

0x603000001c90 is located 24 bytes after 24-byte region [0x603000001c60,0x603000001c78)
SUMMARY: AddressSanitizer: heap-buffer-overflow vector.h:889 in std::vector<unsigned short>::__vdeallocate()

(A second variant with a fully empty/default-constructed generic_configs/functional_configs — the more literal reading of "endpoint_count nonzero, config vectors default-constructed" from the issue — hit a null-pointer SEGV instead, also caught by ASan/UBSan: AddressSanitizer: SEGV ... in std::vector<unsigned short>::__vdeallocate(), plus several UBSan "member call on null pointer" reports leading up to it.)

Before fix — heap-buffer-overflow (issue #65, write_whole_map desync)

about to write_generic_config(target=4) after write_whole_map shrank generic_configs to size 1
=================================================================
==63051==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000001d20 at pc 0x000104351cec bp 0x00016bae21d0 sp 0x00016bae21c8
READ of size 8 at 0x603000001d20 thread T0
    #0 ... std::vector<unsigned short>::__vdeallocate() vector.h:889
    #1 ... std::vector<unsigned short>::__move_assign(...) vector.h:1020
    #2 ... std::vector<unsigned short>::operator=(...) vector.h:303
    #3 rcp::regmap::EndpointGenericConfig::operator=(rcp::regmap::EndpointGenericConfig&&) regmap.hpp:121
    #4 rcp::regmap::Ep0::write_generic_config(...) regmap.hpp:421
    #5 main repro_n2_02.cpp:44

0x603000001d20 is located 64 bytes after 32-byte region [0x603000001cc0,0x603000001ce0)
SUMMARY: AddressSanitizer: heap-buffer-overflow vector.h:889 in std::vector<unsigned short>::__vdeallocate()

Repro: Ep0 constructed against a consistent 4-endpoint map, root client installs a smaller (1-endpoint, consistent) replacement via write_whole_map; endpoint_owner_ stayed at its stale size of 4 pre-fix, so a subsequent write_generic_config(target=4) passed the stale bounds check and indexed 3 slots past the new 1-element generic_configs.

After fix — all three repros are ASan/UBSan-clean

$ ./repro_n2_01_after   # exit 1 — set_endpoint_owner correctly fails closed
set_endpoint_owner failed

$ ./repro_n2_01b_after  # exit 0 — rejected, no crash
about to write_generic_config(target=4) into a 1-length generic_configs vector (endpoint_count=4)
write_generic_config returned ec=4 (rcp/regmap: INVALID_PARAMETER)

$ ./repro_n2_02_after   # exit 0 — rejected, no crash
about to write_generic_config(target=4) after write_whole_map shrank generic_configs to size 1
write_generic_config returned ec=4 (rcp/regmap: INVALID_PARAMETER)

I then also built the real test_regmap binary (including the new regression tests below) via CMake with -DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer -g" -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address,undefined" (matching the new CI job below) and ran it directly:

Randomness seeded to: ...
===============================================================================
All tests passed (132 assertions in 23 test cases)

No ASan/UBSan reports, clean exit.

Regression tests (tests/test_regmap.cpp)

  • A RegisterMap with endpoint_count=4 and empty generic_configs/functional_configs now fails closed: is_endpoint_table_consistent() is false, and set_endpoint_owner/check_read_access/write_generic_config/write_functional_config all return INVALID_PARAMETER instead of an in-range verdict.
  • Same for a nonzero-but-undersized mismatch (endpoint_count=4, vectors resized to 1).
  • write_whole_map rejects a replacement map whose own generic_configs/functional_configs disagree with its endpoint_count, leaving the live map untouched.
  • write_whole_map with a valid replacement correctly resizes endpoint_owner_: growing (2→4 endpoints) discards stale owner assignments and lets the root client write the newly-valid endpoint 4; shrinking (4→1) makes endpoint 4 INVALID_PARAMETER again rather than leaving a stale in-range verdict.

All new tests are tagged with existing requirement IDs (REQ-REGMAP-005, REQ-REGMAP-013) since they reinforce those existing invariants more strictly rather than introducing new requirements — .fusa-reqs.json was intentionally left untouched, per this task's file-scope restriction to include/rcp/regmap.hpp, tests/test_regmap.cpp, and (additively) .github/workflows/ci.yml.

CI

Added a new, gating "ASan/UBSan (regmap)" job to .github/workflows/ci.yml (no || true): configures a separate build-asan CMake dir with -fsanitize=address,undefined, builds only test_regmap, and runs it with halt_on_error=1. Purely additive — no existing job touched or removed. Scoped to test_regmap per the task's "minimal/fast" guidance rather than sanitizer-ifying the whole suite.

Verification checklist

  • cmake -B build -DCMAKE_BUILD_TYPE=Debug -G Ninja && cmake --build build --parallel succeeds
  • ctest --test-dir build --output-on-failure — 53/53 tests pass (including 4 new regression tests in test_regmap)
  • ASan/UBSan repro crash captured on unfixed code (both issues), clean after fix
  • test_regmap built and run under -fsanitize=address,undefined via CMake — clean, 132 assertions / 23 test cases
  • clang-tidy diff-checked against the unfixed header with an explicit broad check set (the project has no .clang-tidy and CI's own clang-tidy step is non-gating via || true, so I verified manually): identical set of warning categories before/after, no new category introduced by this diff
  • DCO sign-off on the commit

Left unresolved / out of scope

  • .fusa-reqs.json was not touched (file-scope restriction for this task); the new tests trace to existing REQ-REGMAP-005/REQ-REGMAP-013 instead of new requirement IDs.
  • This PR does not audit other headers for the same access-check-vector-vs-storage-vector pattern; scope was strictly regmap.hpp per the assigned issues.

Closes #64, closes #65.

Ep0's per-endpoint access checks (check_read_access, check_write_access,
set_endpoint_owner) all bound a target endpoint against endpoint_owner_,
which is sized from RegisterMap::endpoint_count at construction. The
actual writes in write_generic_config/write_functional_config land in
regs_.generic_configs/regs_.functional_configs instead, which were never
required to be the same length as endpoint_count. A RegisterMap with
endpoint_count larger than those vectors let an in-range access check be
followed by an out-of-bounds vector write (cpp-RCP-N2-01, #64).
write_whole_map compounded this: it could install a replacement map with
a different endpoint_count without resizing endpoint_owner_ to match, so
a stale bounds check from the old map size could pass and then index out
of bounds on the new, differently-sized vectors, or new endpoints could
never get an owner assigned (cpp-RCP-N2-02, #65).

Both are fixed via a shared size_invariant_holds() check (generic_configs
.size() == functional_configs.size() == endpoint_count): the Ep0
constructor fails closed (sizes endpoint_owner_ to 0, rejecting all
per-endpoint access) when a RegisterMap violates it, and write_whole_map
now rejects an inconsistent replacement map outright and resizes
endpoint_owner_ to match a valid one. write_generic_config/
write_functional_config additionally bounds-check against the actual
vector being indexed as a second, independent layer of defense.

Verified against AddressSanitizer/UndefinedBehaviorSanitizer: standalone
repros reproduced a real SEGV/heap-buffer-overflow on the pre-fix code
for both issues, and are clean after the fix. Added regression tests
covering both fail-closed construction and write_whole_map resize/reject
behavior, and a new "ASan/UBSan (regmap)" CI job that builds and runs
test_regmap under -fsanitize=address,undefined on every PR.

Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com>
@SoundMatt
SoundMatt merged commit 7174bc6 into main Jul 30, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant