Skip to content

fix: CI gating defects and safety-doc corrections (#80 #81 #67 #68 #83 #82 #78 #79 #69) - #92

Merged
SoundMatt merged 2 commits into
mainfrom
fix/ci-gating-and-safety-docs
Jul 30, 2026
Merged

SoundMatt merged 2 commits into
mainfrom
fix/ci-gating-and-safety-docs

Conversation

@SoundMatt

Copy link
Copy Markdown
Owner

Summary

Process/CI/documentation fixes only. No C++ source, tests, or other docs
touched (verified via diff review — see stat below). Scope:
.github/workflows/ci.yml, .github/workflows/release.yml, HARA.md,
.fusa-hara.json, tara.json/tara.md (relocated to tooling/), and one
new file, .clang-tidy (explicitly called for by issue #80's own text —
see below).

Deliberately untouched per instructions: issue #60 / .github/workflows/ci.yml's
cpfusa analyze || true step, and anything scoped only to issues #23, #58,
#59, #61, #62.

 .clang-tidy                    |  59 +++++++++
 .fusa-hara.json                |   8 +-
 .github/workflows/ci.yml       |  71 +++++++++--
 .github/workflows/release.yml  | 100 +++++++++++++--
 HARA.md                        |  32 +++--
 tara.json => tooling/tara.json |   5 +-
 tara.md => tooling/tara.md     |   5 +-

#80 (cpp-RCP-12) — clang-tidy || true

Removed || true from the clang-tidy job's Run clang-tidy step.

Important finding: before this change, clang-tidy -p build --warnings-as-errors='*' was invoked with no -checks argument and
no .clang-tidy file present anywhere in the repo. With nothing
enabling any checks, clang-tidy exits immediately with the fatal usage
error Error: no checks enabled — meaning || true wasn't just swallowing
findings, it was swallowing a hard configuration failure. This job has
never actually linted a single file in this repo's history.

Fix: added .clang-tidy at the repo root enabling a narrow, curated
subset of bugprone-* checks (use-after-move, infinite loops,
dangling handles, sizeof/memset/string-compare misuse, swapped
arguments, virtual near-misses, etc. — 21 checks total, see the file's
own header comment for the full list and rationale) plus a
HeaderFilterRegex restricting findings to this project's own
include/rcp/ headers (so vendored FetchContent deps like Catch2 don't
get flagged).

This check subset was verified locally (Homebrew LLVM 22 clang-tidy,
since the sandbox has no clang-tidy-14/gcc-12 to exactly match CI) to
have zero findings against the current include/ + tests/ tree.
Broader categories were tested and rejected because they all have
real, pre-existing findings that require C++ source changes out of this
PR's scope:

  • Full bugprone-*: 827 findings (mostly bugprone-throwing-static-initialization
    in test fixtures and bugprone-easily-swappable-parameters)
  • clang-analyzer-*: 34, performance-*: 106, portability-*: 87,
    cert-*: 669, modernize-*/readability-*: 700+ each
  • Even clang-diagnostic-* alone is not currently clean:
    CMakeLists.txt already passes -Wall -Wextra -Wpedantic, and a plain
    local build surfaces ~34 pre-existing -Wmissing-field-initializers
    warnings (e.g. tests/test_legacy_mock.cpp) that don't fail the build
    today only because there's no -Werror. Enabling clang-diagnostic-*
    in the tidy gate would have made it red on day one for reasons unrelated
    to this PR.

Not fully verified: I don't have clang-tidy-14/gcc-12 locally to
byte-for-byte match CI's toolchain, so there's residual risk the exact
finding set differs slightly on Ubuntu/clang-14 vs. my local
AppleClang-compiled-code + Homebrew-LLVM-22-clang-tidy setup. I'm fairly
confident given the checks chosen are AST-pattern based (version-stable),
but flagging this as not 100% verified in the target environment.

.clang-tidy is not in the originally-specified allowed-file list, but
issue #80's own remediation text explicitly calls for it: "narrow
--warnings-as-errors to a defensible check subset via .clang-tidy
config."
There's no other mechanism to narrow clang-tidy's check set, so
I added it as a minimal, directly-authorized exception — flagging this
explicitly per the instructions.

#81 (cpp-RCP-13) — additional || true masking

For every location, decided whether it should gate or is legitimately
advisory, and made the step name/comment say so explicitly:

  • ci.yml cpfusa trace --gaps: kept || true, renamed to
    "Traceability gaps report (informational, non-gating)" with a comment
    pointing at the real gate (cpfusa trace --req-coverage 100, the very
    next step, which already runs without || true).
  • ci.yml cpfusa coverage --dal DAL-B (in the cpfusa report (compliance) job): kept || true. This project's stub bridges can't
    reach the 100%-branch DAL-B threshold this command grades against
    (same reasoning release.yml's own comment already gives for its
    copy of this command), so making it a hard gate would permanently red
    this job for reasons outside this PR's scope. Renamed to spell out
    "artifact only — non-gating" and added a comment. The job's needs:
    list already includes every job that actually gates
    (cpfusa-check/-lint/-cyber/-trace/-verify), so this job's own
    steps are correctly understood as aggregation/artifact-bundling, not an
    additional gate.
  • ci.yml cpfusa report --format json/sarif, cpfusa metrics record: kept || true, renamed to spell out "artifact only —
    non-gating" — these are format conversions / a metrics snapshot, not
    pass/fail-shaped operations.
  • release.yml ISO 26262/IEC 61508/DO-178C gap reports + structural
    coverage report
    (lines ~81-100): kept || true per the issue's own
    guidance (gating lives in ci.yml). Renamed each step to add "artifact
    only — non-gating" explicitly, per the issue's ask that this not rely
    solely on the existing comment surviving future edits.
  • release.yml cpfusa report --format json/html: same treatment,
    renamed + comment added.

New, genuinely gating step added in ci.yml's cpfusa-report job:
"Assert branch coverage data is non-vacuous" — a small jq check that
fails the job if coverage-report.json's summary.totalBranches <= 0.
This is the mechanism that actually closes the "0/0 vacuously passes"
concern from #67/#68, independent of whether the DAL-B threshold itself
gates.

#67 / #68 (cpp-RCP-N2-04/05) — coverage collection

In both ci.yml's coverage job and release.yml's duplicate lcov
invocation (kept in sync since they run the identical commands):

  1. Added --rc lcov_branch_coverage=1 to both the --capture and
    --remove lcov invocations (lcov rewrites branch records on every
    invocation, so it must be on both, not just one).
  2. Added /usr/include/* to the lcov --remove exclusion list, alongside
    the existing */tests/*, */catch2/*, */FetchContent/*,
    */cmake-build*/* globs.
  3. Added the jq-based "totalBranches > 0" hard-fail assertion described
    above.

Verification: I don't have gcov-12/a Linux toolchain locally, so I
could not do a full end-to-end coverage capture. I did verify:

  • The lcov --rc lcov_branch_coverage=1 --remove ... --output-file
    command syntax is accepted by a locally-installed lcov (Homebrew,
    v2.4.0) against a hand-built minimal .info file — it ran successfully
    and correctly filtered branch/line data. Note lcov 2.x prints a
    deprecation warning for lcov_branch_coverage (renamed branch_coverage
    in 2.x) but still honors it; the Ubuntu 22.04 apt lcov package is
    1.x, where lcov_branch_coverage is the only spelling, so this is the
    correct choice for CI.
  • The jq '.summary.totalBranches' path is correct against the actual
    coverage-report.json present in this repo (.summary.totalBranches,
    .summary.hitBranches, top-level .branchPct/.thresholdBranch).

#83 (cpp-RCP-15) — unpinned CI dependencies

  1. ilammy/msvc-dev-cmd@v1 → pinned to
    0b201ec74fa43914dc39ae48a89fd1d8cb592756 # v1 (resolved via
    gh api repos/ilammy/msvc-dev-cmd/git/refs/tags/v1).
  2. RELAY checkout in the relay-conform job → pinned to
    3344b3d1a6a4d870d26c81c6f3c3ff5ca2930e16 # v2.0.2, the latest tagged
    RELAY release at time of writing (gh api repos/SoundMatt/RELAY/tags).
    I checked ROADMAP.md for any stated "always track latest RELAY main"
    intent and found none — the job's own name/comment ("RELAY §12
    conformance", "authoritative conformance validator") doesn't say it's
    meant to float either. Given RELAY v2.0.0 is itself a breaking change
    this whole cpp-RCP repo is currently being brought into conformance
    with, pinning to the latest release (rather than an older one) keeps
    the gate meaningful without floating on an arbitrary future commit.

#82 (cpp-RCP-14) — direct bot push to main

Previously: regenerate-safety-artifacts committed regenerated evidence
directly to main as github-actions[bot] with [skip ci], bypassing
both review and CI.

Chose option (b): keep committing the files into the repo tree (not
artifact-upload-only), but route through a normal PR instead of a direct
push. Reasoning for not choosing artifact-only: I found clear evidence
in this repo that downstream tooling/process relies on these files being
live in the tree, not just attached to a release:

  • sas.md, safety-case.md, and tara.md (now tooling/tara.md) all
    reference each other's root-level filenames as "present" at those
    paths.
  • ROADMAP.md's milestone-61 entry explicitly says these files "are
    deliberately left untouched by this PR... per .github/workflows/ release.yml, every one of them is mechanically regenerated by cpfusa
    against the current... state on the next tag push, so hand-editing them
    here would just be overwritten" — and cites a prior regeneration commit
    (e7a8178) as the established, expected pattern.

So the fix is: the job now commits to a safety-artifacts/<tag> branch
and opens a PR (gh pr create) against main instead of pushing directly.
This also means these commits will get a real CI run before merging
(dropping [skip ci]), which is a bonus improvement — regenerated safety
evidence was previously never CI-validated before landing on main.

#78 / #79 (cpp-RCP-10/11) — tara.json/tara.md vs TARA-ANALYSIS.md

#78 (labeling/scope ambiguity) — durably fixed. Relocated
tara.json/tara.md (cpp-FuSa's own tooling/CI/supply-chain-surface
TARA) to tooling/tara.json/tooling/tara.md, and updated
release.yml's Regenerate TARA step to mv the freshly-generated files
into tooling/ right after cpfusa tara runs (the command has no
--output flag, unlike iso26262/iec61508/etc., so redirect-by-move is
the only option). This survives every future regeneration on a tag, not
just this one, because the workflow step itself now writes to the new
path. TARA-ANALYSIS.md (the real, hand-authored, protocol-level product
TARA) remains at the repo root, unchanged.

I did not add a cross-reference pointer note in TARA-ANALYSIS.md (or
a new root-level note file) as the issue text also suggested, because
TARA-ANALYSIS.md and any new root-level file are both outside this
workstream's strict allowed-file scope
(.github/workflows/*, tara.json/tara.md, HARA.md,
.fusa-hara.json). The relocation itself is the durable fix and makes the
tooling-only scope self-evident from the path alone (per the issue's own
framing); the pointer note is a nice-to-have for discoverability that I'm
flagging as a small, explicit gap for a maintainer or a docs-scoped PR to
pick up.

#79 (factually-wrong content) — bonus, non-durable, partially done.
Confirmed the specific inaccuracies the issue called out: cmake/FetchDeps.cmake
only declares Catch2 via FetchContent (not CLI11/nlohmann-json — checked,
greped for both, found neither anywhere in the repo's CMake files); there
is no .fusa-dispositions.json file anywhere in this repo; and there is no
sign/slsa command or step anywhere in ci.yml/release.yml (confirmed
provenance.json's own format field is "x-FuSa provenance v1", not
actual SLSA). Hand-corrected tooling/tara.json/tooling/tara.md:
removed the "sign"/"SLSA" mitigation bullets from TARA-001 and TARA-003,
and fixed TARA-004's asset description to just "FetchContent: Catch2".
Left TARA-008 (.fusa-dispositions.json waiver log) as-is — removing a
whole threat entry felt like a bigger, riskier edit than a factual-field
correction, so I'm flagging it as a known inaccuracy rather than silently
deleting a finding from a compliance artifact.

This content fix does not survive the next tag. tara.json/tara.md
are wholly regenerated by cpfusa tara on every tag push, so these
hand-edits will be silently overwritten the next time release.yml runs.
Only the #78 relocation (which changes the workflow step's own output
path
, not just file content) is durable across regeneration.

#69 (cpp-RCP-N2-06) — HARA.md H-001 ASIL misclassification

Confirmed via the ISO 26262-3 S/E/C→ASIL determination table: S3/E4/C2
maps to ASIL-C, not ASIL-B, and the document's own rationale text
already stated no decomposition was being claimed to justify the
reduction. Fixed:

  • H-001's row in the Hazard Table: ASIL-B → ASIL-C
  • The document's "Target ASIL" line: ASIL-B (derived) → ASIL-C (derived)
  • SG-001, SG-003, SG-007 in the Safety Goals table: ASIL-B → ASIL-C (these
    are the request-delivery/watchdog/liveness safety goals H-001's own
    decomposition rationale names as the mechanisms — watchdog::Manager/
    StreamWatchdog for SG-003, deadline::Monitor for SG-007 — so they
    inherit H-001's corrected level)
  • The H-001 rationale section: header, "required ASIL is C", decomposition
    intro line, and the two per-mechanism (ASIL-B) labels → (ASIL-C),
    since leaving them at B while claiming "no decomposition" to reach a
    higher-rated hazard would be internally inconsistent
  • Mirrored all of the above in .fusa-hara.json (H-001's risk.asil,
    and SG-001/SG-003/SG-007's asil fields)

Sanity pass on other rows (per the issue's ask): recomputed every
hazard row's S/E/C→ASIL mapping against the same table.

  • H-003, H-006, H-009, H-010, H-011: correct as rated.
  • H-002, H-004, H-005, H-007 (all S2/E3/C2) and H-008 (S3/E2/C2):
    the table technically maps these to ASIL-A (H-002/004/005/007) and
    ASIL-A (H-008), but all five are currently rated ASIL-B —
    over-rated, not under-rated. Since assigning a higher ASIL than the
    literal table minimum is the conservative (safer) direction rather than
    a defect, and the issue's own instructions say not to relitigate rows
    that may be "intentionally conservative," I left these five unchanged.
    Flagging explicitly here per the instructions: I did not confirm with
    anyone that this over-rating is intentional (vs. just copy-paste from
    H-003/H-006's genuinely-B-rated S2/E4/C2 combination) — a maintainer
    may want to either downgrade these five to A or add an explicit
    conservative-margin rationale note. I did not touch them either way.

Verification performed

  • python3 -c "import yaml,sys; yaml.safe_load(open(...))" — both
    workflow files parse.
  • actionlint (installed via Homebrew) — no new findings; the one
    existing shellcheck SC2038 warning on the clang-tidy find | xargs
    line is pre-existing (confirmed by running actionlint against the
    original file at 4b9449a — same warning, same line content, only the
    line number shifted).
  • jq . tooling/tara.json and jq . .fusa-hara.json — both valid JSON.
  • Diff review: only the 7 files listed above changed; no C++
    source/tests/other docs touched.
  • No ctest/C++ build was run for this PR per the task's own note that
    this workstream doesn't require one — confirmed no build files were
    touched.

Closes

#80, #81, #67, #68, #83, #82, #78, #79, #69

Explicitly not touched: #60, #23, #58, #59, #61, #62.

…#82 #78 #79 #69)

Fixes a batch of process/CI/documentation issues, all scoped to
.github/workflows/*.yml, HARA.md/.fusa-hara.json, and tara.json/tara.md.
No C++ source, tests, or other docs are touched. Full accounting per
issue is in the PR description.

Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com>
Passing include/*.hpp directly to clang-tidy alongside tests/*.cpp made
every header fail with a clang-diagnostic-error ('file not found' on its
own #includes), since a header-only library's .hpp files have no
standalone entry in compile_commands.json to supply -I flags from. That
made the newly-degated clang-tidy job fail outright rather than surface
real findings.

Restrict the find to tests/*.cpp, which do have real compile commands
(via the rcp::rcp INTERFACE target); .clang-tidy's existing
HeaderFilterRegex already surfaces findings from include/rcp/ headers
transitively included by those translation units, so header coverage is
unchanged.

Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant