Skip to content

cpfusa analyze CI job uses "|| true", silently swallowing 22 ERROR-severity findings (violates §20.1.2) #60

Closed

Description

@SoundMatt

Problem

The cpfusa analyze CI job cannot fail, because its command is suffixed with || true. On the current green main-branch run, this silently swallowed 22 ERROR-severity static-analysis findings plus 284 warnings.

Evidence

.github/workflows/ci.yml:319:

run: /tmp/cpfusa/build/cpfusa analyze || true

Job id 90483651198 on run 30422915741 (the latest green main-branch run at time of filing, triggered by the v2.18.0 merge PR #57) logs:

Summary: 22 error(s), 284 warning(s), 225 info(s)

yet the job's conclusion is success.

RELAY spec §20.1.2 explicitly requires: "An ERROR-severity finding... MUST fail the job."

Why it matters

22 ERROR-severity static-analysis findings are currently merged into main and shipped in the v2.18.0 release with no CI signal at all — the job shows green regardless of how many errors cpfusa analyze reports. This defeats the purpose of running the analyzer in CI and is a direct violation of the RELAY spec's continuous-conformance requirement for this gate.

Suggested fix

Remove the || true (or replace it with logic that greps the tool's own JSON/exit code for error-severity findings and exits non-zero) so the job actually fails when cpfusa analyze reports ERROR-severity findings, per spec §20.1.2. Then triage and fix (or explicitly waive, if the spec provides a waiver mechanism) the 22 existing errors before re-enabling the gate as blocking.

Activity

  1. SoundMatt commented on Aug 21, 2026

    @SoundMatt
    OwnerAuthor

    Re-verified against current main on 2026-08-21 (part of a parity audit against c-RCP): still open, ci.yml:445 unchanged. New scoping detail: re-ran cpfusa analyze locally (pinned v0.18.0) — all 26 current ERROR findings are noise from cppcheck scanning the vendored build/_deps/catch2-src tree (CMake FetchContent), flagging Catch2's own macros as unknownMacro errors. Scoping cppcheck to just include/ cli/ tests/ gives zero ERROR findings against cpp-RCP's own code — so the fix is a scope/exclude change, not a defect-fixing sprint. Tracked in #123.

  2. SoundMatt commented on Aug 23, 2026

    @SoundMatt
    OwnerAuthor

    Fixed as part of the v3.0.0 ground-up rewrite (Phase 7 batch 3, PR #169). .github/workflows/ci.yml's cpfusa-analyze job's run: line is now bare (run: /tmp/cpfusa/build/cpfusa analyze, line 742) — the || true this issue describes is gone, so an ERROR-severity finding now genuinely fails the job per RELAY spec §20.1.2. The job's real root cause (Catch2's vendored FetchContent source being misreported to cpfusa analyze via a stale compile_commands.json) was also fixed in the same pass via an explicit 'isolate compile_commands.json from vendored build tree' step. Independently re-verified against the current workflow file before closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions