Skip to content

x-FuSa spec §1.6.1 ambiguity: does FUSA-STUB001/002 'participate in check's exit code' mean check itself must scan generated evidence artifacts? #86

Closed

Description

@SoundMatt

Filed while implementing java-FuSa spec v1.13.0/v1.14.0 conformance (SoundMatt/java-FuSa#31).

The ambiguity

§1.6.1 says a Rule A finding (FUSA-STUB001) "participates in check's exit code (§4.1) like any other finding," and Rule B (FUSA-STUB002) is described the same way modulo severity/gating defaults. That phrasing reads naturally for a tool where hara/fmea/tara/safety-case/sas content is itself produced by (or folded into) check's own analysis pass. But per the §9 command catalog, these are separate top-level commands (hara, fmea, tara, safety-case, sas), each writing its own artifact file — check is a distinct command that (in every tool I've looked at, including java-FuSa) analyzes source/config, not previously-generated evidence JSON.

That leaves two materially different implementations, both defensible from the current spec text:

  1. Each artifact command gates its own exit code. fmea/hara/tara/safety-case/sas each run the §1.6.1 scan over their own qualitative fields as they generate the artifact, and exit non-zero themselves (via the same Finding/fingerprint/disposition mechanism §4 already defines) when an unsuppressed Rule A (or, with --strict, Rule B) finding exists. check itself is untouched — it never looks at fmea.json/.fusa-hara.json/etc.
  2. check itself ingests the generated artifacts. check --format json additionally reads fmea.json, .fusa-hara.json, tara.json, safety-case.json, sas.json (whichever are present on disk) and folds §1.6.1 findings into its own findings[]/exit code — i.e. a stub-content defect surfaces in check's output even if nobody re-ran fmea/hara/etc. since the stub content was introduced.

These aren't equivalent: under (1), a stale fmea.json with placeholder text sits silently until someone re-runs fmea; under (2) it's caught by the check gate that most CI pipelines already run every commit. Conversely (2) requires check to grow a new "ingest sibling evidence artifacts" responsibility that today it doesn't have in any tool I've checked, and raises its own questions (does check regenerate-and-scan, or only scan whatever's already on disk and risk being stale itself?).

What java-FuSa did

Went with (1) — each artifact command gates its own exit code independently, using the existing Finding.builder()/fingerprint/.fusa-dispositions.json machinery so it composes with disposition exactly like any other finding, just scoped to that command's own process rather than routed through check. Flagging here rather than silently committing the family to interpretation (1) as the presumed-canonical one, since (2) seems at least as plausible a reading and the two are not interoperable from FuSaOps' point of view (an aggregate that expects (2)'s behavior would see nothing wrong with a project whose fmea.json has drifted stale under (1)).

Ask

Could the spec clarify explicitly which behavior is intended (or document both as acceptable, if that's the actual intent)? Referencing this from SoundMatt/java-FuSa#31 and x-FuSa spec §1.6.1.

Activity

  1. SoundMatt commented on Jul 28, 2026

    @SoundMatt
    OwnerAuthor

    Fixed in #88, released as spec v1.14.1 (FuSaOps v1.142.0).

    New §1.6.1 text makes it normative: detection runs inside each artifact-producing command (hara/fmea/tara/safety-case/sas) over content that command itself just built or loaded, gating that command's own exit code — not inside check. "Participates in the exit-code gate like any other finding" means the finding uses the exact §4 Finding shape and §4.2 fingerprint/§4.1 disposition-matching mechanism check also uses, not that check itself emits it.

    This confirms java-FuSa's interpretation (1) was correct — you're not alone in it, FuSaOps' own cmd_fmea.go/cmd_hara.go/etc. do exactly this.

    Staleness (your concern about a fmea.json sitting untouched with undetected stub content) is handled separately by §1.6 rule 5 (Freshness): check/CI MAY read a sibling artifact's generatedAt and flag it stale without re-running that artifact's own §1.6.1 scan.

    Thanks for filing this rather than guessing — it was a real, load-bearing ambiguity in the text.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions