Summary
testguard claims already distinguishes spec-derived from comment-derived claims, but nothing else in the tool treats them differently — so a claim backed by a written spec and a claim backed by a code comment produce identical green.
Observed
6 claims in <repo>/testguard.claims.json — 0 carry a @claim annotation in source
FE-PERM-WILDCARD high comment 2 faults ...
FE-PERM-REPLY-GATE high spec 1 fault ...
FE-SECRET-OTHER-USER-PASSWORD critical spec 2 faults ...
FE-INBOX-SYSTEM-MESSAGE-INERT high spec 2 faults ...
FE-INBOX-MONOTONIC medium comment 1 fault ...
FE-INBOX-UPDATE-NO-BLANK medium comment 1 fault ...
Three spec, three comment. probe reported 9 killed, 0 unproven with no reference to that split, and status did not mention it at all.
Why the distinction is load-bearing
A comment-sourced claim is materially weaker evidence than a spec-sourced one, because a comment is code that doesn't run. It was written by the same author, at the same moment, with the same misunderstanding as the implementation it sits above. If the author misunderstood the requirement, the comment records the misunderstanding faithfully and the claim inherits it.
A spec/doc/bug-sourced claim has an independent origin — it was written before the code, or by someone else, or by production reality. That independence is the entire epistemic value of the claim.
Treating both as "a claim" flattens the strongest signal the tool already collects.
Proposal
- Report the split in
status and probe, not only in claims:
proven: 9/9 faults killed across 6 claims
provenance: 3 spec · 3 comment · 0 bug ⚠ 3 claims have no source outside the code they guard
- Allow gating on it. A
--min-provenance spec flag (or a config key) lets a team say "comment-only claims don't count toward our gate" without deleting them.
- Extend the vocabulary beyond
spec / comment. Useful kinds, roughly in descending strength: incident (a real production failure), bug (a fix commit), spec, doc, review (raised by a human reviewer), comment, inferred (scaffold's default when nothing was supplied). incident and bug are the strongest possible provenance and there is currently no way to express them.
- Record provenance in the evidence file so it survives into CI artefacts and can be trended. "We converted 4 comment-claims to bug-claims this quarter" is a real maturity signal.
- Relatedly:
claims reports "0 carry a @claim annotation in source" as a flat fact with no severity and no next action — see the separate issue on annotation drift.
Acceptance
- The provenance split is visible without running
claims specifically.
- A team can gate on provenance strength.
incident / bug provenance is expressible.
Summary
testguard claimsalready distinguishes spec-derived from comment-derived claims, but nothing else in the tool treats them differently — so a claim backed by a written spec and a claim backed by a code comment produce identical green.Observed
Three
spec, threecomment.probereported9 killed, 0 unprovenwith no reference to that split, andstatusdid not mention it at all.Why the distinction is load-bearing
A
comment-sourced claim is materially weaker evidence than aspec-sourced one, because a comment is code that doesn't run. It was written by the same author, at the same moment, with the same misunderstanding as the implementation it sits above. If the author misunderstood the requirement, the comment records the misunderstanding faithfully and the claim inherits it.A
spec/doc/bug-sourced claim has an independent origin — it was written before the code, or by someone else, or by production reality. That independence is the entire epistemic value of the claim.Treating both as "a claim" flattens the strongest signal the tool already collects.
Proposal
statusandprobe, not only inclaims:--min-provenance specflag (or a config key) lets a team say "comment-only claims don't count toward our gate" without deleting them.spec/comment. Useful kinds, roughly in descending strength:incident(a real production failure),bug(a fix commit),spec,doc,review(raised by a human reviewer),comment,inferred(scaffold's default when nothing was supplied).incidentandbugare the strongest possible provenance and there is currently no way to express them.claimsreports "0 carry a@claimannotation in source" as a flat fact with no severity and no next action — see the separate issue on annotation drift.Acceptance
claimsspecifically.incident/bugprovenance is expressible.