Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -533,12 +533,36 @@ See [`../workspace/policies/ci-test-diagnostics.md`](../workspace/policies/ci-te
See [`../workspace/policies/pit-mutation-testing.md`](../workspace/policies/pit-mutation-testing.md).
Run PIT with the lifecycle prefix. Reactor-wide (what CI does):
`mvn test-compile org.pitest:pitest-maven:mutationCoverage`; or scoped to one module with
`-f srcmorph/pom.xml`. All three modules gate at `mutationThreshold` 100 — `srcmorph` (807 mutations),
`-f srcmorph/pom.xml`. All three modules gate at `mutationThreshold` 100 — `srcmorph` (830 mutations),
`srcmorph-maven-plugin` (62, the five mojo classes) and `srcmorph-cli` (16). The CLI's
`Main.main(String[])` is the one documented exclusion: it is the process entry point, and the
`smoke-fatjar` release-gating job already runs the real `java -jar` artifact and asserts
`Main#run end.` in its output, which is an end-to-end check a unit mutant cannot reach.

**`LlamaCppJniAiGenerationProvider` is split so the gate can reach it, and the split is load-bearing.**
Everything the provider does that does not touch the native handle lives in
`provider.LlamaCppJniProviderSupport`, which is on the gate; the provider itself is not. That is not
tidiness. PIT re-runs every test covering a mutated line, and while the pure logic sat next to
`model()` the only way to gate any of it was to gate `model()` too -- a ~100-line `ModelParameters`
chain whose sole exerciser is `LlamaCppJniKnobSweepTest`, 36 cases that each load a GGUF (22.9 s).
Excluding those tests instead does not work: `model()`'s mutants would then have no coverage, and a
`NO_COVERAGE` mutant fails a threshold-100 gate exactly like a survivor. **Keep the boundary at
"touches the native handle"** -- moving a pure method back into the provider silently drops it off
the gate, and moving an impure one into the support class reds it.

The extension paid for itself immediately: the class contributed 23 mutations and **five survived**,
all real gaps (the two `known*Values` separator boundaries, both `seed` guards, and the
`drySequenceBreakers` branch). None could have been seen before, because a class that generates no
mutants cannot move the number -- which is exactly why the gate read a stable 775/775 straight
through the 1.1.0-era `dry_penalty_last_n` regression and through its fix.

Cost, measured on one machine rather than estimated: 11:34 min at 807 mutations before, **12:14 min
at 830 after** -- about 40 s for 23 mutations. `excludedTestClasses` was considered and is **not**
used: it would remove a test class from mutant matching for *every* target class, not just this one,
and the measurement says it is not needed. (The intermediate 15:57 min reading was inflated by the
five survivors: PIT exhausts every covering test for a mutant that never dies, while a killed one
stops at the first failure.)

**Two classes are permanently off the gate, and this is worth not re-litigating.** Both have
survivors that are *equivalent mutants*, unkillable through the public API rather than merely
untested. `document.AiMdHeaderCodec`: the colon guard in `read` is reached only after
Expand Down
14 changes: 0 additions & 14 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,6 @@ everything below is genuinely still open.

## Open

- **Put `provider.LlamaCppJniAiGenerationProvider` on the PIT gate.** It is the one production class
where a defect has actually reached users, and it is *not* on the `targetClasses` list — which is
why the gate stayed at 775/775 through the 1.1.0-era `dry_penalty_last_n` regression and through
its fix: the class generates no mutants, so neither the bug nor the tests that now cover it move
the number. Do not read that stability as reassurance; PIT structurally could not have caught this.
It is now worth adding, because the class finally has model-free coverage of the parts that matter
(`buildInferenceParameters`, `warnOnTruncatedAnswer`, `logPromptCacheReuse`, `lazyMode`,
`cacheType`). **Measure before committing to it**: the real-model tests take ~4-16 s each, and PIT
re-runs every test covering a mutated line, so mutants in `model()` — the long `ModelParameters`
chain — could make the run far slower than the current few minutes. If it does, the answer is
probably `excludedTestClasses` for the real-model tests plus mutants restricted to the pure paths,
not dropping the idea. Deliberately out of scope for 1.2.0: it is a build-time question, not a
correctness one.

- **The sixteen GPU classifier fat jars are verified structurally, never launched.** Since 1.2.0
`.github/verify-classifier-fatjars.sh` asserts each is the artifact its name claims (one jar per
classifier, a native for the promised OS/arch, a native set that differs from the default jar's, so
Expand Down
1 change: 1 addition & 0 deletions srcmorph/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -720,6 +720,7 @@ SPDX-License-Identifier: Apache-2.0
<param>net.ladenthin.srcmorph.config.SrcMorphConfiguration</param>
<param>net.ladenthin.srcmorph.provider.LlamaCppJniConfig</param>
<param>net.ladenthin.srcmorph.provider.LlamaCppJniConfigFactory</param>
<param>net.ladenthin.srcmorph.provider.LlamaCppJniProviderSupport</param>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✓ Correctly added to targetClasses for PIT mutation testing at 100% threshold. The new class holds the guardcode that prevented the 1.1.0/1.1.1 regression from being caught, so its presence on the gate directly closes the coverage gap that existed before.

<param>net.ladenthin.srcmorph.engine.EngineSupport</param>
<param>net.ladenthin.srcmorph.engine.GenerateResult</param>
<param>net.ladenthin.srcmorph.engine.CalibrationReport</param>
Expand Down
Loading
Loading