fix: send JSON, not the redacted debug view, on every streaming generation - #429
Merged
Merged
Conversation
…ation
Splitting the parameter object's single renderer into toJson() (the wire form)
and toString() (a redacted debug view, deliberately not valid JSON) turned every
surviving toString() payload call site into a silent trap. Six were repointed in
LlamaModel; LlamaIterator was missed. Every streaming path -- generate(),
generateChat(), the LlamaIterable paths and the Kotlin generateFlow /
generateChatFlow -- therefore handed the native parser
InferenceParameters{keys=[cache_prompt, prompt, stream], values=redacted}
where a request body belonged.
Nothing local could see it. Every test that exercises streaming is model-gated
and self-skips without a GGUF, so a green `mvn test` with 269 skips said nothing
about it; it surfaced on the first full-matrix CI run, on all five model-backed
Java jobs at once. That is the failure mode the redacted form was designed for --
an unparseable body dying at the parser rather than a plausible-looking one
succeeding with different values -- so the design held; the call site did not.
Three model-gated tests passed params.toString() the same way and are repointed
too, and the class javadoc that still described toString as "consumed by the
native server" is corrected -- it would have sent the next reader back into the
same trap.
The guard is an ArchUnit rule: no class outside the parameters package may call a
parameter object's toString() at all, not merely at a known call site. Two notes
on its shape, both found by running it rather than reasoning about it. The
parameters package itself is scoped out because JsonParameters is package-private
with public subclasses, so javac emits a synthetic bridge toString() that appears
in bytecode and in no source file. And it matches an explicit call only: implicit
string concatenation lowers to a concat factory with no toString() call site to
see -- documented on the rule rather than left to be rediscovered. Falsified by
reintroducing the bug: the rule reports both call sites and names them.
Verified: mvn test 1755/0 failures (13 ArchUnit rules), SpotBugs 0, PIT 337/337
at 100%, spotless and javadoc:jar clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH
bernardladenthin
had a problem deploying
to
maven-central
September 10, 2026 21:39 — with
GitHub Actions
Failure
bernardladenthin
had a problem deploying
to
maven-central
September 10, 2026 21:39 — with
GitHub Actions
Failure
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Summary
mainright now. That PR split the parameter object's single renderer intotoJson()(the wire form) andtoString()(a redacted debug view, deliberately not valid JSON). Six payload call sites inLlamaModelwere repointed;LlamaIteratorwas missed — sogenerate(),generateChat(), theLlamaIterablepaths and the KotlingenerateFlow/generateChatFlowall handed the native parserInferenceParameters{keys=[…], values=redacted}where a request body belonged. Three model-gated tests passedparams.toString()the same way and are repointed too.parameterspackage may call a parameter object'stoString()at all. The stale class javadoc that still describedtoStringas "consumed by the native server" is corrected — it would have sent the next reader back into the same trap.Why no local run could see it
Every test that exercises streaming is model-gated and self-skips without a GGUF. A green
mvn testreporting 1755 run / 0 failures / 269 skipped says nothing about any of them, which is exactly why #428 shipped with all its own gates green. The same blind spot is already on record inTODO.md("Nothing asserts a floor on the number of tests actually executed") and this is a second instance of it.Two notes on the rule's shape, both found by running it
parameterspackage is scoped out, and not for convenience:JsonParametersis package-private with public subclasses, so javac emits a synthetic bridgetoString()that exists in bytecode and in no source file. A rule covering the package fails on a method nobody can edit.toString()call site to match. That is documented on the rule rather than left to be rediscovered; no such site exists today (checked).Falsified by reintroducing the bug — the rule reports both call sites and names them — then restored.
Test plan
mvn test1755 / 0 failures (13 ArchUnit rules), SpotBugs 0, PIT 337/337 (100%), spotless andjavadoc:jarcleanInferenceParametersjavadocStill open, and not this PR's: run #933 also shows
Java Tests macOS 14 arm64 (Metal)failing one step earlier, atValidate model files— it never reached the test phase, so it says nothing about this regression either way. It looks like the model-cache/validation path rather than our change; I have not diagnosed it and am not patching around it here.Related issues / PRs
Fixes a regression introduced by #428.
Checklist
CONTRIBUTING.mdandCODE_OF_CONDUCT.md🤖 Generated with Claude Code
https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH
Generated by Claude Code