Skip to content

fix: send JSON, not the redacted debug view, on every streaming generation - #429

Merged
bernardladenthin merged 1 commit into
mainfrom
claude/fix-streaming-payload-tostring
Sep 11, 2026
Merged

fix: send JSON, not the redacted debug view, on every streaming generation#429
bernardladenthin merged 1 commit into
mainfrom
claude/fix-streaming-payload-tostring

Conversation

@bernardladenthin

Copy link
Copy Markdown
Owner

Summary

  • Regression from fix!: make an undeclared wire name unrepresentable, and delete the ones nothing reads #428, red on main right now. That PR split the parameter object's single renderer into toJson() (the wire form) and toString() (a redacted debug view, deliberately not valid JSON). Six payload call sites in LlamaModel were repointed; LlamaIterator was missed — so generate(), generateChat(), the LlamaIterable paths and the Kotlin generateFlow / generateChatFlow all handed the native parser InferenceParameters{keys=[…], values=redacted} where a request body belonged. Three model-gated tests passed params.toString() the same way and are repointed too.
  • Caught by CI run #933, on all five model-backed Java jobs at once (Ubuntu, macOS 15 Metal, macOS 15 no-Metal, Windows Ninja, Windows MSVC). This is the failure mode the redacted form was designed for — an unparseable body dying loudly at the parser rather than a plausible-looking one succeeding with different values — so the design held; the call site did not.
  • Guarded by an ArchUnit rule, not by fixing the one site: no class outside the parameters package may call a parameter object's toString() at all. The stale class javadoc that still described toString as "consumed by the native server" is corrected — it would have sent the next reader back into the same trap.

Why no local run could see it

Every test that exercises streaming is model-gated and self-skips without a GGUF. A green mvn test reporting 1755 run / 0 failures / 269 skipped says nothing about any of them, which is exactly why #428 shipped with all its own gates green. The same blind spot is already on record in TODO.md ("Nothing asserts a floor on the number of tests actually executed") and this is a second instance of it.

Two notes on the rule's shape, both found by running it

  • The parameters package is scoped out, and not for convenience: JsonParameters is package-private with public subclasses, so javac emits a synthetic bridge toString() that exists in bytecode and in no source file. A rule covering the package fails on a method nobody can edit.
  • It matches an explicit call only. Implicit string concatenation lowers to a concat factory and leaves no toString() call site to match. That is documented on the rule rather than left to be rediscovered; no such site exists today (checked).

Falsified by reintroducing the bug — the rule reports both call sites and names them — then restored.

Test plan

  • Affected unit / integration tests pass locally — mvn test 1755 / 0 failures (13 ArchUnit rules), SpotBugs 0, PIT 337/337 (100%), spotless and javadoc:jar clean
  • CI is green on this branch
  • Docs / CHANGELOG updated where applicable — CHANGELOG entry under Unreleased → Fixed, plus the corrected InferenceParameters javadoc

Still open, and not this PR's: run #933 also shows Java Tests macOS 14 arm64 (Metal) failing one step earlier, at Validate model files — it never reached the test phase, so it says nothing about this regression either way. It looks like the model-cache/validation path rather than our change; I have not diagnosed it and am not patching around it here.

Related issues / PRs

Fixes a regression introduced by #428.

Checklist

  • I have read CONTRIBUTING.md and CODE_OF_CONDUCT.md
  • My commits follow Conventional Commits
  • No security-sensitive changes

🤖 Generated with Claude Code

https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH


Generated by Claude Code

…ation

Splitting the parameter object's single renderer into toJson() (the wire form)
and toString() (a redacted debug view, deliberately not valid JSON) turned every
surviving toString() payload call site into a silent trap. Six were repointed in
LlamaModel; LlamaIterator was missed. Every streaming path -- generate(),
generateChat(), the LlamaIterable paths and the Kotlin generateFlow /
generateChatFlow -- therefore handed the native parser

    InferenceParameters{keys=[cache_prompt, prompt, stream], values=redacted}

where a request body belonged.

Nothing local could see it. Every test that exercises streaming is model-gated
and self-skips without a GGUF, so a green `mvn test` with 269 skips said nothing
about it; it surfaced on the first full-matrix CI run, on all five model-backed
Java jobs at once. That is the failure mode the redacted form was designed for --
an unparseable body dying at the parser rather than a plausible-looking one
succeeding with different values -- so the design held; the call site did not.

Three model-gated tests passed params.toString() the same way and are repointed
too, and the class javadoc that still described toString as "consumed by the
native server" is corrected -- it would have sent the next reader back into the
same trap.

The guard is an ArchUnit rule: no class outside the parameters package may call a
parameter object's toString() at all, not merely at a known call site. Two notes
on its shape, both found by running it rather than reasoning about it. The
parameters package itself is scoped out because JsonParameters is package-private
with public subclasses, so javac emits a synthetic bridge toString() that appears
in bytecode and in no source file. And it matches an explicit call only: implicit
string concatenation lowers to a concat factory with no toString() call site to
see -- documented on the rule rather than left to be rediscovered. Falsified by
reintroducing the bug: the rule reports both call sites and names them.

Verified: mvn test 1755/0 failures (13 ArchUnit rules), SpotBugs 0, PIT 337/337
at 100%, spotless and javadoc:jar clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

@bernardladenthin
bernardladenthin merged commit 73062a2 into main Sep 11, 2026
73 of 79 checks passed
@bernardladenthin
bernardladenthin deleted the claude/fix-streaming-payload-tostring branch September 11, 2026 06:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants