Skip to content

Add training-free option-letter readout - #21

Merged
weitianxin merged 2 commits into
mainfrom
feature/cygnet-readout
Oct 2, 2026
Merged

weitianxin merged 2 commits into
mainfrom
feature/cygnet-readout

Conversation

@weitianxin

Copy link
Copy Markdown
Collaborator

What changed

  • add Cygnet-compatible exact A-Z next-token readout for frozen bases and JevAny adapters
  • expose --readout letter in eval, decide, serve, the Python loader, and latency tooling
  • support an optional fixed log-linear blend with the native pointer readout
  • add prompt-control escaping, context checks, selective untied-head row loading, provenance, tests, docs, attribution, and machine-readable results
  • preserve the existing external accuracy chart, including TypeSafe Jev 1.13 at 72.7% Typed Decisions accuracy and JevAny-Qwen3.8-27B at 72.8%

Measured result

Same 231-question JevBench public diagnostic, T=1:

  • 4B native 80.09%; letter 81.39%; fixed 50/50 blend 81.82%
  • 27B native 89.61%; letter 89.61%; fixed 50/50 blend 90.04%

Transfer-v9, 1,046 clean knowable decisions:

  • 4B native 79.16%; letter 75.72%; blend 79.06%
  • 27B native 86.23%; letter 84.23%; blend 86.90%

No positive delta is statistically significant. The result supports letter readout as a paired domain ablation, not a universal replacement. On one warmed H200 diagnostic, letter-only median latency was 1.05x/1.07x faster at 4B/27B; blending nearly doubled median latency because it runs both prefills.

Cygnet's official 73.70 is documented separately as a four-axis composite over 1,624 open and sealed decisions, not accuracy.

Validation

  • 92 targeted tests passed
  • real JevAny-Qwen3.5-4B checkpoint smoke passed on H200, including hostile chat-control text
  • site validation passed: 27 pages, 1,095 links, 239 assets
  • JSON and SVG parse checks passed
  • git diff --check passed
  • independent final review found no blocker

@weitianxin
weitianxin merged commit 0dc7c7d into main Oct 2, 2026
4 checks passed
@weitianxin
weitianxin deleted the feature/cygnet-readout branch October 2, 2026 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant