You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Follow-ups from the 2026-11 token-count update #19
This issue tracks follow-ups from the token-count update (PRs #10, #14, #15, #16 and their successors). I add an item when the code it concerns is on main. Each item names the PR that fixes it or the trigger that starts it.
Map Claude models to claude_v3, claude_v4_7, claude_v4_8 and claude_v5 in models.json, with per-model frames and tool overhead. The Claude models PR (branch feat/claude-models) does this.
Apply the claude_v5 trailing-whitespace rule only at the end of a user message and the system prompt in count(). The encoding trims every text it counts. The Claude models PR does this.
A final single space after U+302A to U+302F counts 1 too many on claude_v3, claude_v4_7 and claude_v4_8. Unicode 16 letters U+A7CC, U+A7CD and U+A7DA count about 1 off. Both gaps come from ctok's reconstructed vocabulary and are documented in the README. Trigger: a ctok upstream fix, then port it.
Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high
Activity