Skip to content

host: the memory review's long user line keeps half its 3,000-character budget when the text is dense in astral characters #2553

Closed

Description

@HugoXK

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment on this issue.

Written-by: FlyBot Lead
Claimed-by: FlyBot Lead
Claim-Id: 330722c0d3c74a9cbc94646afa910af4
Branch: dev/5-memory-review-line-budget
Base: 940815a
At: 2026-11-06T00:08:20Z

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment.

Written-by: FlyBot Lead

What happens

The memory review keeps the user's long line at both ends through ends() in core/src/host/memory.js (lines 1271-1275), on a REVIEW_USER_LINE_MAX budget of 3,000 characters:

const ends = (text, max) => {
  if (fits(text, max)) return text
  const tail = text.slice(codePointStart(text, text.length - Math.floor(max / 3)))
  const head = cut(text, Math.min(Math.ceil(max * 2 / 3), max - tail.length))
  return `${head.text} …(middle cut)… ${tail}`
}

fits() counts code points (fixed in #2341), but the two cut points are still UTF-16 arithmetic:

  • text.length - Math.floor(max / 3) subtracts a character budget from a UTF-16 length;
  • max - tail.length subtracts the tail's UTF-16 length from a character budget, and the remainder is handed to cut(), which also counts units.

So on text dense in astral characters both the tail start and the head budget move the wrong way, and the line keeps half the characters the budget allows.

Measured on live main 940815ab184e0d64e96eadb66e10c54a4d1a9e6e, REVIEW_USER_LINE_MAX = 3000, counting the kept line in characters:

user line (3,000-character budget) characters kept UTF-16 units kept
'a'.repeat(3500) 3,016 3,016
'🚀'.repeat(3500) (3,500 characters, 7,006 units with the User: label) 1,503 3,006

The counts are of the rendered review line, so the User: label and the marker are included; the head under today's code is 2,000 units (1,003 characters) and the tail 1,000 units (500 characters).

A long pasted report full of emoji therefore reaches the review with about half the context, while the unit count looks right — which is why the current test does not catch it.

Not a regression from #2341: that PR changed only the guard (text.length <= max → fits(text, max)); the cut arithmetic is untouched. This is the third stop in the same family — #2339/#2341 fixed the guard here, #2546/#2548 fixed the fits() test in src/host/steward.js, and its two cut points are the subject of the steward follow-up.

Reproduction

No source change, live main:

// the user's line reaches ends() with the 'User: ' prefix, six units
const text = `User: ${'🚀'.repeat(3500)}`
const max = 3000
// today: the kept line is 1,503 characters (head 1,003 + tail 500),
// while the ASCII control keeps 3,016
console.log([...line].length) // count characters, not .length

Expected

For a user line over the 3,000-character budget the review keeps 3,000 characters: the head is the first 2,000 code points and the tail the last 1,000, both taken by counting characters, so a UTF-16 index is never subtracted from a character budget. A line at or under 3,000 characters comes back untouched, and the …(middle cut)… marker and its wording stay as they are.

Scope

  • core/src/host/memory.js: ends() takes both cut points from the code-point count — the head ends at the 2,000th code point, the tail begins at the 1,000th from the end — so a UTF-16 index is never subtracted from a character budget; the text is made well formed first, so a lone surrogate can never reach a cut. codePointStart leaves the import. fits(), REVIEW_USER_LINE_MAX, REVIEW_OTHER_LINE_MAX, reviewLine(), clip() and the marker stay as they are.
  • core/test/memory.test.js: extend the existing case at :570 ('a long user line cut for the review keeps whole characters at both ends (#1918)'), which already splits the rendered line on ' …(middle cut)… ' — do not add a new file or a new test name. It counts with .length (UTF-16), so an astral-dense sample passes at 3,000 units while keeping only 1,500 characters: add a sample of '🚀'.repeat(3500) and assert the kept line is 3,000 characters counted with [...] (head 2,000, tail 1,000), red today at 1,503; keep an ASCII sample as the control (3,016). The comment at :587-592, which explains the unit arithmetic, has to go with the fix.
  • patches/ds-bot/0001-core-fixes.patch regenerated by node scripts/ds-bot-patches.mjs generate — src/host/memory.js is in that patch (it is not in patches/ds-bot/files/), so canonical fails without it; one changelog.d/ fragment.
  • docs/subsystems/core-host.md: the line that owns memory.js states the old unit rule for ends(), so it is corrected in the same change (this is why arch-docs reports 5 changed files) rather than skipped with a trailer.
  • Not the client bundle, not core/client.js, not src/host/diagnostics.js; no plugin change.

Duplicate check

Searched core/src/host/memory.js, core/test/memory.test.js and the closed issue list for REVIEW_USER_LINE_MAX, ends(, "memory review" and "character budget": the only hits are #2339 (closed) and its PR #2341 (closed, merged), which changed the guard and left these two lines — the diff is text.length <= max → fits(text, max), nothing else. No open issue or PR covers the cut itself. Tier: T2 (core/src/host/**).

Activity

  1. HugoXK commented on Nov 6, 2026

    @HugoXK
    ContributorAuthor

    Release

    Written-by: FlyBot Lead
    Released: 330722c0d3c74a9cbc94646afa910af4
    Reason: merged and issue closed
    At: 2026-11-06T02:13:36Z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions