Skip to content

rio-vt: intern extras by content, copy-on-write attach - #1852

Closed
raphamorim wants to merge 1 commit into
mainfrom
extras-interning
Closed

raphamorim wants to merge 1 commit into
mainfrom
extras-interning

Conversation

@raphamorim

Copy link
Copy Markdown
Owner

Workstream 0b of the mode-2027 scoping — and, like 0c/0d, it turned out to fix live bugs on main, not just prepare for clustering.

Two bugs on main

  1. Combining marks leak across hyperlink spans. Every cell written under one OSC 8 open references the same extras slot (the cursor template holds one id). The zero-width attach path mutated that slot in place — so typing a combining mark inside a hyperlink attached the mark to every cell of the span. With rioterm: shape attached combining marks with their base glyph #1850 (marks now shape), that would render visibly on every cell.
  2. The extras id space is a silent cliff. Slots cap at u16::MAX and overflow logs + drops the extras (grid/mod.rs:179). Every occurrence of every combining sequence burned a fresh slot toward it.

The change

  • Content-hash interning: ExtrasTable::alloc reuses the slot of identical content (Extras gains Hash; Hyperlink already had it). Repeated emoji/marks cost one slot total — the mode-2027 prerequisite, where extras become the common case. Sweep/free/clear purge the lookup so stale content can't resolve to a dead slot.
  • Copy-on-write attach: shared slots make in-place mutation illegal; the combining-mark path now clones, extends, and re-interns, giving the marked cell its own (marks + hyperlink) slot while its neighbors keep theirs. Fixes bug 1.
  • Span detection by hyperlink, not slot id (hints + hover walk): a marked cell inside a link legitimately holds a different slot while belonging to the same span. Hyperlink is Arc-backed; comparison is content equality, matching OSC 8 id= semantics. This was subtly wrong before interning too — bug 1's flip side, a span splitting at a marked cell.

One test contract updated: vs16_at_last_column_preserves_base_extras pinned slot-id identity across the VS16 wrap, which encoded the in-place-mutation behavior; it now asserts the content ([ZWJ, FE0F] following the base), which is what consumers read.

Cost

Interning hashes content on alloc only — allocs happen per OSC 8 open and per combining-mark attach, not per cell or per frame. The renderer's per-frame ExtrasMap snapshot is untouched (and gets smaller with dedup).

Verification

Three new tests: intern-by-content (same marks share a slot, different marks don't), the hyperlink-leak regression (mark stays on one cell, neighbor keeps the pristine link slot, both keep the hyperlink), and lookup purging across reclaim. Full suites: 432 rio-vt, 46 librio, 172 rioterm; clippy/fmt clean.

Extras slots are now interned: identical (marks, hyperlink) content
shares one slot, so the u16 id space stops being a per-occurrence
budget whose overflow silently drops extras — repeated emoji and
combining sequences cost one slot total. Shared slots make in-place
mutation illegal, which exposes and fixes a live bug: every cell
written under one OSC 8 template references the same slot, so pushing
a combining mark into it attached the mark to the whole hyperlink
span. The attach path is copy-on-write now: clone, extend, re-intern.

Hyperlink span detection (hints, hover) compares the hyperlink itself
instead of raw extras_id equality: a marked cell inside a link holds a
different slot than its neighbors while belonging to the same span —
under interning, and already before it, id equality was the wrong key.
The sweep and free paths purge the interning lookup so stale content
can't resolve to a dead slot.
@raphamorim

Copy link
Copy Markdown
Owner Author

Folded into the consolidated unicode-foundation PR (all four workstream-0 pieces in one branch for testability, commits preserved).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant