rio-vt: intern extras by content, copy-on-write attach - #1852
Closed
raphamorim wants to merge 1 commit into
Closed
raphamorim wants to merge 1 commit into
raphamorim wants to merge 1 commit into
Conversation
Extras slots are now interned: identical (marks, hyperlink) content shares one slot, so the u16 id space stops being a per-occurrence budget whose overflow silently drops extras — repeated emoji and combining sequences cost one slot total. Shared slots make in-place mutation illegal, which exposes and fixes a live bug: every cell written under one OSC 8 template references the same slot, so pushing a combining mark into it attached the mark to the whole hyperlink span. The attach path is copy-on-write now: clone, extend, re-intern. Hyperlink span detection (hints, hover) compares the hyperlink itself instead of raw extras_id equality: a marked cell inside a link holds a different slot than its neighbors while belonging to the same span — under interning, and already before it, id equality was the wrong key. The sweep and free paths purge the interning lookup so stale content can't resolve to a dead slot.
Owner
Author
|
Folded into the consolidated unicode-foundation PR (all four workstream-0 pieces in one branch for testability, commits preserved). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Workstream 0b of the mode-2027 scoping — and, like 0c/0d, it turned out to fix live bugs on main, not just prepare for clustering.
Two bugs on main
u16::MAXand overflow logs + drops the extras (grid/mod.rs:179). Every occurrence of every combining sequence burned a fresh slot toward it.The change
ExtrasTable::allocreuses the slot of identical content (ExtrasgainsHash;Hyperlinkalready had it). Repeated emoji/marks cost one slot total — the mode-2027 prerequisite, where extras become the common case. Sweep/free/clear purge the lookup so stale content can't resolve to a dead slot.Hyperlinkis Arc-backed; comparison is content equality, matching OSC 8id=semantics. This was subtly wrong before interning too — bug 1's flip side, a span splitting at a marked cell.One test contract updated:
vs16_at_last_column_preserves_base_extraspinned slot-id identity across the VS16 wrap, which encoded the in-place-mutation behavior; it now asserts the content ([ZWJ, FE0F]following the base), which is what consumers read.Cost
Interning hashes content on alloc only — allocs happen per OSC 8 open and per combining-mark attach, not per cell or per frame. The renderer's per-frame
ExtrasMapsnapshot is untouched (and gets smaller with dedup).Verification
Three new tests: intern-by-content (same marks share a slot, different marks don't), the hyperlink-leak regression (mark stays on one cell, neighbor keeps the pristine link slot, both keep the hyperlink), and lookup purging across reclaim. Full suites: 432 rio-vt, 46 librio, 172 rioterm; clippy/fmt clean.