Repository navigation
Merged
Conversation
Row kept its cells in a SmallVec<[Cell; 16]>: ~330 bytes of inline storage per ring slot that real rows (one cell per column) always spill out of. A plain Vec is 24 bytes. The cells are now private (Row::cells / cells_mut) so the row can track `used`, an upper bound on the written prefix: every cell past it is Cell::BLANK. Row::clear, which runs on every bottom-of-screen linefeed on a row recycled from the far end of the scrollback ring (cold in cache), then blanks only that prefix instead of all cols cells. put / erase / fill track it exactly; cells_mut conservatively marks the whole row. A debug assertion in clear checks the invariant across the whole test suite. Call sites outside the crate change mechanically (.cells -> .cells()).
Grid::new blanked every ring slot up front: 10 042 rows x 128 cols x 20-byte cells ~= 26 MB of RSS at startup before any output. Off-screen slots now start empty and get their cells when output first scrolls into them; ED 3 hands history memory back. Every path that brings a slot on screen (scroll_up, resize, reflow) sizes it, so visible rows still always hold cols cells. Grid::slot did (head + idx) % cap, a 64-bit divide on every row lookup (several per printed character); a conditional subtract covers the in-range indices, with % kept for far out-of-range ones.
…rk_all RowDamage::mark binary-searched and inserted for every printed column; printing goes left to right, so append when the column is past the last one (mark_range extends in one go). Damage::mark_all re-walked all rows on every full-screen scroll even when `all` was already set; `all` implies every row is marked, so return early (resize still marks the rows it adds).
vte decodes ground text and calls print() once per character. The scanner now tracks when vte is known to be in its ground state (after a CSI / escape sequence's final byte, an OSC's BEL, a DCS's ST) and then dispatches clean ground text itself, exactly as vte's ground_dispatch would, except that each run of printable ASCII arrives in one new Perform::print_ascii call (default: one print per byte). Anything else handed to vte clears the flag, so uncertain cases keep going through vte. The all-ASCII shortcut that left CSI parsing to vte inside 'ground' text (open_seq) is gone: every CSI / escape sequence now goes through step_seq, which is what tells the scanner where vte returns to ground. Tests: a differential proptest runs random escape-heavy input through the parser with and without the bypass (test-only no_bypass hook) and requires identical event streams, including chunk splits.
Term::print_ascii takes the parser's runs. Within a segment that fits before the right edge every byte behaves identically under print_char (width 1, no placeholder or grapheme handling, no pending wrap after the first, wide-cluster orphans only possible at the two ends), so the row is looked up once, the cells written in one loop (Row::write_ascii) and the damage marked as one span (RowDamage::mark_cols, the same representation as per-column marks). Wrap, IRM and DECAWM-off at the edge still go through print_char a byte at a time. A proptest compares the fast path against one print per character (cells, wrap flags, scrollback, cursor, damage, and the pending-wrap state via a probe print) over random text, wide chars, margins, IRM, DECAWM and alt-screen input.
vte's per-byte CSI state machine was most of the time on SGR-heavy output (42 % of the dense workload). With vte in ground, the scanner now parses a CSI of the usual shape itself (private marker, params with ; and :, up to two intermediates, final byte) and dispatches it, matching vte 0.15 byte for byte, including the empty trailing parameter it always pushes and the csi_param_digits report. Anything vte treats specially (C0 / DEL / CAN / SUB / ESC / non-ASCII inside, misplaced private markers or digits, more than 32 params or 2 intermediates, or the input ending mid-sequence) dispatches nothing and goes to vte as before. vte::Params can only be built by vte, so toastty_parser::Params is now our own type with the same API; parameters vte parses (DCS hooks, CSIs left to it) are copied into it. Tests: differential proptests (fast path vs vte, via the no_bypass hook) over CSI bodies built from every byte class vte distinguishes, plus the parameter-cap and saturation edges.
Every non-ASCII character printed went through a linear scan of kitty's 297-entry placeholder diacritic table before checking whether the cell to its left is a placeholder at all (34 % of the unicode workload), and try_extend_grapheme built the cluster string before deciding whether the character joins it (malloc + free for nearly every CJK / Greek char). Check the cheap conditions first; behaviour is unchanged.
The ground dispatcher's run scan checked one byte per iteration; a SWAR test (high bit, +1 overflow for DEL, -0x20 borrow for C0) does a word at a time. The lowest flagged byte is exact since carries and borrows only reach bytes above it; a unit test covers every byte value at every offset.
fast_csi matched every digit through the full byte-class match; the digits of one number now go through a tight inner loop (the parameter count can only change at ; : or the final byte, so one capacity check covers the number).
Dated note in docs/decisions/scrollback.md: Vec cells instead of the inline SmallVec, on-demand ring slots, written-prefix tracking. A test pins the Row slot size.
…e one read Locks in a fix that came with the ground-text bypass: base's all-ASCII shortcut stopped tracking an open escape / CSI at a DEL (seq_end stops at 0x7F), so a read starting with a byte >= 0x80 didn't abort it (WIRE-17) and vte swallowed the text; the result depended on where the read split. step_seq keeps the sequence open across the DEL. Parser and term tests check split vs one-read agreement for ESC [ 1 DEL | é X, ESC ( DEL | 0x9C a and ESC - DEL | 0x8F 0xBD, and the 3x10 repro (prints U+FFFD X, cursor at column 2).
This was referenced Oct 22, 2026
# Conflicts: # crates/toastty-term/src/link.rs # crates/toastty-term/src/reflow.rs # crates/toastty-term/src/term.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack 3/5. Based on #62. It speeds up the parser and terminal state (
toastty-term,toastty-parser) and makes scrollback memory lazy. The rendered output is unchanged.Changes
Rowcells are a privateVec<Cell>with ausedprefix (8050756).SmallVec<[Cell;16]>inline buffer, so a row shrinks from about 350 B to 56 B.usedis blank, so recycling a cold row on a linefeed blanks only what was written instead of 2.5 KB.cells()/cells_mut()), and a debug assert checks the invariant across the whole test suite.221ef63). That's about 26 MB less at startup, and ED 3 now frees history memory.Grid::slotno longer uses%.mark_allreturns early when everything is already marked (ede4dcd).f24ce69).Perform::print_asciicall. The default is oneprintper byte, so other handlers are unaffected.Term::print_asciiwrites a run one row segment at a time (ff25867): it looks the row up once and marks damage once per segment. Wrapping, insert mode (IRM) and autowrap-off at the edge still go throughprint_char.fc613e2). The result matches vte byte for byte, and anything unusual is handed to vte.toastty_parser::Paramsis now our own type with vte's API.eabb8c5). The 297-entry placeholder-table scan runs only next to a placeholder, and the grapheme check no longer builds aStringper character.fddc47b) and a tighter loop for CSI digits (996a458).docs/decisions/scrollback.md(10f668f). Ring semantics, stable line ids, reflow and eviction are unchanged.Results
Headless
parse_bench(MB/s, best of 5, base and new runs interleaved):seqwith bare LFseqwith CRLF (what a PTY delivers)End to end (
bench/run.py, Xvfb, median of 3, compared with #62):catASCII MB/s / CPU scatSGR MB/s / CPU scatUnicode MB/s / CPU sseqMB/s / CPU sseqis now limited by the PTY and reader, not parsing: headless parsing of the CRLF form is 155 MB/s.Visual evidence
The same scene rendered by #62 and by this branch, as 1280×800 window captures. It has scrollback, SGR attributes, truecolor and 256-colour, CJK, emoji with ZWJ and skin tone, combining marks, tabs, absolute positioning, and a wrapped line.
magick compare -metric AEreports 0 differing pixels.Before (#62):
After (this PR):
Review
An independent reviewer ran a differential fuzz harness: about 6.6M random escape-heavy byte streams fed into the base and new
Term, split into reads of 1–12 bytes, with random resizes. After every chunk it compared a full state dump: every cell in the primary screen, alt screen, scrollback and viewport, plus cursor, modes, title, tab stops, damage and PTY replies.7e06add.fast_csiagainst vte's CSI states, parity of theParamsAPI, theused-prefix invariant on every write path, and that empty ring slots never become visible.Risks
=0.15.0) means re-checking those transitions andfast_csi.Row::cellsis now private. Downstream code usescells()/cells_mut().Tests
Term(fast path vs per-character printing: cells, wrap flags, scrollback, cursor, damage, pending wrap; covering wide chars, margins, IRM, DECAWM off and the alt screen).Row::used, on-demand ring slots and the run scan, plus the DEL split-read regression tests.cargo test --workspacepasses on the rebased branch, and clippy adds no warnings beyond the base.