Skip to content

perf 5/5: startup on worker threads (first frame 82→49 ms), binary −1 MB - #65

Merged
EDToaster merged 10 commits into
perf/renderfrom
perf/idle
Oct 22, 2026
Merged

EDToaster merged 10 commits into
perf/renderfrom
perf/idle

Conversation

@EDToaster

Copy link
Copy Markdown
Owner

Stack 5/5. Based on #64. It covers startup, idle memory and binary size. Launch → first frame drops from about 82 ms to 49 ms. The rendered output and behaviour are unchanged.

What changed

  • GPU context and font database built on worker threads (new toastty::preload).
    • Before, launch → first frame was one serial chain on the main thread: winit opens the display (20–30 ms on X11), then wgpu instance/adapter/device (~20 ms, mostly loading the Vulkan driver), then the scan and ranking of every system font (~35 ms).
    • The GPU and font work need no window, so Preload::spawn starts both at the top of main and init_impl joins them.
    • Any worker failure or panic yields None, and the old inline path runs, so only the timing changes.
  • toastty-render pieces for that.
    • GpuContext holds an instance, adapter and device without a surface.
    • Renderer::with_gpu keeps the pre-picked adapter only if it can present to the window's surface. Otherwise it selects one exactly as Renderer::new does, and new is now with_gpu(None, ..).
    • LoadedFonts / load_fonts is the GPU- and size-independent half of GlyphRasterizer.
  • Shaders compile while the font thread finishes (Renderer::prepare_pipelines, about 5 ms overlapped).
  • Buffer::new_empty skips cosmic-text ranking every installed face against a default Attrs we never shape with (about 10 ms with about 1000 system faces). The bundled font is handed to fontdb as static bytes, not a 170 KB copy.
  • malloc_trim(0) once after the first frame (glibc only, a no-op elsewhere). On the base this commit was written against, it cut idle anonymous RSS from 52 to 41 MB. On top of perf 3/5: parser + grid hot path — 2–7× parse, lazy scrollback (idle RSS 101→60 MB) #63's lazy scrollback, startup frees much less, so the saving is now within noise (table below).
  • Binary −1.0 MB.
    • wgpu's GL backend is dropped. It was linked but unreachable, because the renderer only requests Backends::PRIMARY (Vulkan/Metal/DX12), so no machine loses a working path.
    • opt-level = "s" for cold, code-heavy dependencies that never run per frame or per PTY byte: Mermaid, LaTeX maths, SVG tree building, pulldown-cmark, glTF/OBJ, petgraph. Rendering the demo Markdown document measured the same.
  • Preload's Drop joins untaken workers, so a failed startup never exits while a worker is inside driver or fontconfig setup.

Review

An independent reviewer found no correctness bugs. Things it confirmed:

  • Adapter choice is the same as before on Vulkan, DX12 and Metal.
  • The surface is still created on the main thread on every platform.
  • Every early exit (--help, --print-default-config, CLI errors) returns before the workers start.
  • Worker panics fall back to the inline path.
  • Fonts can't be preloaded for the wrong family; the reload and zoom paths still load inline.
  • Output is pixel-identical: a mixed scene, after two zoom steps, and with a missing font family.

Two small findings are fixed in the last two commits:

  • TOML parser back at opt-level 3. toml, toml_edit and winnow were at "s", but the config parse runs on the main thread before the workers start.
  • Fonts freed on renderer init failure. If renderer init fails, the preloaded font database is now freed instead of being held for the session.

Left as is:

  • malloc_trim timing. It runs inline after the first frame (about 1 ms). It could move to the first idle point.
  • Old-libX11 race. Before libX11 1.8, an ICD making Xlib calls during the worker's device creation could race winit's XOpenDisplay. No known driver does this; Mesa uses xcb.
  • One unexplained exit. The reviewer saw one in about 25 launches, with no log. I could not reproduce it: 60 back-to-back launches of this branch with logging on, none exited early. The only log line is winit's usual XSETTINGS warning under Xvfb, which has no window manager.
  • Untested: the with_gpu fallback (pre-picked adapter can't present to the surface) has no test. Wayland and macOS were not run; Xvfb only covers X11.

Numbers

bench/run.py on Xvfb, 3 interleaved reps, median, compared with #64. Binaries were copied to disk; tmpfs copies misreport RSS.

scenario metric #64 this change
binary size MB 23.6 22.6 −1.0 MB
startup launch → first frame ms 81.6 49.4 0.61×
startup idle RSS MB 60.5 60.2 1.00×
idle_blink CPU % / RSS MB 0.30 / 61.5 0.30 / 60.9 same
idle_steady CPU % / RSS MB 0.00 / 61.0 0.00 / 60.4 same
key_latency key → frame ms p50 / p99 0.63 / 1.70 0.63 / 1.61 same
scrollback_memory final RSS MB 87.6 87.0 same
throughput_ascii MB/s / FPS / peak RSS MB 97.0 / 61.1 / 87.4 101.5 / 61.3 / 86.5 same

The cumulative numbers for the whole stack against main are posted as a comment on this PR.

Visual evidence

The test scene (scrollback, SGR attributes, truecolor and 256-colour, CJK, emoji with ZWJ and skin tone, combining marks, tabs, absolute positioning, a wrapped line), captured from the #64 build and from this branch. magick compare -metric AE reports 0 differing pixels. It is also identical to the #63 capture.

Before (#64):

before (#64): the reference test scene

After (this branch), 0 differing pixels:

after: the same scene, pixel-identical

Testing

Not done

  • Full LTO. It would cut another 1.8 MB, but release builds go from about 40 s to about 2.5 min. Your call.
  • Config reload font rescan. A reload still rescans every system font on the main thread (about 40 ms hitch per save). It's out of scope here.

…e bundled font

Buffer::new shapes an empty line with Attrs::new() (sans-serif 400), and
the first shape of an Attrs makes cosmic-text rank every installed face
against it, opening each variable font for its weight axis: ~10 ms with
~1000 system faces, on the startup critical path, for Attrs we never
shape with. Use Buffer::new_empty: every buffer gets set_text before it
is shaped.

Also hand fontdb the bundled font's static bytes instead of a 170 KB
heap copy.
Launch to first frame was a serial chain on the main thread: winit opens
the display and builds the event loop (~20-30 ms on X11), creates the
window, then the renderer creates the wgpu instance, adapter and device
(~20 ms, mostly loading the Vulkan driver), and the rasterizer scans and
parses every system font and ranks the faces for the configured family
(~35 ms). The GPU and font work need no window, so start them on two
worker threads at the top of main (toastty::preload) and join them in
init_impl, by which time they are usually done.

toastty-render grows the pieces for that:
- GpuContext: instance + adapter + device without a surface.
  Renderer::with_gpu keeps its adapter only if it can present to the
  window's surface, else selects one for the surface exactly as
  Renderer::new does (new is now with_gpu(None, ..)).
- LoadedFonts / load_fonts: the size- and GPU-independent half of
  GlyphRasterizer (font db, resolved family weight, availability), with
  the primary family's face ranking warmed. Renderer::with_loaded_fonts
  and GlyphRasterizer::with_fonts consume it; with_font_ex / new load
  inline as before.

Any worker failure or panic falls back to the inline path, so only the
timing changes. Launch->first frame on Xvfb: 112 -> 65 ms (p50 of 15).
The pipelines need only the device and surface format. Renderer::
prepare_pipelines builds them (and a spare text pipeline that the next
with_loaded_fonts takes), and the binary calls it before joining the
font preload, so ~5 ms of shader compilation overlaps the wait for the
font database instead of following it.
glibc keeps freed heap pages; only the very top of the main heap is
returned automatically. Startup frees a lot (the placeholder-size grid
replaced once the cell size is known, font-scan and driver temporaries
on their threads), so an idle terminal held ~11 MB of free heap. Call
malloc_trim(0) once, after the first frame (~1 ms; a no-op off glibc).
Idle RssAnon 52.3 -> 40.7 MB.
The renderer only requests Backends::PRIMARY (Vulkan/Metal/DX12), so the
gles backend (glow, the EGL loader, wgpu-hal's GL code) was linked but
unreachable. Stripped binary 24.78 -> 24.14 MB.
mermaid-svg, ratex-*, usvg, svgtypes, pulldown-cmark, gltf(-json), tobj,
toml(_edit), winnow, petgraph: run once per block, asset or config load.
Rendering the demo Markdown document (Mermaid + maths) is unchanged
within noise (median 228-244 ms either way). Stripped binary
24.14 -> 23.70 MB.
If startup fails before init_impl takes the preloaded GPU context or
fonts, wait for the workers rather than exiting while one is still
inside driver or fontconfig setup.
It runs on the main thread before the startup workers are spawned, so
it is on the startup path the table's comment promises to leave at 3.
@EDToaster
EDToaster added this pull request to stack #66 October 22, 2026 18:08
# Conflicts:
#	crates/toastty-render/src/lib.rs
@EDToaster
EDToaster merged commit 04acc33 into main Oct 22, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant