Skip to content

feat(sweep): the drag-select box drawn natively at Explorer's latency (#338) - #340

Merged
Maxaubert merged 20 commits into
mainfrom
feat/338-native-sweep-overlay
Oct 10, 2026
Merged

Maxaubert merged 20 commits into
mainfrom
feat/338-native-sweep-overlay

Conversation

@Maxaubert

@Maxaubert Maxaubert commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Closes #338.

On Windows 11 the drag-select box (the sweep) is drawn by DirectComposition on Prism's own window, from the real cursor, every compositor frame, as File Explorer draws its box. Today's DOM box is the automatic fallback everywhere else. Spec and plan: docs/superpowers/specs/2026-10-09-native-sweep-overlay-design.md and docs/superpowers/plans/2026-10-09-native-sweep-overlay.md. Measurements: research/prism/2026-10-09-native-sweep-spike.md.

What changed

  • native/sweep/: Prism's own N-API addon (C++, /MT, /W4 /WX). It runs one native thread, one D3D11 + DComp device and one topmost DComp target per window. The thread samples the cursor late in the frame, 1000 us before the next compositor tick. A commit that lands under 100 us before the tick counts as a miss; more than 14 misses in 1440 up-frames widens the lead by 250 us, up to 2000 us, after which it samples at the tick. Anchor delays: K auto 2, wheel 0, resize 2. The thread never calls into JS. selfTest() covers the box math.

  • tools/build-sweep.mjs (npm run build:sweep, part of package, CI and release):

    • runs node-gyp against the Node that Electron embeds;
    • fails if dumpbin /dependents lists anything other than KERNEL32, USER32, d3d11, dxgi, dcomp or node.exe;
    • runs the self-test in plain Node.
  • src/main/sweepOverlay.ts decides, per window, whether the box is drawn natively, and forwards the page's messages validated and in physical pixels. The fallback has no setting and switches on when:

    • the addon is missing or fails to load;
    • the compositor clock is missing (Windows 10);
    • the session is Remote Desktop;
    • the run is --e2e or uses --sweep-overlay=off;
    • a target or an HRESULT fails;
    • the thread stops moving (500 ms watchdog).

    Main also ends the box itself on blur, minimize, hide, close, a dead page and a new document. The addon loads 400 ms after the first window's first load, so nothing is added to the startup path.

  • useSweep: a MOUSE sweep sends begin / update (with its cause) / end. The anchor goes through each list's new fromList option (the inverse of toList), the clip is the scroller's exact visible rect, and the colours are the band's own computed colours. Nothing is sent per pointer move. A pen sweep keeps the DOM box. The band stays mounted and placed, and is display: none only while main has said native: true.

  • The spike's driver is removed (sweepSpike.ts, measure.mjs, spike-checks.mjs). tools/sweep-latency/addon-check.mjs is the offscreen check for the product addon.

  • Version 0.100.0. In docs/features.md (where docs: split CLAUDE.md into docs/features.md and docs/ui-direction.md #339 moved it from CLAUDE.md), the native box rule replaces "the rest of the trail is Chromium's, and was left"; CLAUDE.md's native-module line and docs/build-and-conventions.md name the addon and node-gyp.

Numbers (spike gate, injected sweeps, 1x)

Lag median (frames) p95
Native, late 1000 us 0.26 to 0.31 at every speed (300 to 4000 CSS px/s) 0.35 to 0.44
File Explorer, same motion 0.72 to 0.86
DOM box (today) 1.06
Native sampled at the tick 1.15
  • Steady SD: native 0.05 to 0.11; Explorer 0.35 to 2.0.
  • Frames shown a frame late: 0.14 to 0.47 %. The adaptive run stayed at 1000 us with 0 widenings.
  • Box gone after release: median 0 frames, max 1.
  • Origin: 0 px, normal and maximised.
  • Owner by eye, under Wind: 1500 us was "not quite there"; 1000 us "seems like it works".
  • Offscreen check, product addon: 50 of 50 quits with a box up exited 0. 0 failed joins in 5,000 start-stops, 0 idle wakeups. A second window attaches and detaches.

Gate (this PR)

  • npm run typecheck, npm run lint (0 errors), npm test: 2700 passed.

  • npm run build:sweep: imports KERNEL32, d3d11, dcomp, USER32, node.exe; self-test passed.

  • e2e, one process at a time: bigTree, treeNav, selection, marquee, marqueeQuiet, marqueeEdge, sweepLag, sweepOverlay (new) and treeVerbs all passed, and startup stayed under 800 ms (median 663). sweepOverlay was proved against a build with end taken out of hide(): it failed, then passed.

  • npm run package built Prism-Setup-x64-0.100.0.exe with resources\sweep\prism_sweep.node. The unpacked build on a scratch profile logged sweep-overlay native: true, window: 0 and exited 0. Not installed.

  • CI is green (check, bundled-indexer, gate). check built and self-tested the addon on windows-latest (build 26100, Node 22 host). binding.gyp defaults enable_thin_lto, which Node 24's common.gypi needs and a Node 22 host does not define.

Unsigned: the .node is a PE file and ships unsigned like the rest of the app. It must join the SignPath signing set when Prism is enrolled.

Hands-on list (installed build)

  • Fast sweeps, unzoomed and zoomed in Wind (both the transform and the render engine).
  • Auto-scroll down and up.
  • A wheel turn under a held button: the anchored edge stays on its row.
  • Escape mid-sweep; Alt+Tab and Win+D mid-sweep; a release outside the window.
  • A second monitor at another scale, with the window dragged across.
  • Acrylic on and off, mica, maximised and fullscreen, and a live material switch. These were not measured in the spike.
  • A light theme, a dark theme and a picked accent.
  • The tree and the Explorer; two Prism windows; quit with a box up.
  • The marks following the box on a fast sweep, zoomed in.
  • A toast or tooltip over the list mid-sweep (it is now under the box); square corners.

Owner's calls

  • Windows 10 keeps today's DOM box (there is no compositor clock).
  • The marks now trail the box by Chromium's frames. This is the inverse of today. The marks that stand at the release are still the box's.
  • Spec gate wording: criterion 1 now compares against Explorer's median, as asked. Criterion 2 was changed on the measuring agent's own initiative: "at least 1 frame below the DOM box" became "below the DOM box's median", because the DOM box measures 1.06 frames and the old bar could not be met. Please confirm.
  • CPU while a box is up: 0.95 to 1.15 % of a core in the spike, which included per-frame logging that this PR removes. This is borderline against "under 1 %".

Found, not changed

The Explorer's list is a controlled scroll. During auto-scroll it writes the previous step back a beat later, so the list steps back one step between ticks. This is not new. The DOM box and the native anchor both follow it. It is recorded in the research file.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t

Maxaubert and others added 20 commits October 9, 2026 21:52
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
Pen sweeps keep the DOM box; ClearView at the BeginDraw atlas offset; a
bounded clock wait that never spins on an occluded display; anchor
delays per scroll cause; a thread never left joinable at CRT teardown;
state resent on reload; Desktop Duplication pointer interpolated at
present time; spike runs in its own profile; dumpbin allow-list; the
owner's open calls named.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
Task 1, steps 1 to 3 of the native sweep overlay plan.

- native/sweep: the spike addon. A topmost DirectComposition target on
  the BrowserWindow's own HWND, two 1x1 premultiplied surfaces filled by
  ClearView at the BeginDraw offset, five nearest-neighbour visuals, one
  heap-allocated thread on the compositor clock (100 ms timeout, occluded
  return then a 16 ms wait), GetCursorPos + ScreenToClient, a commit only
  on change, and a late-in-the-frame sampling mode. No JS callbacks.
- src/main/sweepSpike.ts, with hooks in index.ts, the preload and
  useSweep: SPIKE ONLY, behind --sweep-spike, reverted at step 6.
- tools/sweep-latency: spike-checks.mjs (the offscreen checks: 50 parked
  launches, 100 thread start/stops each, app.quit() with a box up),
  latency.cpp (Desktop Duplication, passive by default), measure.mjs (the
  owner's run in one command) and analyze.mjs (the gate numbers).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
- Late mode: a command landing during the late-sample wait consumed the
  wake event (an end waited a tick); it is set again. The wait is no
  longer counted as work, which made the CPU gate fail falsely.
- The both run: the native box's opaque edge hid the DOM edge exactly
  when the origin was right, and its fill tinted the DOM edge out of
  tolerance. The native box is drawn with no fill and shifted 4 px
  (--sweep-spike-origin=4,4); analyze.mjs takes the shift off.
- The DOM band's edge is lime in the dom and both runs: its theme colour
  is the marked rows' own hue. Strip runs outside the list's clip are
  dropped.
- The dom run loads the addon (no attach) for the same QPC clock.
- measure.mjs: press after the beep, a slow auto-scroll, and do not move
  or maximise the window while recording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
#338)

Late mode never waited: the CREATED frame's startTime is a period behind
the tick, so the target was already past (sample-after-tick 0 in every
late run). The target now rolls forward to the first one ahead of the tick.

sweep-latency.exe --inject-plan (owner-allowed runs only): scripted steady
sweeps at about 1 kHz with press and release, every point logged to
inject.csv, aborted by the guard's file or a cursor that strays, the button
released on every way out. analyze.mjs reads that track (--pointer inject),
adds a very fast class, per-phase steady stats and the native commit's
match against the glass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
The owner, zoomed in under Wind, judged a 1000 us lead "seems like it
works" (1500 us "not quite there"). The lead is a constant with the
measured reason; a commit landing within 100 us of the tick counts as a
miss (calibrated against Desktop Duplication), and more than 1% misses
over ten seconds of sweeping widens the lead 250 us at a time up to
2000 us, then falls back to sampling at the tick. The spec's gate now
compares against File Explorer's median. The injector gained auto-scroll,
wheel and Escape segments for the step 5 runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
Task 1 is done (GO); main no longer loads the spike behind
--sweep-spike. native/sweep stays as task 4's starting point.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
Three page-to-main messages (begin, update, end) and main's state answer,
each parsed field by field: finite, bounded coordinates, a dpr of 0.25 to
8, integer colour bytes, a known cause. Only the known fields are copied.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
Main maps the page's CSS px to the window's physical client pixels: the
anchor rounds, the clip grows outward, the edge is what Chromium draws for
one CSS px, plus the client-to-target origin the spike measured (0, 0,
normal and maximised). The renderer reads the band's computed colours as
straight-alpha bytes (rgb, rgba, color(srgb), transparent).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
The spike's addon at product quality: box.h (Explorer's box in integer
pixels, an edge the clip cut off is not drawn), overlay.cc (one thread,
one device, a topmost DComp target per window, late sampling 1000 us
before the tick with the adaptive widening, K[auto] 2, K[scroll] 0,
K[resize] 2), sweep.cc (the JS surface and selfTest). /MT, /W4 /WX.

tools/build-sweep.mjs builds it for the Node Electron embeds, fails on
any import outside the system allow-list, and runs the self-test in plain
Node. tools/sweep-latency/addon-check.mjs is the offscreen check: 50 of
50 quits with a box up exited 0, no failed join in 5,000 start-stops,
zero idle wakeups, a second window attaches and detaches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
src/main/sweepOverlay.ts loads the addon after the first window has
painted, decides per window whether the box is native (off for a missing
or broken addon, no compositor clock, Remote Desktop, --e2e, the
--sweep-overlay=off flag, a failed target, a thread that stopped or
failed), tells the page on every load and change, forwards begin, update
and end validated and in physical pixels, and ends the box itself on
blur, minimize, hide, closed, a dead page and a new document. Under e2e
it records every message on globalThis.__e2eSweepOverlay instead.
electron-builder ships vendor/sweep as resources/sweep. startup e2e:
median 663 ms to first frame (bound 800).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
A mouse sweep sends main its anchor (the press point mapped back through
the list's own fromList, the inverse of toList), the scroller's visible
clip, the dpr and the band's computed colours in the move that starts it;
an update with its cause ('auto' for our own auto-scroll write, 'scroll',
'resize') when the list moves under it; and an end from every path that
hides the band. A pen sweep sends nothing. The band stays mounted and
placed and is display: none only while main has said the box is native,
so the fall back is a flag. The preload's bridge (sweepOverlay) replaces
the spike's hook; the spike's driver (sweepSpike.ts, measure.mjs,
spike-checks.mjs) is removed, addon-check.mjs replaces the latter.
marquee, marqueeQuiet, marqueeEdge and sweepLag pass (DOM box under e2e).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
sweepOverlay: in the tree and the Explorer's list, one begin per mouse
sweep anchored at the press with the scroller's clip, the page's dpr and
the band's colours; nothing per pointer move; an 'auto' update per
auto-scroll step whose anchor moved by the scroll; 'scroll' for a wheel
turn; one end for a release, Escape and a lost focus; nothing for a pen;
a fresh state on a reload; the DOM box drawn throughout. Proved against
the build with the end taken out of hide(): the release, Escape and blur
assertions failed, then passed with it back (3 of 3 runs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
The check job builds vendor/sweep with node-gyp and MSVC, fails on any
import outside the system allow-list and runs the self-test; release.yml
builds it before electron-builder copies it into resources/sweep.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
CLAUDE.md's "the rest of the trail is Chromium's, and was left" becomes
the native box's rule: the owner's words, what draws it, the late sample
and its measurements, the fall backs, the no-JS-callback rule, the e2e.
The native-module note names Prism's own addon and node-gyp. The spec
records the clip as the scroller's exact visible rect (no -2/+2: the box
draws above the header) and the calibrated miss rule (100 us, 14 in
1440).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
Main split CLAUDE.md (#339): the native box's rule moves to
docs/features.md where the trail paragraph went, the native-module note
to docs/build-and-conventions.md, and CLAUDE.md's own native-module line
names the addon. Version stays 0.100.0 (main is 0.99.2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
CI runs Node 22, whose config.gypi lacks enable_thin_lto, which Node
24's common.gypi (the target's headers) asks for: gyp failed with "name
'enable_thin_lto' is not defined". binding.gyp gives it a default, off;
a Node 24 machine keeps its own. Reproduced and fixed locally under
Node 22.23.3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
Review of #340 against the spec:
- The page hid its DOM box for a sweep main never took (native came
  true mid-sweep) and showed it again only at the next move when main
  turned native off. It now hides only for a sweep begun while native,
  and repaints the moment the state changes. The sweepOverlay e2e drives
  both through a main e2e hook and fails on the old code.
- The native edge was round(dpr); Chromium floors a 1px border
  (MEASURED: 1 px at 150 and 175 %), so it drew twice as thick there.
- A lost D3D device drew nothing on every later sweep with no failing
  HRESULT; the thread now checks GetDeviceRemovedReason at begin and
  each up frame.
- Detach drops the target state at once, so a reused window handle
  cannot be read as failed; a begin refused for a failed target turns
  the box off for the session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01FHHaWKR4M5QtW7Wecyuk4t
@Maxaubert
Maxaubert merged commit 1b4e21a into main Oct 10, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The drag-select box draws in a native overlay, at Explorer's latency

1 participant