fix(mac): stop gating drag selection detection on cursor shape - #21
fix(mac): stop gating drag selection detection on cursor shape#21llong8 wants to merge 1 commit into
Conversation
Fast drags could be missed entirely: macOS does not refresh the cursor shape while the pointer is moving quickly (updates lag until the pointer rests), so a fast selection swipe may never show an I-beam at mouse-down, mid-drag or mouse-up, and the gesture was dropped. Align macOS drag detection with the Windows implementation, which never gated drags on cursor shape: any qualifying drag now attempts the side-effect-free AX read. The clipboard fallback (simulated Cmd+C) does have side effects, so it now requires cursor evidence for the gesture (an I-beam seen at mouse-down, mid-drag or mouse-up), with user-triggered getCurrentSelection() exempt as before. Double-click and shift-click keep the existing cursor gate: those are stationary gestures where the cursor has had time to update.
ffa4025 to
26fd72d
Compare
0xfullex
left a comment
There was a problem hiding this comment.
Thanks for digging into this - the symptom is real and worth fixing, and the mid-drag sampling idea is a reasonable thing to try. But I don't think the root cause is settled yet, and I'd rather not remove the drag cursor gate until it is. Below is what I found while reviewing.
On the root cause
The PR attributes the failure to macOS not refreshing the cursor shape during fast pointer movement. I'm skeptical:
- A drag gesture typically spans 200-800 ms. WindowServer composites the cursor at 60 Hz or better, and
kCGEventLeftMouseDraggedarrives at the device sampling rate (100-125 Hz for a mouse, higher for a trackpad). That leaves dozens of opportunities for the cursor to change mid-gesture. - There is an internal tension in the argument: if the target app were genuinely too busy to update its cursor, the AX read would hit the same bottleneck -
AXUIElementCopyAttributeValuealso waits on the target process's main thread. Removing the cursor gate would not recover the selection in that case either.
Two alternative explanations fit the same symptom, and I think both are more likely.
(a) The gesture starts and ends outside the text. Press down in the margin above a paragraph (arrow cursor) -> drag across the text -> release below the paragraph (arrow cursor again). Both sampling points see an arrow, so isValidCursor is false and the selection is dropped. This is a known limitation of the current approach, and it's a very common way to select a whole paragraph - starting from the margin avoids clipping inline links and doesn't require aiming at the first character.
This also explains the "pause before releasing" workaround better than speed does. Slow, deliberate selections tend to end inside the text (you aim at the last character), so mouse-up sees an I-beam. Fast selections tend to overshoot past the text, so mouse-up sees an arrow. Speed and release position correlate, but the causation is in the release position.
(b) Thread-safe function queue pressure. This one I'd particularly like your input on. ProcessMouseEvent does not run on the event tap thread - it's dispatched to the Node main thread:
hook->mouse_tsfn.NonBlockingCall(mouseEventCtx, ProcessMouseEvent); // selection_hook.mm:2045The queue is bounded at DEFAULT_MOUSE_EVENT_QUEUE_SIZE (512), and per #19 events are silently dropped once it fills. A fast drag is exactly the workload that floods it with kCGEventLeftMouseDragged events. If mouse-up gets delayed or dropped behind a backlog, selection detection never runs at all - no cursor involved.
If (b) is a real contributor, this PR makes it worse: sampling [NSCursor currentSystemCursor] on every drag event adds a cross-process round trip to WindowServer on the main thread, which slows queue drain and increases backlog.
On removing the cursor gate for drags
I'd like to keep it. The cursor check isn't a precondition for reading the selection - it's an intent classifier: is this drag a text selection, or is it drawing on a canvas, rubber-band selecting files, dragging a control, or moving a window? Removing it means every left-drag over 8 px is treated as a selection attempt.
The "AX reads are side-effect free" argument doesn't fully cover this. The module's actual output is the text-selection event, and for downstream consumers (translation popups, dictionary tools) a spurious event is the side effect. Concretely: there is no selection-change detection anywhere in this codebase - I checked both src/mac and src/windows. GetSelectedText returns whatever kAXSelectedTextRange currently holds on the focused element. So dragging a brush in a canvas app while some property-panel text field still holds a stale selection would emit that stale selection.
On the Windows comparison: the drag branch there doesn't consult the cursor, correct - but it isn't unguarded either. It requires the window under the pointer to be the same one as at mouse-down and that the window hasn't moved:
if (hwnd && hwnd == lastWindowHandler) {
GetWindowRect(hwnd, ¤tWindowRect);
if (!HasWindowMoved(currentWindowRect, lastWindowRect)) { ... }
}That's what filters title-bar drags on Windows. This PR removes the macOS gate without adding an equivalent, which would leave macOS looser than Windows rather than consistent with it.
On mid-drag sampling
I want to be upfront that I'm undecided about this part too, so I'm not proposing it as a drop-in replacement. It does work in principle - I've confirmed the cursor does update mid-drag in Chrome - but I have two concerns:
Cost. The sampling runs on the Node main thread (see above), not on the tap thread. [NSCursor currentSystemCursor] queries WindowServer and constructs an NSCursor/NSImage each call, and there's no @autoreleasepool in this file, so the temporaries accumulate until the run loop drains. The !wasIBeamDuringDrag short-circuit only kicks in after a hit - in the failing case (arrow throughout, i.e. exactly the bug being fixed) it never short-circuits and pays full cost for the entire gesture.
Sampling accuracy. Because ProcessMouseEvent is dispatched asynchronously, [NSCursor currentSystemCursor] reads the cursor at the time the main thread gets around to the event, not at the time the event occurred. Unlike pMouseEvent->pos or evFlags, which are snapshotted at event time, the cursor is global mutable state with no binding to the event being processed. Under queue pressure the two can drift apart significantly. (This affects the existing isLastMouseDownValidCursor sampling too, so it's a pre-existing issue rather than something this PR introduces.)
If we do go this route, sampling should probably be throttled - we only need to know whether an I-beam appeared at some point, so checking every N ms rather than every event would cut the cost by an order of magnitude while keeping plenty of opportunities to catch it.
Questions
- Could you describe the failing gesture more precisely - specifically, were the press and release points inside or outside the text area? If outside, that points at explanation (a).
- Which apps did you test in - Chromium-based, native AppKit, or both? Cursor updates during drag are app-dependent (
NSTrackingEnabledDuringMouseDragis off by default, and cursor rects stop drivingcursorUpdateonce an app enters its mouse-tracking loop), so this matters for how far mid-drag sampling generalizes. - Did you try keeping the gate and just adding
wasIBeamDuringDragtoisValidCursor? If that alone didn't fix it, which app did it fail in? That would tell us a lot. - Any chance you can check whether mouse-up events are actually arriving during the failing gesture? A temporary log at the top of the
kCGEventLeftMouseUpbranch would confirm or rule out explanation (b).
To be clear about where I stand: I'm not asking you to adopt a specific alternative. I'd just like to nail down the root cause before we trade away the intent classifier, since that trade is hard to walk back once downstream consumers start seeing spurious events. If it turns out the cursor genuinely is unusable for this across a meaningful set of apps, I agree it's the wrong signal - but then I'd want to replace it with a better intent signal (for example, comparing the selection range between mouse-down and mouse-up, so a drag that doesn't change the selection stays silent) rather than none.
Problem
On macOS, selecting text with a quick swipe often fails to emit
text-selection. Repro: press and drag quickly across a paragraph and release — nothing fires. Selecting the same text slowly works, and so does pausing for a moment before releasing the button.Cause
MouseEventCallbackgates drag detection on the I-beam cursor, sampled at mouse-down and mouse-up. The deeper issue (which also explains the "pause before release" workaround above): macOS doesn't refresh the cursor shape while the pointer is moving quickly — apps only update the cursor once the pointer rests. So on a fast swipe the gesture may never show an I-beam at any point, and no amount of extra sampling fixes it. Relying on cursor shape fundamentally cannot catch fast drags.Fix
Restructure the macOS gating to match what the Windows implementation already does:
src/windowshas always behaved (it only consults cursor shape for the clipboard decision, never for drag detection).kCGEventLeftMouseDragged) or mouse-up (newgesture_has_cursor_evidenceflag). User-triggeredgetCurrentSelection()is exempt, same as the existingis_triggered_by_usersemantics on Windows.Behavior notes
Testing
On macOS 15 (arm64), Electron host app: fast swipes that previously missed now emit
text-selection; slow selections, double-click and shift-click behave as before; scrollbar drags without a lingering selection stay silent.