Repository navigation
[perf] Linux: every workspace switch and sidebar hover re-crawls the whole worktree (bursts to 377k inotify watches) #7877
Description
Where did this happen?
Other (Linux desktop app, AppImage)
Version
1.30.2 Measured against a build of main at 2370f60 (includes #7860 and #7865).
What happened?
On a Linux monorepo, switching between workspaces and hovering workspaces in the sidebar keeps the host service busy re-crawling whole worktrees. Each monorepo worktree is ~23.3k inotify watches (~5.4k directories + ~17.9k files, since the chokidar backend watches both). The host service spends a lot of its time above one core, and the watch count bursts to hundreds of thousands.
Two causes, both still present on current main:
-
Every switch disposes the watcher and the next switch re-crawls it. FsWatcherManager.subscribe() disposes a root's watcher the moment its last listener goes (packages/workspace-fs/src/watch.ts). useWorkspaceEvent("fs:events", …) re-runs its effect on workspaceId, so switching away sends fs:unwatch, and switching back attaches and crawls the whole tree again. When switches come faster than disposal, the crawls stack. We measured bursts of up to 377k watches.
-
Hovering a workspace in the sidebar attaches a recursive watch of that worktree. DashboardSidebarHoverCardOverlay calls useDiffStats(hoveredId), which subscribes to git:changed. That makes GitWatcher attach a recursive root watch for the hovered workspace, just to show the card's two diff numbers. With the watch count as the signal, hovering alone added 6–26k watches per sweep, and once 287k.
This is different from #6729 (which workspaces GitWatcher watches) and #4198 / #6004 (how often git runs, and what each call costs). It also isn't covered by #7865, which moved scans off the terminal event loop (responsiveness), or #7860 (ignore-matching cost). Neither changes how much gets watched, or how often it is re-crawled.
Steps to reproduce
- On Linux, open a large repo (ours: ~17.9k tracked files) with 3+ worktrees as Superset workspaces.
- Watch the host service's inotify count:
for fd in /proc/<host-service pid>/fdinfo/*; do grep -c '^inotify' $fd; done | paste -sd+ | bc. - Switch between two of those workspaces every ~20 s. Each switch drops the count and re-crawls about 23k watches, at 150–400% host-service CPU. Faster switching stacks the crawls.
- Hover the sidebar rows of workspaces that aren't open, holding each until its card shows. The count climbs while hovering.
Logs, screenshots, or recordings
Controlled A/B on one machine (Linux container on a Proxmox VM, RTX 2060). Each arm ran a freshly restarted app. Two builds of main at 2370f60, packaged identically: control (no changes), and fixed, which is the same commit plus the two linked PRs and two small ignored-dir listing changes. Those two had no observable effect in this run and aren't proposed yet. Both ran the same scripted 30-minute workload: 88 deep-link workspace switches every 20 s (80 warm across 3 monorepo worktrees, 8 cold), plus 6 sidebar hover sweeps over 6 workspaces. The sweeps drove the real pointer through XTest, and every card was confirmed by screenshot. Samples were taken every 5 s: per-process CPU from /proc, and inotify watches from /proc/<pid>/fdinfo.
| control | fixed | change | |
|---|---|---|---|
| Superset CPU total (core-seconds per hour) | 4472 | 2900 | −35% |
| host service incl. git | 3466 | 1878 | −46% |
| host service above one core | 43% of samples | 20% | |
| inotify watches p50 / max | 23k / 377k | 68k / 97k | |
| watches added by hovering alone | 6–26k per sweep, once 287k | 0 |
The fixed build's median watch count is higher on purpose: it keeps up to 3 recently used workspaces attached, instead of re-crawling them on every switch.
Fixes proposed in:
Activity
github-actions commentedon Sep 26, 2026
I've written up the triage for this issue below.
Triage summary
Both causes check out against the code on
main. The host service keeps one recursive root watcher per worktree path and tears it down the moment its last subscriber leaves. Two renderer patterns make subscribers come and go a lot. The first is the workspace view'sfs:eventssubscription, which is tied to the currentworkspaceId. The second is the sidebar hover card, which subscribes togit:changedfor whichever row is hovered. On Linux the chokidar backend puts an inotify watch on every file and directory, so each come-and-go crawls the whole worktree again. For a repo with about 18k files that is roughly 23k watches each time. Nothing keeps a watcher alive after its last subscriber leaves, which explains the burst pattern in the report.Affected code
packages/workspace-fs/src/watch.ts:375-390,release()removes the listener, and iflisteners.size === 0it deletes the entry fromthis.watchersand awaitsdisposeWatcherState()right away. Nothing delays or caches that. The nextsubscribe()for the same path hits the!state && !pendingbranch (watch.ts:347-369), which callscreateWatcher()and crawls the full tree again.fs:unwatch.apps/desktop/src/renderer/hooks/host-service/useWorkspaceEvent/useWorkspaceEvent.ts:89-100,168: the effect depends onworkspaceId. When the workspace changes, its cleanup callsbus.unwatchFs(oldId).packages/workspace-client/src/lib/eventBus.ts:593-605: the client refcount drops to 0 and it sendsfs:unwatchat once.packages/host-service/src/events/event-bus.ts:487-493:stopFsWatch()disposes the subscription, which leads to the release path in item 1.DashboardSidebarHoverCardOverlay.tsx:36callsuseDiffStats(hoveredId ?? ""). The call isn't gated onopen, so it fires wheneverhoveredIdis set.useDiffStats.ts:52-57subscribes togit:changed.useWorkspaceEvent.ts:150, which callsbus.watchGit(id).git-watcher.ts:307-313:watchWorkspace()callsattachFromDb().git-watcher.ts:814-817:startWorktreeWatch()callsservice.watchPath({ absolutePath: worktreePath }), which is a recursive watch of the whole worktree.unwatchWorkspace()(git-watcher.ts:320-327) callsstopWatching(), and the watcher is disposed again. A sweep over N rows therefore costs N full crawls, and quick moves can overlap them.Root cause
The watcher's lifetime is tied exactly to whether anyone is subscribed, and the subscriptions it follows are short-lived UI state (the current workspace, the hovered row).
fs:watchand GitWatcher share one watcher per path throughFsWatcherManager, so the open workspace's git and fs interest don't double the cost. A workspace that isn't open, whether it was just switched away from or is only being hovered, pays a full crawl every time.This is separate from #7860 and #7865. Neither changes how long a watcher lives in
watch.ts, or whether the hover card subscribes.Reproduction
/proc/<pid>/fdinfowhile switching between two large worktrees every ~20 s, then hover over rows for workspaces that aren't open.bun:testsketch inpackages/workspace-fs: set up anFsWatcherManagerwith a fake backend that countssubscribecalls. Callsubscribe(root, l1), release it, thensubscribe(root, l2). The test would assert the backend was subscribed twice, which shows the re-crawl. A keep-alive fix would bring that down to one.DashboardSidebarHoverCardOverlaywith a mocked event bus. ChanginghoveredIdA → B → A should producewatchGit/unwatchGitfor each change.Suggested fix direction
watch.ts:380-389, put the watcher in a small LRU (say, 3 roots) or give it a grace timer instead of disposing it. A latersubscribe()would pick it up again without a crawl. Doing this inFsWatcherManagercovers bothfs:watchand GitWatcher. The LRU cap needs to count against inotify limits, andclose()must still drain it.useDiffStats(id, { live: false })so the hover card only runs the query (cached,staleTime: Infinity) and never callswatchGit. At minimum, gate it onopen. The diff stats for the active workspace already come fromDashboardSidebarWorkspaceStatusProvider.tsx:320.createWatchercrawl sooner when a switch arrives mid-crawl. Thepending.controller.abort()path exists (watch.ts:381-384), but it only helps if the backend actually stops crawling when aborted.