Skip to content

[perf] Linux: every workspace switch and sidebar hover re-crawls the whole worktree (bursts to 377k inotify watches) #7877

Open

Description

@pgilmore

Where did this happen?

Other (Linux desktop app, AppImage)

Version

1.30.2 Measured against a build of main at 2370f60 (includes #7860 and #7865).

What happened?

On a Linux monorepo, switching between workspaces and hovering workspaces in the sidebar keeps the host service busy re-crawling whole worktrees. Each monorepo worktree is ~23.3k inotify watches (~5.4k directories + ~17.9k files, since the chokidar backend watches both). The host service spends a lot of its time above one core, and the watch count bursts to hundreds of thousands.

Two causes, both still present on current main:

  1. Every switch disposes the watcher and the next switch re-crawls it. FsWatcherManager.subscribe() disposes a root's watcher the moment its last listener goes (packages/workspace-fs/src/watch.ts). useWorkspaceEvent("fs:events", …) re-runs its effect on workspaceId, so switching away sends fs:unwatch, and switching back attaches and crawls the whole tree again. When switches come faster than disposal, the crawls stack. We measured bursts of up to 377k watches.

  2. Hovering a workspace in the sidebar attaches a recursive watch of that worktree. DashboardSidebarHoverCardOverlay calls useDiffStats(hoveredId), which subscribes to git:changed. That makes GitWatcher attach a recursive root watch for the hovered workspace, just to show the card's two diff numbers. With the watch count as the signal, hovering alone added 6–26k watches per sweep, and once 287k.

This is different from #6729 (which workspaces GitWatcher watches) and #4198 / #6004 (how often git runs, and what each call costs). It also isn't covered by #7865, which moved scans off the terminal event loop (responsiveness), or #7860 (ignore-matching cost). Neither changes how much gets watched, or how often it is re-crawled.

Steps to reproduce

  1. On Linux, open a large repo (ours: ~17.9k tracked files) with 3+ worktrees as Superset workspaces.
  2. Watch the host service's inotify count: for fd in /proc/<host-service pid>/fdinfo/*; do grep -c '^inotify' $fd; done | paste -sd+ | bc.
  3. Switch between two of those workspaces every ~20 s. Each switch drops the count and re-crawls about 23k watches, at 150–400% host-service CPU. Faster switching stacks the crawls.
  4. Hover the sidebar rows of workspaces that aren't open, holding each until its card shows. The count climbs while hovering.

Logs, screenshots, or recordings

Controlled A/B on one machine (Linux container on a Proxmox VM, RTX 2060). Each arm ran a freshly restarted app. Two builds of main at 2370f60, packaged identically: control (no changes), and fixed, which is the same commit plus the two linked PRs and two small ignored-dir listing changes. Those two had no observable effect in this run and aren't proposed yet. Both ran the same scripted 30-minute workload: 88 deep-link workspace switches every 20 s (80 warm across 3 monorepo worktrees, 8 cold), plus 6 sidebar hover sweeps over 6 workspaces. The sweeps drove the real pointer through XTest, and every card was confirmed by screenshot. Samples were taken every 5 s: per-process CPU from /proc, and inotify watches from /proc/<pid>/fdinfo.

control fixed change
Superset CPU total (core-seconds per hour) 4472 2900 −35%
host service incl. git 3466 1878 −46%
host service above one core 43% of samples 20%
inotify watches p50 / max 23k / 377k 68k / 97k
watches added by hovering alone 6–26k per sweep, once 287k 0

The fixed build's median watch count is higher on purpose: it keeps up to 3 recently used workspaces attached, instead of re-crawling them on every switch.

Fixes proposed in:

Activity

  1. github-actions commented on Sep 26, 2026

    @github-actions
    Contributor

    I've written up the triage for this issue below.


    Triage summary

    Both causes check out against the code on main. The host service keeps one recursive root watcher per worktree path and tears it down the moment its last subscriber leaves. Two renderer patterns make subscribers come and go a lot. The first is the workspace view's fs:events subscription, which is tied to the current workspaceId. The second is the sidebar hover card, which subscribes to git:changed for whichever row is hovered. On Linux the chokidar backend puts an inotify watch on every file and directory, so each come-and-go crawls the whole worktree again. For a repo with about 18k files that is roughly 23k watches each time. Nothing keeps a watcher alive after its last subscriber leaves, which explains the burst pattern in the report.

    Affected code

    1. The watcher is disposed as soon as its last listener goes. In packages/workspace-fs/src/watch.ts:375-390, release() removes the listener, and if listeners.size === 0 it deletes the entry from this.watchers and awaits disposeWatcherState() right away. Nothing delays or caches that. The next subscribe() for the same path hits the !state && !pending branch (watch.ts:347-369), which calls createWatcher() and crawls the full tree again.
    2. Switching workspaces sends fs:unwatch.
      • apps/desktop/src/renderer/hooks/host-service/useWorkspaceEvent/useWorkspaceEvent.ts:89-100,168: the effect depends on workspaceId. When the workspace changes, its cleanup calls bus.unwatchFs(oldId).
      • packages/workspace-client/src/lib/eventBus.ts:593-605: the client refcount drops to 0 and it sends fs:unwatch at once.
      • packages/host-service/src/events/event-bus.ts:487-493: stopFsWatch() disposes the subscription, which leads to the release path in item 1.
    3. Hovering attaches a GitWatcher root watch.
      • DashboardSidebarHoverCardOverlay.tsx:36 calls useDiffStats(hoveredId ?? ""). The call isn't gated on open, so it fires whenever hoveredId is set.
      • useDiffStats.ts:52-57 subscribes to git:changed.
      • That goes through useWorkspaceEvent.ts:150, which calls bus.watchGit(id).
      • git-watcher.ts:307-313: watchWorkspace() calls attachFromDb().
      • git-watcher.ts:814-817: startWorktreeWatch() calls service.watchPath({ absolutePath: worktreePath }), which is a recursive watch of the whole worktree.
      • When the hover moves on, unwatchWorkspace() (git-watcher.ts:320-327) calls stopWatching(), and the watcher is disposed again. A sweep over N rows therefore costs N full crawls, and quick moves can overlap them.

    Root cause

    The watcher's lifetime is tied exactly to whether anyone is subscribed, and the subscriptions it follows are short-lived UI state (the current workspace, the hovered row). fs:watch and GitWatcher share one watcher per path through FsWatcherManager, so the open workspace's git and fs interest don't double the cost. A workspace that isn't open, whether it was just switched away from or is only being hovered, pays a full crawl every time.

    This is separate from #7860 and #7865. Neither changes how long a watcher lives in watch.ts, or whether the hover card subscribes.

    Reproduction

    • Manual: follow the issue's steps. Watch the inotify count via /proc/<pid>/fdinfo while switching between two large worktrees every ~20 s, then hover over rows for workspaces that aren't open.
    • bun:test sketch in packages/workspace-fs: set up an FsWatcherManager with a fake backend that counts subscribe calls. Call subscribe(root, l1), release it, then subscribe(root, l2). The test would assert the backend was subscribed twice, which shows the re-crawl. A keep-alive fix would bring that down to one.
    • Hover: render DashboardSidebarHoverCardOverlay with a mocked event bus. Changing hoveredId A → B → A should produce watchGit/unwatchGit for each change.

    Suggested fix direction

    1. Keep recent watchers alive. When the refcount reaches 0 in watch.ts:380-389, put the watcher in a small LRU (say, 3 roots) or give it a grace timer instead of disposing it. A later subscribe() would pick it up again without a crawl. Doing this in FsWatcherManager covers both fs:watch and GitWatcher. The LRU cap needs to count against inotify limits, and close() must still drain it.
    2. Stop the hover card from watching. Pass something like useDiffStats(id, { live: false }) so the hover card only runs the query (cached, staleTime: Infinity) and never calls watchGit. At minimum, gate it on open. The diff stats for the active workspace already come from DashboardSidebarWorkspaceStatusProvider.tsx:320.
    3. Consider cancelling a pending createWatcher crawl sooner when a switch arrives mid-crawl. The pending.controller.abort() path exists (watch.ts:381-384), but it only helps if the backend actually stops crawling when aborted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions