Skip to content

perf(web): Go to file, language bar, compare counts and cold ref reads without serial round trips (QW-027/028/059/087) - #253

Merged
PastaPastaPasta merged 8 commits into
masterfrom
fix/qw-code-browse-perf
Sep 30, 2026
Merged

PastaPastaPasta merged 8 commits into
masterfrom
fix/qw-code-browse-perf

Conversation

@PastaPastaPasta

Copy link
Copy Markdown
Owner

Code-browse performance fixes from the bonsia QA wave: QW-028, QW-027, QW-087 and QW-059. The page-budget spec now holds each of these to request and time budgets.

What was slow, and why

Every slow path read one thing at a time and waited for each round trip:

  • Go to file. It listed the tree one directory read after another: about 100–120 reads in a row on dashpay/dash.
  • Language bar. It used the same walk.
  • "Count lines". It diffed four files at a time, one chunk query per blob, with no pipelining.
  • Merge-base search. It crossed one pack block per round trip.
  • Cold repo pages. The chrome continued dash's ~700-row ref timeline six pages in a row.

What changes

  • Chunk queries are gathered (queueChunkSeqs). Platform chunk reads of one pack copy asked for in the same event-loop turn share one query, up to Platform's 100 rows. Reads that run side by side now cost a few queries instead of one each.
  • Parallel tree walk. walkFiles keeps up to 16 tree reads in flight.
  • Sequential read-ahead. When a history walk moves through a pack block by block, its source requests the next 3 blocks ahead.
  • Merge-base prefetch. The merge-base search requests the parents of its queued commits ahead of time, bounded by its commit cap.
  • Go to file (QW-028), now a component shared by the repo home and every directory page:
    • First results come from the history index the file list's column has already loaded, so they need no read. The tree walk then replaces them with the exact list, which matters because an index delta can still name a deleted file.
    • fzy-style fuzzy ranking, with matched characters marked: netproc finds src/net_processing.cpp, and validation.cpp ranks src/validation.cpp first.
    • ↑ / ↓ / Enter / Esc work in the list, and t focuses the box from anywhere on the page.
  • Counting (QW-027).
    • Counting uses 24 parallel reads; showing patches uses 12.
    • A change with more edits than a diff shows is still counted, from the same xdiff pass, so the totals are complete. The last compare left 16 files out.
    • Up to 300 files are counted automatically (was 100).
  • Renames (QW-059). The rename read budget is 400 blobs, read 16 at a time (was 64 blobs, 6 at a time). That is enough to pair git's renames on v22.0.0...v23.0.0.
  • Ref timeline (QW-087). When a public repo's chrome finds its first refUpdate or protectedRefUpdate page full on a cold read, it reads the rest as 8 refNameHash key ranges at once. It uses refs.ts's keyset scan, on the refState index. A node that does not honour the ranges falls back to the serial $createdAt pages. Delta reads are unchanged.
  • Budgets. e2e/page-budget.spec.ts gains cb-1 and cb-2 on the dash mirror. They run on moutai, or anywhere with E2E_SHOWCASE_DASHPAY set.

Measured on bonsia (local next build against the live devnet, read-only; dash mirror)

before (QA) after
Go to file, first results for validation.cpp 14.4–14.7 s, 100–120 requests, src/validation.cpp 3rd 8 ms, 0 requests, src/validation.cpp 1st
Go to file, exact walk finished same walk ~1.9 s, 16–17 requests
netproc, ↓ + Enter, t no match; keys did nothing src/net_processing.{h,cpp}; opens the choice; t focuses
Language bar once in view (cold) 12.8 s 2.4 s, 19 requests (1.5 s, 4 requests after Go to file shared the walk)
Compare v22.0.0...v23.0.0 listed 19–21 s, 257 requests, 1,539 files 16 s, 74 requests, 1,530 files (git: 1,530)
…Count lines 176 s, 1,287 requests, +83833 −61969, 16 files not counted 13.7 s, 42 requests, +112310 −69084 (git diff -M: identical), none left out
Commit f5979f7 (747 files) counted 33 s, 489 requests ~11 s after listing, 92 requests in all
Cold dash home, ref timeline 6 serial continuation reads 8 ranges sent within ~0.4 s of the composite; rows at 2.9 s

The two new page-budget tests pass against that build: E2E_PORT=4397 E2E_SKIP_BUILD=1 E2E_DEVNET=bonsia E2E_SHOWCASE_DASHPAY=7A1MEu… playwright test page-budget.spec.ts -g "code browsing budgets" reports 2 passed.

Tests

  • New: fuzzy.test.ts, go-to-file.test.tsx (index first, then the walk; arrows and Enter; t; Esc), chunk-gather.test.ts, and the sequential read-ahead case in read-ahead.test.ts.
  • chrome.test.ts: key ranges in one round trip, and the fallback when a node does not honour them.
  • file-diff.test.ts: changes too large to show are counted.
  • Gates: tsc, vitest (4347 passed), eslint on the changed files.
  • code-review findings on this part were addressed (one diff pass for counting, the walk rejects on a callback error, clearing gathered queries, positions under Unicode case folding).

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 39 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: bebae58e-7874-4868-9547-a7331e14fc72

📥 Commits

Reviewing files that changed from the base of the PR and between 00d82bb and 6926b39.

📒 Files selected for processing (28)
  • forge-web/components/repo/diff-view.tsx
  • forge-web/components/repo/go-to-file.test.tsx
  • forge-web/components/repo/go-to-file.tsx
  • forge-web/components/repo/repo-home-content.tsx
  • forge-web/components/repo/tree-content.tsx
  • forge-web/e2e/page-budget.spec.ts
  • forge-web/lib/browse/read-ahead.test.ts
  • forge-web/lib/browse/reader.ts
  • forge-web/lib/repo/chrome.test.ts
  • forge-web/lib/repo/chrome.ts
  • forge-web/lib/repo/refs.ts
  • forge-web/lib/view/blame.test.ts
  • forge-web/lib/view/browse-fallback.test.ts
  • forge-web/lib/view/browse-source.test.ts
  • forge-web/lib/view/browse-source.ts
  • forge-web/lib/view/chunk-gather.test.ts
  • forge-web/lib/view/commit-log.ts
  • forge-web/lib/view/file-diff.test.ts
  • forge-web/lib/view/file-diff.ts
  • forge-web/lib/view/fuzzy.test.ts
  • forge-web/lib/view/fuzzy.ts
  • forge-web/lib/view/pull-diff.ts
  • forge-web/lib/view/renames.test.ts
  • forge-web/lib/view/renames.ts
  • forge-web/lib/view/repo-facts.ts
  • forge-web/lib/view/text-diff.ts
  • forge-web/lib/view/zip.test.ts
  • forge-web/lib/view/zip.ts
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

PastaPastaPasta and others added 2 commits September 30, 2026 10:29
…thout serial reads

QW-028: Go to file answers from the history index at once (then the tree walk, now 16 trees at a time), ranks fuzzily as GitHub's finder does, takes the arrow keys, Enter and the t hotkey, and is on every directory.
QW-027: chunk reads asked for in one turn share a query; a history walk reads the next pack blocks ahead; the merge-base search asks for its frontier's parents ahead; counting runs 24 files at a time and counts changes too large to show. dash v22.0.0...v23.0.0 counts in ~14 s and 42 requests (was 176 s, 1,287 and partial).
QW-059: the rename read budget (400 blobs, 16 at a time) reaches git's pairs on that compare: 1,530 files, +112310 -69084.
QW-087: a cold read of a long ref timeline goes out as 8 key ranges at once, not six pages in a row.
The page-budget spec holds these to request and time budgets on the dash mirror.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…parallel walk

Review: a change with more edits than a diff shows is now counted from the same xdiff pass (no second, larger search on the main thread); an exception inside the walk's tree-read callback rejects the walk instead of leaving it pending; clearing the chunk cache drops gathered queries too; Go to file keeps matched positions on the path when lowercasing would change its length.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
PastaPastaPasta and others added 6 commits September 30, 2026 11:10
The repo chrome read a long ref timeline as keyset ranges but only fell back to the $createdAt read when a range was refused, so rows a node silently left out were kept for the whole session (later reads fetch only newer rows). Group both ref-update types per ref, run the prevOid completeness check readAllRefUpdates runs, and re-read the ranged types serially when it fails. A range read that throws now falls back too instead of failing the timeline.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s in flight

queueChunkSeqs waited on setTimeout(0), which background tabs clamp to ~1 s, stalling every serial chunk round; it now yields one macrotask through a MessageChannel post (setTimeout only where there is none). Chunk query sends also share one session-wide limit of 16 in flight, so a diff counting many files over several packs plus rename detection cannot burst ~50 proof-verified queries at once. The per-origin slot helper now hands a released slot straight to the next waiter.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On a repo past the walk's 5000-file cap the partial walk replaced the complete index and results shrank. The walk is now the list only when it finished or there is no index, the 'Searched the first N files' note shows only when the truncated walk is in use, and the highlighted row resets whenever the results change so Enter always opens one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
walkFiles queued child trees and counted files in read-completion order, so a truncated walk (the language bar, Go to file) kept a timing-dependent set. Reads still run up to 16 at once, but their entries are taken from the head of the started list, as a serial walk would, so the set is the same on every visit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the blob budget at 400 and 1 MiB per blob, rename detection could download ~400 MiB. Bytes are now counted as reads land and no further candidate is read once 64 MiB is spent: the basename phase keeps the pairs it read (the two sides are read interleaved, pair by pair) and skips the matrix; a matrix whose reads ran out is skipped rather than pair with a worse source than git. Typical source trees stay far below the cap.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@PastaPastaPasta

Copy link
Copy Markdown
Owner Author

Pre-merge check: a code-review-validator pass found one blocker and four should-fix items. All five are fixed on this branch (4ba8109..6926b39), and a second review of those fixes came back clean.

  • Blocker: the key-range ref read could lose rows without noticing. completeTimelines now runs the same prevOid completeness check that readAllRefUpdates uses, grouped across both ref-update types. When it fails, it re-reads serially. A range read that throws also falls back to the serial read. Tests cover a missing middle row, a throwing range, and the healthy cross-type case.
  • Background tabs. Chunk-query gathering now yields through a MessageChannel instead of setTimeout(0), which background tabs throttle to about 1 s.
  • Go to file. The tree walk replaces the index list only when the walk finished. The "first N files" note shows only when it applies. The highlighted row resets whenever the results change.
  • Tree walk. walkFiles handles reads in the order they started, so a truncated walk returns the same files on every load.
  • Fan-out. At most 16 chunk queries are in flight at once. Rename detection has a 64 MiB read budget and alternates base and head reads.

Gates after merging current master (including #252, #257 and #258): tsc --noEmit, vitest run (4505 passed), and eslint on the changed files. CI is green, including Playwright on bonsia.


🤖 Posted autonomously by Claude on behalf of pasta.

@PastaPastaPasta
PastaPastaPasta merged commit 20a29f5 into master Sep 30, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant