Skip to content

Skip untouched Drive mirrors and expose sync queue diagnostics - #394

Merged
jlewi merged 2 commits into
mainfrom
jlewi/drive-queue-monitoring
Sep 24, 2026
Merged

jlewi merged 2 commits into
mainfrom
jlewi/drive-queue-monitoring

Conversation

@jlewi

@jlewi jlewi commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Mirroring a Drive folder currently marks every empty checksum as pending work, including files never downloaded or edited. This can enqueue hundreds of untouched notebooks ahead of new upstream creations. Require an OPFS log reference or cached legacy model for unknown-checksum work, while retaining pending creation, failed-download retries and conflict handling. Untouched placeholders now show as not-downloaded and are excluded from the status page’s bulk Sync Required action.

Add owner-side queue diagnostics to Drive status: an hour of peak queue depth in 10-second buckets, an eligible-to-dequeue wait histogram, and current eligible/delayed/active counts, oldest eligible wait and active duration. Metrics are bounded, shared across tabs and available even when the view was closed. Repeated wake/manual requests preserve already-eligible timestamps. Diagnostics time out after ten seconds with a specific error; charts show their last update time. The added RPC advances the storage-owner protocol to 3; existing tabs may need reopening together after deployment.

Validation: runme run build test; full app suite (1,769 passing tests with two workers); focused review regressions (299 passing); browser inspection of both charts. An existing Actions lifecycle test hit its five-second timeout in the default-parallel full run; the complete rerun passed without changing timeouts. Regressions cover 1,000 untouched mirrors, bulk-sync exclusion, legitimate pending work, deduplication, eligibility/backoff timing, bounded history and two-tab RPC access. Standalone app typecheck has 125 errors on both base and branch, with no added file/error-code diagnostics.

Design and acceptance cases are in docs-dev/design/20260923_drive_queue_over_enqueuing.md and docs-dev/CUJs/drive-sync-recovery.md.

Fixes #393

Signed-off-by: Jeremy lewi <jeremy@lewi.us>
Signed-off-by: Jeremy lewi <jeremy@lewi.us>
@jlewi

jlewi commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator Author

Reviewed the source-selection predicate, local/OPFS recovery paths, queue transitions and timing, owner RPC/protocol compatibility, chart retention/polling, and the status page's bulk action. No external review threads were present.

Addressed findings:

  • P1: Untouched Drive placeholders were excluded from automatic reconciliation but still labeled pending in the status table, so “Sync Required” could enqueue the entire folder again. They now have a not-downloaded state, filter support and regression coverage excluding them from bulk sync; explicit opening still downloads them.
  • P2: Diagnostics inherited the five-minute mutation timeout, leaving stale charts visible during an unresponsive worker and eventually showing an unrelated creation warning. The read RPC now times out after ten seconds with a diagnostic-specific message, and charts show the last snapshot time.
  • Kept the file table's vertical scrolling bounded beneath the new charts.

Focused review regressions: 299 passing tests across storage, RPC, queue, Actions and status/chart UI. runme run build test also passed. Final full-suite result is recorded in the PR description.

@jlewi
jlewi enabled auto-merge (squash) September 24, 2026 03:23
@jlewi
jlewi merged commit 708698e into main Sep 24, 2026
3 checks passed
@jlewi
jlewi deleted the jlewi/drive-queue-monitoring branch September 24, 2026 03:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skip untouched Drive placeholders during reconciliation and expose queue health

1 participant