You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(index): a draining backlog no longer fails the nightly index (M1 card) - #1662
Fix false index stall alarms when incoming transcripts keep the historical backlog minimum out of reach. Fully measured passes compare their own before/after inventory; deferred and unknown inventories preserve the count. Legacy sidecars and callers retain compatibility.
Validation: 63 focused pytest tests passed (index convergence, calendar deferrals, calendar custody); ruff and diff checks passed. Two new tests failed before the fix, including the four-pass M1 reproduction ending at stalled_passes=3. The genuinely non-draining test alarms on pass 3. Updated the existing growing-input CLI test to require successful progress without an alarm.
Implementer: Codex. No reviewer requested. Local commit only; no push or live deployment.
Round 2: held-minimum growing backlog emits INDEX_BACKLOG_GROWING and persists growing_passes without failing the job. 63 focused tests passed; ruff and diff checks passed. Local head df9e69f; not pushed.
Implementer: Codex (gpt-6.1-sol). Reviewer: Claude lead, 2 rounds (r1 defect: the old non-convergence signal was dropped; r2 keeps it as a non-failing INDEX_BACKLOG_GROWING warning). Etan saw 'BrainLayer index exited 1' on the M1 (2026-11-05): four draining passes were counted as stalled against the all-time minimum.
Fix nightly index failure when backlog is draining slowly
Changes _CalendarIndexCheckpoint.finish to detect per-pass stalls by comparing each pass's backlog to that pass's own starting backlog_before, instead of the best backlog seen so far. The best-backlog comparison remains a fallback when no starting measurement exists.
Adds a persisted growing_passes counter tracked against the best measured backlog, reset when the backlog reaches zero or improves. Prints INDEX_BACKLOG_GROWING when the threshold is reached, but this warning does not trigger an alarm.
Behavioral Change: passes whose backlog shrinks but stays above zero no longer count as stalled in init.py
We reviewed changes in ebd98da...8fffe93 on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.
Some issues found as part of this review are outside of the diff in this pull request and aren't shown in the inline review comments due to GitHub's API limitations. You can see those issues on the DeepSource dashboard.
AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.
The reason will be displayed to describe this comment to others. Learn more.
Cell variable after defined in loop
A variable used in a closure is defined in a loop.
This will result in all closures using the same value for the closed-over variable, which can pave way for a hideous bug.
Every Value below was measured by this run. A row this machine cannot measure says n/a — <reason> instead of a number; baselines in Notes name their own machine, method and date and were not measured here.
Hosted success gives no verdict on mapped bytes, search p50/p95, idle CPU, hotlane backlog CPU, or fallback replay debt.
Row
Status
Value (measured by this run)
Method
Notes
commit provenance
🟢 GREEN
measured 8fffe939f992 == PR head · checkout 8fffe939f992
commit graph + live PR head · in-process · runner
Which commit this whole table is about. On a pull_request event the checkout is GitHub's synthetic merge ref, whose sha is not on the PR — #759's table printed 13fa724278bf while that PR's head was 4632f979 — so this row names the PR-head parent instead, the sha a reviewer can actually see. The comparison sha is read live from repos/{owner}/{repo}/pulls/{n} when the table is collected, not taken from the event payload, because the payload cannot know the run has been overtaken. Residual window, stated rather than papered over: a push landing between that read and the comment being posted is not caught here — the run for that push refreshes the table.
baseline attestation
🟢 GREEN
baseline f421d1a7c5e6 matches the main attestation (run 42134449957 · main ebd98da4243a · 2026-11-06T00:07:31Z)
main attestation artifact via Actions API · in-process · runner
What every comparison is measured AGAINST, and who says so. The baseline fields of tests/fixtures/sprint_gate/corpus.json (queries, latency_baseline_ms, thresholds) are compared to the ratchet-attestation artifact of the latest successful push or (no-input) workflow_dispatch run of ratchet-attest.yml on main, fetched through the Actions API — a PR run cannot write to another run's artifacts. A field that differs is RED unless that main run measured the new value. The calibrated socket collector can license p50/p95; every absent measured path stays locked, so missing collection never passes as permission for a hand edit. Boundary: the comparator is this PR's checkout of ci_ratchet_table.py, diff-reviewable, not tamper-proof.
provenance
🟢 GREEN
stamped 8fffe939f992 == HEAD, tree clean
wheel stamp · in-process · runner
Sha half of #749 keg-mode provenance: a keg built from this wheel can answer __build_sha__. The helper-age and served-process predicates need a running BrainBar and are measured only by scripts/sprint_gate.py on an installed Mac. The sha here is the checkout's — the explicit PR head in ratchet.yml — because that is what publish.yml stamps at release time; the PR-head sha this table describes is the one in commit provenance above.
fallback replay debt
⚪ n/a
n/a — no fallback queue on this machine: the pending memories live in ~/Gits/*/docs.local/decisions, and docs.local/ is gitignored, so a runner checkout has no copy of them to count
docs.local walk · machine with the fallback queue
local-only; not enforced by hosted CI. intended_brain_store: true with no chunk_id means a memory reached disk and never reached the DB, so it answers no brain_search. Budget: 0. Any pending or unparseable file is a finding, never a band -- 122 of these sat from 2026-06-28 to 2026-09-05 because nothing counted them where a reader would look. Measured by walking the tree, so it is only ever measured on a machine that HAS the tree.
pre-push: delete-only push runs no suite
🟢 GREEN
0 suite starts · exit 0
real checkout hook · delete-only stdin · sentinel suite runner
Bug SHA: 44ecbf5ba6f1ada3b965797559eee90c94b45595 (RED replay, 2026-10-06). Fix: #1084; measured at 8fffe939f9923ce752124bef8cb3ef3ab862fd35. Ceiling: 0 suite starts; hook must exit 0 and announce the delete-only skip.
mapped bytes
⚪ n/a
n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would
socket · installed Mac
local-only; not enforced by hosted CI. Baseline 26.2 GB — installed Mac, socket, 2026-09-03, after R2 drained 15,070 → 0. Up from 16.8 GB because the drain left more vectors mapped under the same cap: the change is the drain, not a leak. Not measured by this run.
search p50/p95
⚪ n/a
n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would
socket · installed Mac
local-only; not enforced by hosted CI. Margin p50: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Margin p95: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Calibrated on MacBook-Pro.local at 2026-09-01T08:42:22Z under active_sprint_load (tests/fixtures/sprint_gate/corpus.json). Not measured by this run.
idle CPU
⚪ n/a
n/a — no BrainBar daemon at /tmp/brainbar.sock: this row needs the daemon, its hybrid helper and the indexed corpus running together, and no GitHub-hosted runner has them (macOS included) — only a self-hosted Darwin/arm64 runner on an installed Mac would
ps sampling · installed Mac
local-only; not enforced by hosted CI. Ceiling: average CPU < 30% over a 60 s window (resource_budget in scripts/sprint_gate.py), ratified and kept as a hard budget. Margin daemon: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Margin helper: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Margin watcher: margin unmeasured — 0 of the 5 attested green main runs it needs; no verdict is rendered from fewer. Needs the BrainBar daemon, helper and watcher actually running. Not measured by this run.
hotlane backlog CPU
⚪ n/a
n/a — live hotlane CPU evidence requires an installed Mac
CPU/wall for backlog-encoding cycles · rolling 600 s · installed Mac
local-only; not enforced by hosted CI. Default budget 60% of one core; BRAINLAYER_HOTLANE_CPU_BUDGET_PERCENT overrides. At least 30s of active backlog cycles required; yielded time excluded. Measures installed code.
signature_valid
⚪ n/a
n/a — the macOS signature-parity job is trigger-gated and did not run on this PR: it touches no release or signing path (pyproject.toml, scripts/release-*, scripts/brainlayer-version-check.sh, publish.yml, ratchet.yml) and carries no ratchet:signatures label — a GitHub macOS runner bills at ~10× Linux minutes and rebuilds the keg venv from source
codesign · installed keg
scripts/release-verify-signatures.sh <keg> codesign-verifies every *.so/*.dylib under libexec/venv. The macOS parity job installs the published tap formula (etanhey/layers/brainlayer), so this row measures the release path — formula, published sdist and Homebrew's relocation — and not this PR's tree. Release-time baseline for the same keg on a different machine: 442 valid / 0 invalid — installed Mac (M4 Max), brew --prefix brainlayer 1.5.11, 2026-09-03.
BrainBar renders no enrichment status
🟢 GREEN
19 actual-view captures; active pending-store/replay retained
macOS DEBUG binary · AppKit actual views · Vision OCR + icon RGBA comparison · private synthetic state
Missing render capability/evidence is RED. Historical data is retained; this is source-build proof, not installed-app proof.
Watcher heartbeat freshness and recovery
⚪ n/a
n/a — macOS native probe trigger-gated: no affected path or ratchet:heartbeat label
private Python producer -> compiled Swift consumer
Preserves genuine silence; independent agent stores cannot revive the watcher. Not a load or installed-UI measurement.
maintenance quiesce: services stay down under a reviving watchdog
⚪ n/a
n/a — macOS replay trigger-gated: no affected path or ratchet:quiesce label
real launchd · GitHub macOS runner · real maintenance + fleet-watchdog
Fix: #1076 (bc6d690f4331387485678d0247e19e14e6ef8d97). Ceiling: 0 revivals; supervisor held, both resumed, no hold marker.
Synthetic private fixtures; SDK absence gated on merged R10b. Exact-head candidate proof, not deployment.
🟢 GREEN measured, within budget · 🔴 RED measured, out of budget — a finding to clear before merge · ⚪ n/a not measurable on this machine, never guessed.
No RED rows.
Measured on Linux/x86_64 · measured 8fffe939f992 · PR head 8fffe939f992 · checkout 8fffe939f992 · run · updated 2026-11-06 00:26:41 UTC
The reason will be displayed to describe this comment to others. Learn more.
Cell variable after defined in loop
A variable used in a closure is defined in a loop.
This will result in all closures using the same value for the closed-over variable, which can pave way for a hideous bug.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![deepsource-io[bot]](https://avatars.githubusercontent.com/in/16372?s=60&v=4){kind=small}
Fix false index stall alarms when incoming transcripts keep the historical backlog minimum out of reach. Fully measured passes compare their own before/after inventory; deferred and unknown inventories preserve the count. Legacy sidecars and callers retain compatibility.
Validation: 63 focused pytest tests passed (index convergence, calendar deferrals, calendar custody); ruff and diff checks passed. Two new tests failed before the fix, including the four-pass M1 reproduction ending at stalled_passes=3. The genuinely non-draining test alarms on pass 3. Updated the existing growing-input CLI test to require successful progress without an alarm.
Implementer: Codex. No reviewer requested. Local commit only; no push or live deployment.
Round 2: held-minimum growing backlog emits INDEX_BACKLOG_GROWING and persists growing_passes without failing the job. 63 focused tests passed; ruff and diff checks passed. Local head df9e69f; not pushed.
Implementer: Codex (gpt-6.1-sol). Reviewer: Claude lead, 2 rounds (r1 defect: the old non-convergence signal was dropped; r2 keeps it as a non-failing INDEX_BACKLOG_GROWING warning). Etan saw 'BrainLayer index exited 1' on the M1 (2026-11-05): four draining passes were counted as stalled against the all-time minimum.
🤖 Generated with Claude Code
Note
Fix nightly index failure when backlog is draining slowly
_CalendarIndexCheckpoint.finishto detect per-pass stalls by comparing each pass's backlog to that pass's own startingbacklog_before, instead of the best backlog seen so far. The best-backlog comparison remains a fallback when no starting measurement exists.growing_passescounter tracked against the best measured backlog, reset when the backlog reaches zero or improves. PrintsINDEX_BACKLOG_GROWINGwhen the threshold is reached, but this warning does not trigger an alarm.Macroscope summarized 0412a16.