Repository navigation
ci: harden the corpus/matrix against non-node failures; tighten pass criteria - #15146
Conversation
…criteria Maintainability audit follow-through. Plumbing: the runner is now under vue-tsc (tsconfig include) so frontend renames break the harness at PR time instead of red-ing the weekly as a false ecosystem FAIL; registry refresh retries and falls back to the cached snapshot (with a 90% shrink guard against API shape drift); corpus fetch retries and refuses to ship a partial corpus when >5% of fetches fail; superseded ~1GB corpus caches are pruned (10GB repo budget); a Thursday cron keeps the cache inside GitHub's 7-day eviction window; matrix-verdict runs on partial shards instead of skipping; build_matrix no longer rewrites a pack's own scripts/* imports to ours, records entry truncation, and fails friendly on a missing corpus. Measurement: packs now execute under real timers (setupFiles was a mergeConfig-concatenation no-op that smuggled in the unit suite's fake timers). Criteria reworked against baselines measured over the full 1,879-pack registry: any-entry-loaded >= 95% (97.4), NEW entry-files- clean >= 91% (94.1), registerNodeDef >= 99%, worst-op >= 99%, plus a week-over-week delta gate (entry-clean must not drop >1.5pp; metrics carried between runs by the actions cache) and a harness gate - the runner self-checks that the default workflow materialized, and the verdict is withheld as HARNESS FAILURE (exit 2) when most packs fail it, so a degraded harness can never report a vacuously clean ecosystem or a false FAIL. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🎭 Playwright: ✅ 1802 passed, 0 failed · 2 flaky📊 Browser Reports
📦 Bundle Size
⚡ Performance Report
Absolute values
Raw data{
"timestamp": "2026-08-12T17:59:44.837Z",
"gitSha": "a2433fdb7cba04fef9da6392562e46f24bc71566",
"branch": "benjcooley/matrix-hardening",
"measurements": [
{
"name": "canvas-idle",
"durationMs": 2083.6530000000266,
"styleRecalcs": 9,
"styleRecalcDurationMs": 8.408000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 553.132,
"heapDeltaBytes": -18601152,
"heapUsedBytes": 45715976,
"domNodes": -282,
"jsHeapTotalBytes": 22781952,
"scriptDurationMs": 17.774,
"eventListeners": -153,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "canvas-idle",
"durationMs": 2051.8560000000434,
"styleRecalcs": 9,
"styleRecalcDurationMs": 8.610000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 539.454,
"heapDeltaBytes": 5876992,
"heapUsedBytes": 70191200,
"domNodes": 18,
"jsHeapTotalBytes": 24379392,
"scriptDurationMs": 18.825000000000003,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "canvas-mouse-sweep",
"durationMs": 2012.7650000000017,
"styleRecalcs": 75,
"styleRecalcDurationMs": 46.675000000000004,
"layouts": 12,
"layoutDurationMs": 3.7160000000000006,
"taskDurationMs": 968.4029999999999,
"heapDeltaBytes": -12841672,
"heapUsedBytes": 51663044,
"domNodes": -281,
"jsHeapTotalBytes": 24092672,
"scriptDurationMs": 138.587,
"eventListeners": -153,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "canvas-mouse-sweep",
"durationMs": 1856.5029999999751,
"styleRecalcs": 76,
"styleRecalcDurationMs": 44.892,
"layouts": 12,
"layoutDurationMs": 3.7120000000000006,
"taskDurationMs": 914.7449999999999,
"heapDeltaBytes": -150356,
"heapUsedBytes": 64220088,
"domNodes": 59,
"jsHeapTotalBytes": 25690112,
"scriptDurationMs": 130.195,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1721.1070000000177,
"styleRecalcs": 32,
"styleRecalcDurationMs": 17.561,
"layouts": 6,
"layoutDurationMs": 0.5139999999999999,
"taskDurationMs": 384.40999999999997,
"heapDeltaBytes": 8853696,
"heapUsedBytes": 73291148,
"domNodes": 77,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 18.970000000000002,
"eventListeners": 19,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1740.762000000018,
"styleRecalcs": 31,
"styleRecalcDurationMs": 18.285,
"layouts": 6,
"layoutDurationMs": 0.6839999999999999,
"taskDurationMs": 381.41600000000005,
"heapDeltaBytes": 8805928,
"heapUsedBytes": 73287952,
"domNodes": 78,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 19.455,
"eventListeners": 19,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "dom-widget-clipping",
"durationMs": 606.9229999999948,
"styleRecalcs": 11,
"styleRecalcDurationMs": 7.268,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 388.505,
"heapDeltaBytes": -11203232,
"heapUsedBytes": 53244132,
"domNodes": 18,
"jsHeapTotalBytes": 25427968,
"scriptDurationMs": 62.832,
"eventListeners": 2,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "dom-widget-clipping",
"durationMs": 601.8990000000031,
"styleRecalcs": 11,
"styleRecalcDurationMs": 8.554999999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 402.3349999999999,
"heapDeltaBytes": -11144296,
"heapUsedBytes": 53282464,
"domNodes": 18,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 61.779999999999994,
"eventListeners": 2,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-idle",
"durationMs": 2009.7840000000247,
"styleRecalcs": 9,
"styleRecalcDurationMs": 9.179000000000004,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 682.1239999999999,
"heapDeltaBytes": 7062244,
"heapUsedBytes": 66462000,
"domNodes": -281,
"jsHeapTotalBytes": 3510272,
"scriptDurationMs": 104.682,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-idle",
"durationMs": 2038.7190000000146,
"styleRecalcs": 7,
"styleRecalcDurationMs": 6.437999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 708.5319999999999,
"heapDeltaBytes": 7311888,
"heapUsedBytes": 67346676,
"domNodes": -281,
"jsHeapTotalBytes": 4296704,
"scriptDurationMs": 118.64200000000001,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-pan",
"durationMs": 2176.536999999996,
"styleRecalcs": 69,
"styleRecalcDurationMs": 14.249999999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1304.6470000000002,
"heapDeltaBytes": 4081692,
"heapUsedBytes": 64805648,
"domNodes": -284,
"jsHeapTotalBytes": 2916352,
"scriptDurationMs": 441.461,
"eventListeners": -147,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "large-graph-pan",
"durationMs": 2262.878999999998,
"styleRecalcs": 67,
"styleRecalcDurationMs": 12.544,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1379.235,
"heapDeltaBytes": 4633264,
"heapUsedBytes": 65776564,
"domNodes": -289,
"jsHeapTotalBytes": 4227072,
"scriptDurationMs": 482.26099999999997,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "large-graph-zoom",
"durationMs": 3314.5789999999806,
"styleRecalcs": 65,
"styleRecalcDurationMs": 16.045,
"layouts": 60,
"layoutDurationMs": 8.135,
"taskDurationMs": 1627.186,
"heapDeltaBytes": 5675060,
"heapUsedBytes": 67988348,
"domNodes": -287,
"jsHeapTotalBytes": 5869568,
"scriptDurationMs": 579.5310000000001,
"eventListeners": -153,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.699999999999818
},
{
"name": "large-graph-zoom",
"durationMs": 3394.386000000054,
"styleRecalcs": 65,
"styleRecalcDurationMs": 17.354,
"layouts": 60,
"layoutDurationMs": 7.804,
"taskDurationMs": 1682.4580000000003,
"heapDeltaBytes": 10231896,
"heapUsedBytes": 72356324,
"domNodes": -289,
"jsHeapTotalBytes": 5607424,
"scriptDurationMs": 597.318,
"eventListeners": -153,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.670000000000012,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "minimap-idle",
"durationMs": 2056.3250000000153,
"styleRecalcs": 8,
"styleRecalcDurationMs": 8.408999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 768.754,
"heapDeltaBytes": 7216724,
"heapUsedBytes": 67546312,
"domNodes": -283,
"jsHeapTotalBytes": 3772416,
"scriptDurationMs": 125.67,
"eventListeners": -181,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "minimap-idle",
"durationMs": 2036.9079999999258,
"styleRecalcs": 8,
"styleRecalcDurationMs": 7.795999999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 669.1010000000001,
"heapDeltaBytes": 7559368,
"heapUsedBytes": 68253520,
"domNodes": -284,
"jsHeapTotalBytes": 2723840,
"scriptDurationMs": 104.946,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 601.1570000000006,
"styleRecalcs": 46,
"styleRecalcDurationMs": 10.777,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 418.30500000000006,
"heapDeltaBytes": -10910564,
"heapUsedBytes": 53588884,
"domNodes": 18,
"jsHeapTotalBytes": 26214400,
"scriptDurationMs": 123.531,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 567.5900000001093,
"styleRecalcs": 45,
"styleRecalcDurationMs": 9.735999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 398.90299999999996,
"heapDeltaBytes": -11027732,
"heapUsedBytes": 53588968,
"domNodes": 16,
"jsHeapTotalBytes": 26738688,
"scriptDurationMs": 122.62100000000001,
"eventListeners": 6,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "subgraph-idle",
"durationMs": 2039.6939999999972,
"styleRecalcs": 8,
"styleRecalcDurationMs": 8.504,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 500.449,
"heapDeltaBytes": 5827932,
"heapUsedBytes": 70431188,
"domNodes": 16,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 15.368,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-idle",
"durationMs": 2011.7979999999989,
"styleRecalcs": 10,
"styleRecalcDurationMs": 9.619000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 519.879,
"heapDeltaBytes": -18305380,
"heapUsedBytes": 46116128,
"domNodes": -1,
"jsHeapTotalBytes": 22257664,
"scriptDurationMs": 14.022,
"eventListeners": -153,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1693.7880000000405,
"styleRecalcs": 75,
"styleRecalcDurationMs": 36.479,
"layouts": 16,
"layoutDurationMs": 4.4719999999999995,
"taskDurationMs": 769.9149999999998,
"heapDeltaBytes": -3409456,
"heapUsedBytes": 61009640,
"domNodes": 60,
"jsHeapTotalBytes": 25427968,
"scriptDurationMs": 92.984,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1713.2299999999532,
"styleRecalcs": 75,
"styleRecalcDurationMs": 41.481,
"layouts": 16,
"layoutDurationMs": 4.814000000000001,
"taskDurationMs": 794.8779999999999,
"heapDeltaBytes": -3589820,
"heapUsedBytes": 60671436,
"domNodes": 61,
"jsHeapTotalBytes": 24903680,
"scriptDurationMs": 95.001,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-transition-enter",
"durationMs": 1434.6939999999222,
"styleRecalcs": 18,
"styleRecalcDurationMs": 31.151999999999997,
"layouts": 14,
"layoutDurationMs": 13.481999999999998,
"taskDurationMs": 974.8440000000003,
"heapDeltaBytes": 31041528,
"heapUsedBytes": 99113860,
"domNodes": 13673,
"jsHeapTotalBytes": 16252928,
"scriptDurationMs": 44.510000000000005,
"eventListeners": 2375,
"totalBlockingTimeMs": 149,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "viewport-pan-sweep",
"durationMs": 8575.391000000025,
"styleRecalcs": 249,
"styleRecalcDurationMs": 38.01200000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 4920.692000000001,
"heapDeltaBytes": 17359812,
"heapUsedBytes": 76974756,
"domNodes": -284,
"jsHeapTotalBytes": 7897088,
"scriptDurationMs": 1633.764,
"eventListeners": -133,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "viewport-pan-sweep",
"durationMs": 8359.383000000093,
"styleRecalcs": 250,
"styleRecalcDurationMs": 37.914,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 4649.485000000001,
"heapDeltaBytes": 8694984,
"heapUsedBytes": 68116776,
"domNodes": -284,
"jsHeapTotalBytes": 6324224,
"scriptDurationMs": 1447.607,
"eventListeners": -163,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "vue-large-graph-idle",
"durationMs": 17911.49699999994,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 17864.126,
"heapDeltaBytes": -33103708,
"heapUsedBytes": 167619232,
"domNodes": -8312,
"jsHeapTotalBytes": -7282688,
"scriptDurationMs": 608.245,
"eventListeners": -16391,
"totalBlockingTimeMs": 0,
"frameDurationMs": 18.886666666666617,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "vue-large-graph-idle",
"durationMs": 17439.45700000006,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 17386.266999999996,
"heapDeltaBytes": -52339048,
"heapUsedBytes": 167866676,
"domNodes": -8312,
"jsHeapTotalBytes": -7278592,
"scriptDurationMs": 599.689,
"eventListeners": -16387,
"totalBlockingTimeMs": 0,
"frameDurationMs": 17.776666666666763,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-pan",
"durationMs": 21376.580999999987,
"styleRecalcs": 145,
"styleRecalcDurationMs": 18.35399999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 21313.078,
"heapDeltaBytes": -52956552,
"heapUsedBytes": 153875984,
"domNodes": -8316,
"jsHeapTotalBytes": -9641984,
"scriptDurationMs": 921.387,
"eventListeners": -16385,
"totalBlockingTimeMs": 383,
"frameDurationMs": 17.776666666666642,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-pan",
"durationMs": 21811.20599999997,
"styleRecalcs": 151,
"styleRecalcDurationMs": 19.55000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 21746.98,
"heapDeltaBytes": -48092368,
"heapUsedBytes": 176709760,
"domNodes": -8312,
"jsHeapTotalBytes": -7348224,
"scriptDurationMs": 963.0049999999999,
"eventListeners": -16381,
"totalBlockingTimeMs": 497,
"frameDurationMs": 18.333333333333332,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "workflow-execution",
"durationMs": 490.7990000000382,
"styleRecalcs": 22,
"styleRecalcDurationMs": 26.659000000000002,
"layouts": 5,
"layoutDurationMs": 1.954,
"taskDurationMs": 150.673,
"heapDeltaBytes": 5519396,
"heapUsedBytes": 69218648,
"domNodes": 154,
"jsHeapTotalBytes": 5505024,
"scriptDurationMs": 15.573,
"eventListeners": 99,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.670000000000012,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "workflow-execution",
"durationMs": 471.05099999998856,
"styleRecalcs": 12,
"styleRecalcDurationMs": 18.768,
"layouts": 2,
"layoutDurationMs": 0.4630000000000001,
"taskDurationMs": 109.489,
"heapDeltaBytes": 5047684,
"heapUsedBytes": 68636112,
"domNodes": 121,
"jsHeapTotalBytes": 4980736,
"scriptDurationMs": 10.068000000000001,
"eventListeners": 97,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
}
]
} |
benjcooley
left a comment
There was a problem hiding this comment.
Automated review — posted by an AI agent (Claude Code)
CI/tooling hardening for the weekly census: retries, cache fallbacks, a harness self-check gate, two new PASS criteria, a week-over-week delta gate, corpus pruning, real timers for the matrix suite.
Verified correct (not assumed): the setupFiles fix is real — base vite.config.mts does set ['./vitest.timer.setup.ts','./vitest.setup.ts'], so the old setupFiles: [] really was a mergeConfig concat no-op; tsconfig already excludes src/__ecs_matrix__; the wReorder/graph.add type changes are behaviour-preserving; String(i).split(':').pop() ?? '' is a no-op (split always yields ≥1 element); and the poison-corpus job's exit-1 assertion holds (7 packs, only poison-op-break trips the self-check, so 86% clears the 50% harness gate).
The theme of the findings: three of the guards this PR adds don't actually guard — each passes trivially in the case it was written to catch.
| Severity | Count |
|---|---|
| BLOCKER | 3 |
| SHOULD FIX | 5 |
| TRIVIAL | 1 |
| Total | 9 |
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## benjcooley/matrix-detection-proof #15146 +/- ##
==================================================================
Coverage 80.81% 80.81%
==================================================================
Files 1873 1873
Lines 121954 121954
Branches 38028 38028
==================================================================
Hits 98558 98558
Misses 22849 22849
Partials 547 547
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
|
Re-baseline under real timers - run 31624366070, all seven jobs green, VERDICT: PASS, no floor adjustment needed:
The fake-timer removal moved the ecosystem numbers by ~0.1pp - measurement conditions changed safely. Corpus grew to 1,883 packs / 5,630 entries between runs (registry drift, revalidation working). Two new instruments reporting for the first time:
Delta gate shows |
Review follow-ups: metrics saves carry a run_attempt suffix and the summarizer skips the delta when the restored metrics came from this same run (a re-run can no longer compare a run against itself); the corpus mass-failure gate divides by attempted fetches instead of the mostly cached target list, with a >=5 floor; the import-rewrite prefix no longer hijacks a pack's own ./scripts/ modules and again matches ./../ parent references; metrics are written only on a PASSing verdict so a failing run cannot ratchet the baseline down to its own eroded numbers; non-JSON registry responses fall back to the cached snapshot; unreadable shard manifests are skipped instead of killing the verdict; runner self-check uses floors rather than pins on today's workflow shape; the verdict job header documents the current five-criteria contract; cache pruning runs only from the default branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
benjcooley
left a comment
There was a problem hiding this comment.
APPROVED — posted by an AI agent (Claude Code)
(GitHub blocks a formal Approve on your own PR, so this is a comment review.)
95b983f9 resolves all nine. The three BLOCKERs I re-verified by execution rather than by reading:
| # | Severity | Status |
|---|---|---|
| 1 | BLOCKER | ✅ Delta self-comparison. Note the run_attempt suffix alone would not have fixed this — restore-keys: matrix-metrics- still serves attempt 1's save to attempt 2. The actual fix is the runId stamp plus the summarizer's skip branch; the unique key just stops the overwrite. Both together are correct. |
| 2 | BLOCKER | ✅ Denominator is now ok+empty+failed+drifted, excluding cached, so a 100% outage of an incremental run's attempted fetches trips. The >= 5 floor is a sensible small-sample guard. |
| 3 | BLOCKER | ✅ PREFIX — ran all six cases against the new regex: bare "scripts/app.js" and pack-local "./scripts/app.js" correctly NOT rewritten; "../", "./../", "/" and "../../" all correctly rewritten. Both the hijack and the regression I flagged are closed. |
| 4 | SHOULD FIX | ✅ Metrics written only when not breaches, and the save step gated on steps.verdict.outcome. The comment about the restored previous metrics still sitting in the directory on FAIL is the subtle part — a file-presence guard alone would have re-saved the stale baseline under a newer key. Good catch on your side. |
| 5 | SHOULD FIX | ✅ try/except (OSError, ValueError) with continue — a bad shard manifest is skipped, not fatal. |
| 6 | SHOULD FIX | ✅ Prune now requires github.ref_name == github.event.repository.default_branch. |
| 7 | SHOULD FIX | ✅ json.loads wrapped and re-raised as RegistryUnavailable, so an HTML body falls back to the cached snapshot. |
| 8 | SHOULD FIX | ✅ Floors (4) rather than pins (7), with the rationale in the comment. Catches "nothing loaded" without failing on a legitimate workflow edit. |
| 9 | TRIVIAL | ✅ Header now documents all five criteria, the harness withhold gate, and the 0/1/2 exit codes. |
The measured-baseline callout in the new header (run 31624366070, real timers) is the right way to make those floors auditable later.
1172789
into
benjcooley/matrix-detection-proof
Follow-through on the maintainability audit: the weekly must not red (or hollow out) for reasons unrelated to custom nodes. Stacked on #15145 (-> #15110 -> #15081) so this diff is only the hardening.
Plumbing (no measurement change)
tsconfigincludesscripts/registry-census/**/*.ts; 6 real errors fixed, all inside the harness file). A frontend rename ofregisterNodeDef/createReroute/graphToPromptnow breaks the renaming PR loudly instead of red-ing the next weekly as a false ecosystem FAIL. Generatedsrc/__ecs_matrix__stays excluded.--retry 5, falls back to the cached snapshot on failure, and refuses a snapshot that shrank >10% (API shape drift, not mass unpublishing). Previously one 5xx killed corpus -> shards -> verdict for the week.registry-corpus-*entries are pruned after each save (repo budget is 10GB shared; unpruned, the census would evict other workflows' caches within weeks). A Thursday cron keeps the cache inside GitHub's 7-day eviction window.matrix-verdictrunsif: always(): one failed shard degrades to a partial verdict, not a skipped check../scripts/utils.jsto OUR module - wrong-code-under-test); entry-cap truncation is now recorded in the manifest (the comment claimed it already was); missing corpus fails friendly.Measurement changes
setupFiles: []was a mergeConfig-concatenation no-op, so all 1,879 packs executed under the unit suite'svi.useFakeTimers()by accident. Now assigned explicitly:vitest.setup.tskept, fake timers dropped. Poison proof re-validated under real timers (14/14 channels).The old gated metric was only "pack had >=1 entry load" - 59-of-60 entries failing still counted as loaded. The delta gate catches gradual erosion (97.4 -> 95.9) that absolute floors ignore.
Validated locally
pnpm typecheckgreen with the runner includedDETECTION PROOF PASSED(14 checks incl. the two new self-check assertions), summarizer exit 1 naming three breachesentry files clean dropped 2.4ppNote: the fake-timer removal shifts measurement conditions; the first CI run of this PR re-baselines under real timers, and if entry-clean lands under the 91% floor the floor gets adjusted here before merge (that is what the review run is for).
Deferred by design (discussed): shard autoscaling vs registry growth; registerNodeDef floor kept as a pure harness canary; V1 DEFS drift is now loud via the self-check.