Repository navigation
ci: weekly full-registry corpus refresh (cached, ETag-revalidated) - #15081
benjcooley wants to merge 5 commits into
Conversation
Vendors the census scanner from ComfyUI_ECS_Compat_Check (pure stdlib, adapted to a single cacheable CENSUS_ROOT) and runs it weekly: registry snapshot re-pinned, corpus restored from the actions cache, drifted packs revalidated per-pack by tarball ETag HEAD-check (--revalidate, added here), scan table in the run Summary, results + lockfile as artifact, cache saved under a lockfile-hash key so it updates exactly when a pack moves. Radar, never a gate: the job fails only on mechanical errors. Validated locally against the study's corpus: byte-identical census (4,983 packs, 701/14.1% confirmed-break, 37.3% of downloads); warm scan ~50s; revalidation ~1s per 30 packs at 16 workers.
📝 WalkthroughWalkthroughThis change adds a registry snapshot refresh tool, a cached corpus fetcher with ETag revalidation, shared census paths, and a scheduled GitHub Actions workflow. It also documents local execution and ignores the generated census directory. ChangesRegistry Census
Estimated code review effort: 3 (Moderate) | ~30 minutes Sequence Diagram(s)sequenceDiagram
participant GitHubActions
participant refresh_registry
participant fetch_corpus
participant CensusCache
GitHubActions->>refresh_registry: refresh registry snapshot
refresh_registry-->>GitHubActions: write registry snapshot
GitHubActions->>CensusCache: restore cached corpus
GitHubActions->>fetch_corpus: revalidate and fetch corpus
fetch_corpus-->>GitHubActions: write corpus and lockfile
GitHubActions->>CensusCache: save corpus using lockfile hash
Important Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 inconclusive)
✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🎭 Playwright: ✅ 1806 passed, 0 failed · 1 flaky📊 Browser Reports
📦 Bundle: 8.71 MB gzip ⚪ 0 BDetailsSummary
Category Glance App Entry Points — 3.67 kB (baseline 3.67 kB) • ⚪ 0 BMain entry bundles and manifests Status: 1 unchanged Graph Workspace — 1.36 MB (baseline 1.36 MB) • ⚪ 0 BGraph editor runtime, canvas, workflow orchestration Status: 3 unchanged Views & Navigation — 124 kB (baseline 124 kB) • ⚪ 0 BTop-level views, pages, and routed surfaces Status: 17 unchanged Panels & Settings — 565 kB (baseline 565 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens Status: 26 unchanged User & Accounts — 27 kB (baseline 27 kB) • ⚪ 0 BAuthentication, profile, and account management bundles Status: 10 unchanged Editors & Dialogs — 125 kB (baseline 125 kB) • ⚪ 0 BModals, dialogs, drawers, and in-app editors Status: 8 unchanged UI Components — 67.1 kB (baseline 67.1 kB) • ⚪ 0 BReusable component library chunks Status: 14 unchanged Data & Services — 3.51 MB (baseline 3.51 MB) • ⚪ 0 BStores, services, APIs, and repositories Status: 17 unchanged Utilities & Hooks — 550 kB (baseline 550 kB) • ⚪ 0 BHelpers, composables, and utility bundles Status: 38 unchanged Vendor & Third-Party — 16.3 MB (baseline 16.3 MB) • ⚪ 0 BExternal libraries and shared vendor chunks Status: 17 unchanged Other — 14.1 MB (baseline 14.1 MB) • ⚪ 0 BBundles that do not match a named category Status: 285 unchanged ⚡ Performance Report
✅ No regressions detected. All metrics
Historical variance (last 15 runs)
Trend (last 15 commits on main)
Raw data{
"timestamp": "2026-08-12T04:41:01.973Z",
"gitSha": "764a378d87f749bc876cc11e72e18224f360146d",
"branch": "benjcooley/registry-census",
"measurements": [
{
"name": "canvas-idle",
"durationMs": 1990.9289999999942,
"styleRecalcs": 10,
"styleRecalcDurationMs": 6.109999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 383.61,
"heapDeltaBytes": 5801280,
"heapUsedBytes": 70216604,
"domNodes": 20,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 12.942,
"eventListeners": 6,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "canvas-idle",
"durationMs": 2014.4649999999729,
"styleRecalcs": 10,
"styleRecalcDurationMs": 6.101999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 400.16300000000007,
"heapDeltaBytes": 5837664,
"heapUsedBytes": 70206948,
"domNodes": 20,
"jsHeapTotalBytes": 24903680,
"scriptDurationMs": 12.863999999999999,
"eventListeners": 6,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66999999999998,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "canvas-mouse-sweep",
"durationMs": 1751.940999999988,
"styleRecalcs": 73,
"styleRecalcDurationMs": 26.704999999999995,
"layouts": 12,
"layoutDurationMs": 2.979,
"taskDurationMs": 689.705,
"heapDeltaBytes": 159452,
"heapUsedBytes": 64701884,
"domNodes": 55,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 88.908,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "canvas-mouse-sweep",
"durationMs": 1702.538000000004,
"styleRecalcs": 71,
"styleRecalcDurationMs": 23.736,
"layouts": 12,
"layoutDurationMs": 2.4499999999999997,
"taskDurationMs": 616.5500000000001,
"heapDeltaBytes": 185880,
"heapUsedBytes": 64604944,
"domNodes": 53,
"jsHeapTotalBytes": 25427968,
"scriptDurationMs": 79.556,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1685.0190000000111,
"styleRecalcs": 32,
"styleRecalcDurationMs": 13.356,
"layouts": 6,
"layoutDurationMs": 0.573,
"taskDurationMs": 314.14200000000005,
"heapDeltaBytes": 8771116,
"heapUsedBytes": 73191704,
"domNodes": 79,
"jsHeapTotalBytes": 24379392,
"scriptDurationMs": 14.632000000000001,
"eventListeners": 19,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1740.3400000000602,
"styleRecalcs": 31,
"styleRecalcDurationMs": 13.687,
"layouts": 6,
"layoutDurationMs": 0.66,
"taskDurationMs": 307.60200000000003,
"heapDeltaBytes": 8759380,
"heapUsedBytes": 73250328,
"domNodes": 78,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 15.391,
"eventListeners": 21,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "dom-widget-clipping",
"durationMs": 454.371999999978,
"styleRecalcs": 13,
"styleRecalcDurationMs": 6.091000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 280.32599999999996,
"heapDeltaBytes": -11217692,
"heapUsedBytes": 52888456,
"domNodes": 22,
"jsHeapTotalBytes": 25952256,
"scriptDurationMs": 37.900000000000006,
"eventListeners": 2,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666682,
"p95FrameDurationMs": 16.799999999999727
},
{
"name": "dom-widget-clipping",
"durationMs": 496.77900000000363,
"styleRecalcs": 12,
"styleRecalcDurationMs": 5.0470000000000015,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 285.339,
"heapDeltaBytes": -11353712,
"heapUsedBytes": 53159648,
"domNodes": 20,
"jsHeapTotalBytes": 25165824,
"scriptDurationMs": 40.667,
"eventListeners": 2,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000273
},
{
"name": "large-graph-idle",
"durationMs": 2024.182000000053,
"styleRecalcs": 10,
"styleRecalcDurationMs": 5.786999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 546.092,
"heapDeltaBytes": 7376800,
"heapUsedBytes": 67254328,
"domNodes": -282,
"jsHeapTotalBytes": 2985984,
"scriptDurationMs": 65.054,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-idle",
"durationMs": 2037.399999999991,
"styleRecalcs": 10,
"styleRecalcDurationMs": 6.1,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 583.2230000000001,
"heapDeltaBytes": 7331100,
"heapUsedBytes": 67339836,
"domNodes": -281,
"jsHeapTotalBytes": 2461696,
"scriptDurationMs": 71.427,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-pan",
"durationMs": 2099.4610000000193,
"styleRecalcs": 69,
"styleRecalcDurationMs": 10.546000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 978.982,
"heapDeltaBytes": 4495852,
"heapUsedBytes": 66983556,
"domNodes": -280,
"jsHeapTotalBytes": 4489216,
"scriptDurationMs": 296.374,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-pan",
"durationMs": 2096.806000000015,
"styleRecalcs": 68,
"styleRecalcDurationMs": 9.323,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 946.626,
"heapDeltaBytes": 4942596,
"heapUsedBytes": 68015948,
"domNodes": -282,
"jsHeapTotalBytes": 3964928,
"scriptDurationMs": 292.24899999999997,
"eventListeners": -147,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "large-graph-zoom",
"durationMs": 3104.241000000002,
"styleRecalcs": 66,
"styleRecalcDurationMs": 11.279000000000002,
"layouts": 60,
"layoutDurationMs": 6.2090000000000005,
"taskDurationMs": 1149.6339999999998,
"heapDeltaBytes": -8362704,
"heapUsedBytes": 53923640,
"domNodes": 14,
"jsHeapTotalBytes": 5607424,
"scriptDurationMs": 356.317,
"eventListeners": -181,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-zoom",
"durationMs": 3069.4599999999355,
"styleRecalcs": 65,
"styleRecalcDurationMs": 11.015999999999998,
"layouts": 60,
"layoutDurationMs": 6.3340000000000005,
"taskDurationMs": 1130.941,
"heapDeltaBytes": 22777212,
"heapUsedBytes": 85556104,
"domNodes": 14,
"jsHeapTotalBytes": 7864320,
"scriptDurationMs": 362.665,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "minimap-idle",
"durationMs": 2004.8320000000217,
"styleRecalcs": 9,
"styleRecalcDurationMs": 5.332,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 556.614,
"heapDeltaBytes": 5142496,
"heapUsedBytes": 66184984,
"domNodes": -281,
"jsHeapTotalBytes": 4296704,
"scriptDurationMs": 70.33,
"eventListeners": -179,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "minimap-idle",
"durationMs": 2020.232999999962,
"styleRecalcs": 9,
"styleRecalcDurationMs": 5.358000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 564.4449999999999,
"heapDeltaBytes": 6764612,
"heapUsedBytes": 68175964,
"domNodes": -280,
"jsHeapTotalBytes": 3772416,
"scriptDurationMs": 68.58900000000001,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 534.3260000000214,
"styleRecalcs": 46,
"styleRecalcDurationMs": 7.152000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 337.486,
"heapDeltaBytes": -10578160,
"heapUsedBytes": 53930612,
"domNodes": 18,
"jsHeapTotalBytes": 25690112,
"scriptDurationMs": 93.931,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.663333333333338,
"p95FrameDurationMs": 16.700000000000273
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 527.1240000000716,
"styleRecalcs": 47,
"styleRecalcDurationMs": 7.693000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 316.515,
"heapDeltaBytes": -10997352,
"heapUsedBytes": 53482184,
"domNodes": 20,
"jsHeapTotalBytes": 25952256,
"scriptDurationMs": 91.044,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-idle",
"durationMs": 1996.530000000007,
"styleRecalcs": 10,
"styleRecalcDurationMs": 5.649999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 396.02000000000004,
"heapDeltaBytes": 5820924,
"heapUsedBytes": 70381496,
"domNodes": 20,
"jsHeapTotalBytes": 24903680,
"scriptDurationMs": 11.479,
"eventListeners": 6,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-idle",
"durationMs": 2020.4779999999687,
"styleRecalcs": 11,
"styleRecalcDurationMs": 6.458,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 384.289,
"heapDeltaBytes": 5934936,
"heapUsedBytes": 70419660,
"domNodes": 22,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 12.174000000000001,
"eventListeners": 6,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1676.0949999999752,
"styleRecalcs": 76,
"styleRecalcDurationMs": 25.928,
"layouts": 16,
"layoutDurationMs": 3.11,
"taskDurationMs": 570.1820000000001,
"heapDeltaBytes": -3722732,
"heapUsedBytes": 60701100,
"domNodes": 62,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 59.413999999999994,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.670000000000012,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1675.5589999999643,
"styleRecalcs": 76,
"styleRecalcDurationMs": 26.777,
"layouts": 16,
"layoutDurationMs": 3.19,
"taskDurationMs": 590.425,
"heapDeltaBytes": -3875800,
"heapUsedBytes": 60816296,
"domNodes": 62,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 60.92900000000001,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-transition-enter",
"durationMs": 932.9500000000053,
"styleRecalcs": 19,
"styleRecalcDurationMs": 24.562999999999995,
"layouts": 15,
"layoutDurationMs": 8.148,
"taskDurationMs": 716.739,
"heapDeltaBytes": 28601816,
"heapUsedBytes": 95983216,
"domNodes": 13673,
"jsHeapTotalBytes": 16777216,
"scriptDurationMs": 26.678999999999995,
"eventListeners": 2373,
"totalBlockingTimeMs": 107,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "viewport-pan-sweep",
"durationMs": 8111.953999999969,
"styleRecalcs": 249,
"styleRecalcDurationMs": 30.743,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 3496.4719999999998,
"heapDeltaBytes": 8971152,
"heapUsedBytes": 72025752,
"domNodes": -281,
"jsHeapTotalBytes": 8683520,
"scriptDurationMs": 991.347,
"eventListeners": -133,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "viewport-pan-sweep",
"durationMs": 8112.972000000013,
"styleRecalcs": 249,
"styleRecalcDurationMs": 27.836999999999996,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 3432.459,
"heapDeltaBytes": 9953316,
"heapUsedBytes": 69506132,
"domNodes": -282,
"jsHeapTotalBytes": 3964928,
"scriptDurationMs": 973.8469999999999,
"eventListeners": -133,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-idle",
"durationMs": 12280.342000000019,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 12254.829,
"heapDeltaBytes": -34635532,
"heapUsedBytes": 171833848,
"domNodes": -8312,
"jsHeapTotalBytes": -11476992,
"scriptDurationMs": 461.966,
"eventListeners": -16389,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-idle",
"durationMs": 11539.904999999977,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 11522.005,
"heapDeltaBytes": -55784620,
"heapUsedBytes": 166965116,
"domNodes": -8312,
"jsHeapTotalBytes": -10690560,
"scriptDurationMs": 397.30000000000007,
"eventListeners": -16391,
"totalBlockingTimeMs": 0,
"frameDurationMs": 17.219999999999953,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "vue-large-graph-pan",
"durationMs": 15465.168000000005,
"styleRecalcs": 75,
"styleRecalcDurationMs": 13.279000000000012,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 15421.270999999999,
"heapDeltaBytes": -28927800,
"heapUsedBytes": 176870492,
"domNodes": -8312,
"jsHeapTotalBytes": -3153920,
"scriptDurationMs": 706.6179999999999,
"eventListeners": -16387,
"totalBlockingTimeMs": 35,
"frameDurationMs": 17.223333333333237,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-pan",
"durationMs": 14404.854999999998,
"styleRecalcs": 72,
"styleRecalcDurationMs": 12.568999999999997,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 14362.612000000003,
"heapDeltaBytes": -59917992,
"heapUsedBytes": 158101716,
"domNodes": -8312,
"jsHeapTotalBytes": -8589312,
"scriptDurationMs": 727.3629999999999,
"eventListeners": -16389,
"totalBlockingTimeMs": 14,
"frameDurationMs": 17.219999999999953,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "workflow-execution",
"durationMs": 427.35699999997223,
"styleRecalcs": 14,
"styleRecalcDurationMs": 12.261999999999999,
"layouts": 2,
"layoutDurationMs": 0.31499999999999984,
"taskDurationMs": 75.85799999999999,
"heapDeltaBytes": 5069128,
"heapUsedBytes": 68653448,
"domNodes": 128,
"jsHeapTotalBytes": 5242880,
"scriptDurationMs": 6.796999999999999,
"eventListeners": 97,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000273
},
{
"name": "workflow-execution",
"durationMs": 433.8639999999714,
"styleRecalcs": 14,
"styleRecalcDurationMs": 15.05,
"layouts": 4,
"layoutDurationMs": 1.1790000000000003,
"taskDurationMs": 87.95100000000002,
"heapDeltaBytes": 5056296,
"heapUsedBytes": 68592012,
"domNodes": 123,
"jsHeapTotalBytes": 4980736,
"scriptDurationMs": 7.075999999999999,
"eventListeners": 99,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000273
}
]
} |
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #15081 +/- ##
==========================================
- Coverage 80.81% 80.71% -0.11%
==========================================
Files 1873 1882 +9
Lines 121954 122466 +512
Branches 37495 37739 +244
==========================================
+ Hits 98558 98847 +289
- Misses 22859 23056 +197
- Partials 537 563 +26
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
benjcooley
left a comment
There was a problem hiding this comment.
Automated review — posted by an AI agent (Claude Code)
Weekly full-registry census: new workflow + vendored Python scripts. The headline problem is that the corpus fetch silently extracts nothing on GNU tar (ubuntu-latest), and every downstream failure mode scores the affected pack as clean — so the census would report a confident zero.
| Severity | Count |
|---|---|
| BLOCKER | 3 |
| SHOULD FIX | 4 |
| TRIVIAL | 1 |
| Total | 8 |
| if frozen and prev.get('etag') and etag and etag != prev['etag']: | ||
| return (pack_id, 'drifted', etag) | ||
|
|
||
| args = ['tar', '-xz', '-f', tar_path, '-C', dest] |
There was a problem hiding this comment.
BLOCKER — tar -xz --include=*.js is a bsdtar-only option. ubuntu-latest ships GNU tar, which errors out on it. The return code is discarded, so on CI every pack extracts to nothing, gets marked .done with status empty, and the zeroed corpus is cached and never revalidated.
This validated locally on macOS (bsdtar) — that's why it looked fine. GNU equivalent is --wildcards '*.js'.
Posted by an AI agent (Claude Code).
There was a problem hiding this comment.
Fixed in 00e45f0 - and this was the catch of the PR: validated on macOS bsdtar, would have shipped a silently-empty corpus reporting 100% clean on CI. The fix avoids tar filter dialects entirely: extract whole to scratch (universal flags only), filter extensions in Python, check the return code, swap in only on success. Revalidated by force-refetching two packs through the new path (extension-filtered trees, markers written, census unchanged).
| timeout-minutes: 60 | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v7 |
There was a problem hiding this comment.
BLOCKER — actions/checkout@v7 isn't in .pinact.yaml's ignore list (repo standard is v6), and the actions/cache/restore@v5 / save@v5 sub-action names aren't exempted either. Confirmed: the validate-pins check is currently failing on this PR.
Posted by an AI agent (Claude Code).
There was a problem hiding this comment.
Fixed in 00e45f0: the two cache sub-action lines are SHA-pinned (caa29612…# v5) - you were right that pinact matches names exactly, so the actions/cache ignore entry never covered cache/restore/cache/save; checkout@v7 itself is ignore-listed and passes. validate-pins' own log flags exactly lines 47/92, both now pinned.
| n = len(rx.findall(line)) | ||
| if n: | ||
| counts[k] += n | ||
| results[x['id']] = { |
There was a problem hiding this comment.
BLOCKER — Packs that failed to fetch / bad-url / unsupported-host still get a zero-count result row and stay in the denominator, so they're counted as clean. The scanner never reads corpus.lock.json, so fetch_corpus's "honest status" tracking never reaches the census. Combined with the tar bug above, a fully-empty corpus reports 100% clean.
Posted by an AI agent (Claude Code).
There was a problem hiding this comment.
Fixed in 00e45f0, via the disk rather than the lockfile: the .done marker is written only after a successful fetch+extract, so marker-on-disk is the fetch-status ground truth and survives lockfile schema drift (a lock-status filter turned out to exclude 4,960 legacy entries that predate status recording). Unfetched packs are excluded from the scanned population and reported as their own census line: unscannable (excluded) 9 packs … unknown, not clean. Combined with the tar fix, an empty corpus now reports 0 scanned and exits 2 instead of 100% clean.
| live = _head_etag(url) | ||
| if live is None or not prev.get('etag') or live == prev['etag']: | ||
| return (pack_id, 'cached', prev.get('etag')) | ||
| shutil.rmtree(dest, ignore_errors=True) |
There was a problem hiding this comment.
SHOULD FIX — --revalidate rmtrees the cached pack before the refetch. If the refetch fails, the pack is gone and the run scores it as clean — directly contradicting the "a network blip must not evict corpus" comment right here. Fetch to a temp dir and swap on success.
Posted by an AI agent (Claude Code).
There was a problem hiding this comment.
Fixed in 00e45f0: no rmtree before refetch anywhere - the download extracts to scratch, filters into a staged tree, and only on success replaces dest. A failed refetch leaves the previous copy untouched, which is what that comment always promised.
| # not evict corpus. | ||
| live = _head_etag(url) | ||
| if live is None or not prev.get('etag') or live == prev['etag']: | ||
| return (pack_id, 'cached', prev.get('etag')) |
There was a problem hiding this comment.
SHOULD FIX — not prev.get('etag') short-circuits to cached, so a pack whose ETag was never recorded (host sends none, or the lockfile was lost) is pinned to its first fetch forever. Also _head_etag (no -L, first ETag) vs the GET path (-L, last ETag) compare different values, so redirecting hosts will look permanently changed.
Posted by an AI agent (Claude Code).
There was a problem hiding this comment.
Fixed in 00e45f0: a missing recorded ETag with a live one present now refetches once to establish the record (no more first-fetch-forever pin; a host sending no ETag at all stays cached since there is nothing to compare), and _head_etag now follows redirects (-L) keeping the LAST ETag, mirroring the GET parse - redirecting hosts compare the same value both ways.
| uses: actions/cache/save@v5 | ||
| with: | ||
| path: .census | ||
| key: registry-corpus-${{ hashFiles('.census/corpus.lock.json') }} |
There was a problem hiding this comment.
SHOULD FIX — hashFiles on a missing lockfile yields '', so an if: always() save after an early failure writes a partial .census under key registry-corpus- — the exact string used as the restore-keys prefix. That poisoned entry is then the best prefix match for every later run.
Posted by an AI agent (Claude Code).
There was a problem hiding this comment.
Fixed in 00e45f0: the save gates on steps.fetch.outcome == 'success' && hashFiles(…) != '' - an early failure can no longer save a partial .census, least of all under the empty-hash key that doubles as the restore-keys prefix.
| return '/'.join(parts[-2:]) | ||
|
|
||
|
|
||
| def fetch(x): |
There was a problem hiding this comment.
SHOULD FIX — This dead fetch() builds a shell string from the registry-supplied repo URL and runs it with shell=True (command injection if ever wired up), and re-introduces the unconditional .done marker the real fetcher was fixed to avoid. Delete it rather than leaving it as a trap for the next editor.
Posted by an AI agent (Claude Code).
There was a problem hiding this comment.
Deleted in 00e45f0 - fetch(), slug_of, the INC shell string, and the ThreadPoolExecutor import are gone; the scanner reads the corpus and nothing else.
| unc = [v for v in results.values() if any(v['counts'].get(k) for k in UNCOND)] | ||
| vue = [v for v in results.values() if any(v['counts'].get(k) for k in VUE)] | ||
| both = [v for v in results.values() if any(v['counts'].get(k) for k in {**UNCOND, **VUE})] | ||
| n = len(results) |
There was a problem hiding this comment.
TRIVIAL — len(s)/n*100 and /TOT are unguarded, so an empty snapshot crashes with ZeroDivisionError. The module-level snapshot load also bypasses the friendly "corpus not fetched" guard.
Posted by an AI agent (Claude Code).
There was a problem hiding this comment.
Fixed in 00e45f0: missing snapshot exits 2 with the refresh instruction before the module-level load, zero scannable packs exits 2 with its own message, and the download denominators guard against zero.
…nsus tar --include is bsdtar-only: on GNU tar (ubuntu CI) it errors with the return code discarded, producing an empty corpus that scanned 100% clean - caught in review, invisible to the macOS validation. Extraction now uses only universal flags (extract whole, filter extensions in Python), checks the return code, and stages the replacement tree before swapping, so a failed refetch never evicts the previous copy. --revalidate treats a missing recorded ETag as drift (refetch once to establish it) and the HEAD check follows redirects taking the last ETag, matching the GET path. The scanner excludes unfetched packs from the scanned population via the .done marker (disk ground truth; unknown is never clean) and reports them as their own census line; the dead shell=True fetch() is deleted; empty-registry and missing-snapshot paths exit with messages instead of ZeroDivisionError. The workflow pins the cache sub-actions by SHA (pinact matches names exactly; the actions/cache ignore entry does not cover restore/save) and gates the save on the fetch step plus a present lockfile so a partial .census can never save under the empty-hash key, which equals the restore-keys prefix. Revalidated end-to-end locally: two packs force-refetched through the new extraction (extension-filtered trees, markers written), full census byte-identical on the break figures (701 / 14.1% / 37.3%) over 4,974 scanned packs with 9 unscannable now explicitly excluded.
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/registry-census/fetch_corpus.py`:
- Around line 254-261: Update the return logic in the function containing the
counts.get('failed') check so any failed fetch reports the error and returns a
nonzero status. Preserve the existing successful return of 0 when no fetches
failed, ensuring the workflow cannot publish or cache a partial census
successfully.
- Around line 153-154: Update the frozen-mode validation in the corpus fetch
flow so an existing previous ETag requires the response ETag to be present and
equal to it; return the existing drifted result when the response ETag is
missing or differs. Keep normal-mode behavior and the existing lockfile ETag
handling unchanged.
In `@scripts/registry-census/scan_registry.py`:
- Around line 113-131: The download percentages in the summary loop over
shipsjs, api, unc, vue, and both use TOT, which excludes unscannable packs,
while the label says “of registry downloads.” Update this reporting in the
census summary to either use reg_dl as the denominator and identify the values
as registry-wide lower bounds, or keep TOT and relabel them as percentages of
scanned downloads.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 444e9aca-8f6e-4e53-8525-20f2c7b944c1
📒 Files selected for processing (7)
.github/workflows/ci-weekly-registry-census.yaml.gitignorescripts/registry-census/README.mdscripts/registry-census/fetch_corpus.pyscripts/registry-census/paths.pyscripts/registry-census/refresh_registry.pyscripts/registry-census/scan_registry.py
| if frozen and prev.get('etag') and etag and etag != prev['etag']: | ||
| return (pack_id, 'drifted', etag) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Fail frozen mode when the response has no ETag.
If prev['etag'] exists and the GET response has no ETag, this condition is false. The code then replaces the cached corpus but keeps the old lockfile ETag. The frozen run reports success without verifying the pinned artifact.
Proposed fix
- if frozen and prev.get('etag') and etag and etag != prev['etag']:
+ if frozen and prev.get('etag') and etag != prev['etag']:
return (pack_id, 'drifted', etag)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/registry-census/fetch_corpus.py` around lines 153 - 154, Update the
frozen-mode validation in the corpus fetch flow so an existing previous ETag
requires the response ETag to be present and equal to it; return the existing
drifted result when the response ETag is missing or differs. Keep normal-mode
behavior and the existing lockfile ETag handling unchanged.
| # A failed fetch is not a clean pack. Surface it rather than caching it. | ||
| if counts.get('failed'): | ||
| print( | ||
| f"\n{counts['failed']} fetches failed; re-run to retry " | ||
| f'(they are not marked done and will not be skipped)', | ||
| file=sys.stderr, | ||
| ) | ||
| return 0 |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Return failure when a pack fetch fails.
The workflow continues to scan, publish, and cache after a failed fetch because this function returns zero. A failed fetch is a mechanical pipeline error. Return nonzero after reporting it so the weekly job cannot publish a partial census as successful.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/registry-census/fetch_corpus.py` around lines 254 - 261, Update the
return logic in the function containing the counts.get('failed') check so any
failed fetch reports the error and returns a nonzero status. Preserve the
existing successful return of 0 when no fetches failed, ensuring the workflow
cannot publish or cache a partial census successfully.
| TOT = sum(v['downloads'] for v in results.values()) or 1 | ||
| shipsjs = [v for v in results.values() if v['js_loc'] > 0] | ||
| api = [v for v in results.values() if v['counts'].get('uses_api')] | ||
| unc = [v for v in results.values() if any(v['counts'].get(k) for k in UNCOND)] | ||
| vue = [v for v in results.values() if any(v['counts'].get(k) for k in VUE)] | ||
| both = [v for v in results.values() if any(v['counts'].get(k) for k in {**UNCOND, **VUE})] | ||
| n = len(results) | ||
| reg_dl = sum(dl.values()) or 1 | ||
| print(f'\n=== FULL REGISTRY CENSUS (n={n} scanned packs, {TOT:,} downloads) ===') | ||
| print( | ||
| f'{"unscannable (excluded)":24s} {len(unfetched):5d} packs ' | ||
| f'{sum(dl.get(x["id"], 0) for x in unfetched) / reg_dl * 100:5.1f}% of registry downloads ' | ||
| f'(failed/bad-url/unsupported/never fetched - unknown, not clean)' | ||
| ) | ||
| for lbl, s in (('ships frontend JS', shipsjs), ('uses Comfy JS API', api), | ||
| ('UNCONDITIONAL break', unc), ('VUE-MODE break', vue), | ||
| ('ANY confirmed break', both)): | ||
| print(f'{lbl:24s} {len(s):5d} packs ({len(s)/n*100:5.1f}%) ' | ||
| f'{sum(v["downloads"] for v in s)/TOT*100:5.1f}% of registry downloads') |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Correct the download-percentage denominator label.
TOT excludes unscannable packs, but Lines 130-131 label each value as a percentage “of registry downloads.” This overstates the coverage of the reported percentage when excluded packs have downloads. Either divide by reg_dl and label the result as a registry-wide lower bound, or retain TOT and say “of scanned downloads.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/registry-census/scan_registry.py` around lines 113 - 131, The
download percentages in the summary loop over shipsjs, api, unc, vue, and both
use TOT, which excludes unscannable packs, while the label says “of registry
downloads.” Update this reporting in the census summary to either use reg_dl as
the denominator and identify the values as registry-wide lower bounds, or keep
TOT and relabel them as percentages of scanned downloads.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci-weekly-registry-census.yaml:
- Line 47: Update the cache action references in the workflow’s restore and save
steps to use the repository-approved mutable `v5` tags: change
`actions/cache/restore` and `actions/cache/save` to their respective `@v5`
references, removing the SHA pins.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 870a2c19-2c08-451f-82fb-ac2df53b3395
📒 Files selected for processing (1)
.github/workflows/ci-weekly-registry-census.yaml
| # Restore the newest corpus regardless of key: the primary key never | ||
| # hits (run-scoped), so restore-keys always serves the latest save. | ||
| - name: Restore corpus cache | ||
| uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5.1.0 |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
config="$(fd -HI -t f '^\.pinact\.yaml$' . | head -n 1)"
test -n "$config" || { echo "No .pinact.yaml found" >&2; exit 1; }
rg -n -C 3 'ignore|actions/(cache|checkout)' "$config" || true
rg -n 'uses:\s*actions/cache/(restore|save)@' \
.github/workflows/ci-weekly-registry-census.yamlRepository: Comfy-Org/ComfyUI_frontend
Length of output: 615
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- .pinact.yaml ---'
cat -n .pinact.yaml
printf '%s\n' '--- pinact references and validation commands ---'
rg -n -i -C 2 'pinact|action.?pin|workflow.*lint|lint.*workflow' \
.github package.json Makefile* README.md 2>/dev/null || true
printf '%s\n' '--- cache action references ---'
rg -n -C 1 'actions/cache/(restore|save)@' .github/workflowsRepository: Comfy-Org/ComfyUI_frontend
Length of output: 3502
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
cat -n .github/workflows/ci-validate-action-pins.yamlRepository: Comfy-Org/ComfyUI_frontend
Length of output: 721
Use the repository-approved actions/cache references.
.pinact.yaml excludes actions/cache from SHA pinning at v5. Change lines 47 and 97 to actions/cache/restore@v5 and actions/cache/save@v5.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci-weekly-registry-census.yaml at line 47, Update the
cache action references in the workflow’s restore and save steps to use the
repository-approved mutable `v5` tags: change `actions/cache/restore` and
`actions/cache/save` to their respective `@v5` references, removing the SHA
pins.
Source: Learnings
Paths-filtered pull_request trigger: self-testing for census changes, and the registration path that lets workflow_dispatch resolve before the file reaches the default branch.
benjcooley
left a comment
There was a problem hiding this comment.
APPROVED — posted by an AI agent (Claude Code)
(GitHub blocks a formal Approve on your own PR, so this is a comment review.)
00e45f06 + 8a6e0a24 resolve all eight findings. One inline TRIVIAL nit remains, and I owe a correction on my own earlier finding — see #2.
| # | Severity | Status |
|---|---|---|
| 1 | BLOCKER | ✅ Extraction now uses only universal flags, checks the return code, filters extensions in Python, and stages before swapping. Verified INCLUDES is a tuple (line 44) — required, since str.endswith rejects a list — so the new filter is sound. |
| 2 | BLOCKER | ✅ Fixed, but my diagnosis was half wrong. The real cause was that pinact matches names exactly, so the actions/cache ignore entry never covered actions/cache/restore / save; SHA-pinning those fixes it. The checkout@v7 half of my finding was incorrect — validate-pins now passes with @v7 still in place. Sorry for the misdirection. |
| 3 | BLOCKER | ✅ The .done marker is now the fetch-status ground truth, and it's written only after a successful extract (the rc != 0 path returns before it). Unfetched packs are excluded from the denominator and get their own census line. "Unknown is never clean" is the right invariant. |
| 4 | SHOULD FIX | ✅ The cached tree stays until the replacement is fully staged; a failed refetch no longer evicts it. |
| 5 | SHOULD FIX | ✅ Both halves — a missing recorded ETag now refetches once to establish the record, and _head_etag uses -L + last-ETag to match the GET path. |
| 6 | SHOULD FIX | ✅ Save gated on steps.fetch.outcome == 'success' && hashFiles(...) != '', so the empty-hash key that collides with the restore-keys prefix is unreachable. |
| 7 | SHOULD FIX | ✅ Dead shell=True fetch() deleted outright. |
| 8 | TRIVIAL | ✅ or 1 guards, plus an explicit missing-snapshot exit and an empty-results exit, both with real messages. |
The end-to-end revalidation — census byte-identical on the break figures (701 / 14.1% / 37.3%) over 4,974 scanned packs with 9 now explicitly excluded — is the right way to prove the extraction rewrite didn't change what's measured.
| timeout-minutes: 60 | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v7 |
There was a problem hiding this comment.
TRIVIAL — Consistency only, not a gate: validate-pins passes with @v7, but all 80 other actions/checkout uses in .github/workflows/ are @v6, and .pinact.yaml records ref: v6 for it. Worth matching so this file isn't the lone outlier.
Posted by an AI agent (Claude Code).
|
First CI execution (run 31555968423, green, ~6 minutes total cold): every reviewed behavior visible in the logs.
Steady-state weekly cost: ~3-4 minutes (ETag revalidation + scan). The next census-touching push doubles as the warm-path proof. |
The scan's false-positive/negative rate was judged too high to act on; packs are measured by executing their code (the ecosystem matrix, stacked PR) instead. The weekly job now maintains the pack corpus - registry re-pin, fetch, ETag revalidation, lockfile-keyed cache - as the matrix's input. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci-weekly-registry-census.yaml:
- Around line 60-67: Update the weekly registry census flow around Refresh
registry snapshot and fetch_corpus.py --revalidate to remove packs no longer
present in the refreshed registry snapshot, including stale corpus directories
and lockfile entries restored from cache. Implement pruning before or within
fetch_corpus.py, ensure all consumers exclude removed pack IDs, and add coverage
using a removed-pack fixture.
- Around line 69-76: Update fetch_corpus.py so the fetch step exits
unsuccessfully whenever counts.failed is nonzero, including partial-fetch
failures that retain previous lock entries. Then change the “Upload corpus
lockfile” step to run only after a fully successful fetch instead of always(),
preserving artifact upload for complete results while preventing stale
lockfiles.
In `@scripts/registry-census/README.md`:
- Around line 29-35: Update the “In CI” section of the README to document that
the workflow also runs on pull requests, limited by changes to
scripts/registry-census/** and .github/workflows/ci-weekly-registry-census.yaml,
while preserving the existing weekly and dispatch behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 4feb5b14-63df-44df-9bb4-073288b45e9a
📒 Files selected for processing (2)
.github/workflows/ci-weekly-registry-census.yamlscripts/registry-census/README.md
| # Re-pin the registry snapshot every run: it is the corpus's target | ||
| # list, and new packs only enter the corpus through it. | ||
| - name: Refresh registry snapshot | ||
| run: python3 scripts/registry-census/refresh_registry.py | ||
|
|
||
| - name: Fetch corpus and revalidate drifted packs | ||
| id: fetch | ||
| run: python3 scripts/registry-census/fetch_corpus.py --revalidate |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
ast-grep outline scripts/registry-census/fetch_corpus.py --items all
rg -n -C 6 \
'rmtree|remove|CORPUS|LOCKFILE|registry_snapshot|registry_js|\.done' \
scripts .github --glob '*.py' --glob '*.yaml' --glob '*.yml'Repository: Comfy-Org/ComfyUI_frontend
Length of output: 26066
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- registry-census files ---'
git ls-files scripts/registry-census .github/workflows/ci-weekly-registry-census.yaml | sort
printf '%s\n' '--- fetch_corpus.py ---'
cat -n scripts/registry-census/fetch_corpus.py | sed -n '95,260p'
printf '%s\n' '--- registry-census outlines ---'
for f in $(git ls-files scripts/registry-census '*.py' | sort); do
echo "### $f"
ast-grep outline "$f" --items all
done
printf '%s\n' '--- workflow cache and census steps ---'
cat -n .github/workflows/ci-weekly-registry-census.yaml | sed -n '35,125p'
printf '%s\n' '--- references to registry_js and corpus.lock.json ---'
rg -n -C 5 'registry_js|corpus\.lock\.json|CORPUS|LOCKFILE|registry_snapshot\(\)|registry\.json' \
scripts/registry-census .github/workflows/ci-weekly-registry-census.yaml \
--glob '*.py' --glob '*.yaml' --glob '*.yml'Repository: Comfy-Org/ComfyUI_frontend
Length of output: 33896
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- all registry-census references ---'
rg -n -C 5 \
'registry-census|registry_js|corpus\.lock\.json|\.census|ecosystem matrix|census' . \
--glob '!node_modules/**' --glob '!dist/**' --glob '!build/**' \
--glob '!*.lock' --glob '!pnpm-lock.yaml' \
| sed -n '1,500p'
printf '%s\n' '--- registry-census README ---'
cat -n scripts/registry-census/README.md
printf '%s\n' '--- workflow tail ---'
cat -n .github/workflows/ci-weekly-registry-census.yaml | sed -n '78,180p'
printf '%s\n' '--- tests and fixtures near registry census ---'
git ls-files | rg '(^|/)(test|tests|__fixtures__|fixtures)(/|$)|registry.*census|census.*registry' || true
printf '%s\n' '--- static contract check ---'
python3 - <<'PY'
import ast
from pathlib import Path
path = Path("scripts/registry-census/fetch_corpus.py")
tree = ast.parse(path.read_text())
calls = []
for node in ast.walk(tree):
if isinstance(node, ast.Call):
fn = node.func
if isinstance(fn, ast.Attribute):
name = f"{ast.unparse(fn.value)}.{fn.attr}"
else:
name = ast.unparse(fn)
if any(term in name.lower() for term in ("rmtree", "remove", "listdir", "scandir", "walk")):
calls.append((node.lineno, name))
print("filesystem calls relevant to cleanup/enumeration:")
for item in calls:
print(item)
print("has target snapshot load:", any(
isinstance(n, ast.Call)
and isinstance(n.func, ast.Name)
and n.func.id == "json.load"
for n in ast.walk(tree)
))
print("has corpus-root enumeration outside fetch payload check:",
any(
isinstance(n, ast.Call)
and isinstance(n.func, ast.Attribute)
and n.func.attr in {"listdir", "scandir"}
for n in ast.walk(tree)
))
PYRepository: Comfy-Org/ComfyUI_frontend
Length of output: 50384
Remove packs absent from the refreshed snapshot. fetch_corpus.py processes only current targets and never removes stale directories or lockfile entries. A restored cache can therefore retain removed packs under .census/corpus/registry_js. Prune stale pack IDs, or filter them in every consumer, and add a removed-pack fixture.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci-weekly-registry-census.yaml around lines 60 - 67,
Update the weekly registry census flow around Refresh registry snapshot and
fetch_corpus.py --revalidate to remove packs no longer present in the refreshed
registry snapshot, including stale corpus directories and lockfile entries
restored from cache. Implement pruning before or within fetch_corpus.py, ensure
all consumers exclude removed pack IDs, and add coverage using a removed-pack
fixture.
| - name: Upload corpus lockfile | ||
| if: always() | ||
| uses: actions/upload-artifact@v6 | ||
| with: | ||
| name: corpus-lock | ||
| path: .census/corpus.lock.json | ||
| retention-days: 30 | ||
| if-no-files-found: warn |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
file=".github/workflows/ci-weekly-registry-census.yaml"
printf '%s\n' '--- workflow excerpt ---'
sed -n '1,120p' "$file"
printf '%s\n' '--- relevant references ---'
rg -n -C 3 'corpus\.lock|fetch|refresh|cache|upload-artifact|steps\.' "$file" .github 2>/dev/null || true
printf '%s\n' '--- repository files related to census ---'
git ls-files | rg '(^|/)(census|corpus|registry)|ci-weekly-registry-census' || trueRepository: Comfy-Org/ComfyUI_frontend
Length of output: 50383
🏁 Script executed:
#!/bin/bash
set -eu
for file in scripts/registry-census/refresh_registry.py scripts/registry-census/fetch_corpus.py; do
printf '\n--- %s: size ---\n' "$file"
wc -l "$file"
printf '%s\n' '--- outline ---'
ast-grep outline "$file" || true
done
printf '\n--- lockfile operations ---\n'
rg -n -C 5 'corpus\.lock|write_text|open\(|rename|replace|unlink|remove|os\.|Path\(' \
scripts/registry-census/refresh_registry.py scripts/registry-census/fetch_corpus.pyRepository: Comfy-Org/ComfyUI_frontend
Length of output: 16263
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- refresh_registry.py ---'
cat -n scripts/registry-census/refresh_registry.py
printf '%s\n' '--- fetch_corpus.py main and fetch result handling ---'
sed -n '193,265p' scripts/registry-census/fetch_corpus.py | cat -n
printf '%s\n' '--- repository references to corpus-lock artifact ---'
rg -n -C 4 'corpus-lock|corpus\.lock\.json|fetch_corpus\.py' --glob '!scripts/registry-census/fetch_corpus.py' --glob '!scripts/registry-census/refresh_registry.py' .Repository: Comfy-Org/ComfyUI_frontend
Length of output: 10911
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
from pathlib import Path
import re
workflow = Path(".github/workflows/ci-weekly-registry-census.yaml").read_text()
refresh = Path("scripts/registry-census/refresh_registry.py").read_text()
fetch = Path("scripts/registry-census/fetch_corpus.py").read_text()
checks = {
"cache restores .census": "path: .census" in workflow,
"upload runs always": re.search(
r"- name: Upload corpus lockfile\s+if:\s+always\(\)", workflow
) is not None,
"fetch has id": re.search(
r"- name: Fetch corpus and revalidate drifted packs\s+id:\s+fetch", workflow
) is not None,
"fetch reads existing lock": re.search(
r"if os\.path\.exists\(LOCKFILE\):\s+with open\(LOCKFILE, encoding='utf-8'\)", fetch
) is not None,
"fetch writes lock after worker results": fetch.index(
"with open(LOCKFILE, 'w', encoding='utf-8')"
) > fetch.index("results = list("),
"refresh can exit before writing snapshot": (
"raise SystemExit(f'registry API request failed on page {page}')" in refresh
and refresh.index("raise SystemExit") < refresh.index("with open(dest, 'w'")
),
"partial fetch failures return success": re.search(
r"if counts\.get\('failed'\):.*?return 0", fetch, re.S
) is not None,
}
for name, passed in checks.items():
print(f"{'PASS' if passed else 'FAIL'}: {name}")
if not all(checks.values()):
raise SystemExit(1)
upload = re.search(
r"- name: Upload corpus lockfile(?P<body>.*?)(?=\n\s*# Key derived|\Z)",
workflow,
re.S,
).group("body")
print("upload condition:", re.search(r"if:\s*(.+)", upload).group(1))
print("fetch failure policy: returns 0 after reporting failed packs")
print("lockfile write occurs only after all fetch results are collected")
PYRepository: Comfy-Org/ComfyUI_frontend
Length of output: 558
Gate the lockfile artifact on a complete fetch result.
always() can upload a restored lockfile when refresh fails. fetch_corpus.py also returns success when individual fetches fail, while retaining their previous lock entries. Therefore, checking only steps.fetch.outcome == 'success' does not prevent stale ETags. Return a failure when counts.failed is nonzero, then gate the upload on that outcome or create a run-scoped lockfile.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci-weekly-registry-census.yaml around lines 69 - 76,
Update fetch_corpus.py so the fetch step exits unsuccessfully whenever
counts.failed is nonzero, including partial-fetch failures that retain previous
lock entries. Then change the “Upload corpus lockfile” step to run only after a
fully successful fetch instead of always(), preserving artifact upload for
complete results while preventing stale lockfiles.
| ## In CI | ||
|
|
||
| `.github/workflows/ci-weekly-registry-census.yaml` runs weekly and on | ||
| dispatch: corpus restored from the actions cache, registry re-pinned, | ||
| drifted packs revalidated by ETag, `corpus.lock.json` uploaded as an | ||
| artifact, cache re-saved under a lockfile-hash key (so the cache updates | ||
| exactly when a pack moves). |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
workflow=".github/workflows/ci-weekly-registry-census.yaml"
if [ -f "$workflow" ]; then
printf '%s\n' "--- workflow triggers ---"
sed -n '1,90p' "$workflow"
else
printf '%s\n' "Workflow not found at expected path; locating candidates:"
fd -i 'registry.*census|census.*registry' .github scripts 2>/dev/null || true
fi
printf '%s\n' "--- README section ---"
sed -n '20,45p' scripts/registry-census/README.md
printf '%s\n' "--- trigger references ---"
rg -n -C 3 'pull_request|paths:|ci-weekly-registry-census|schedule:|workflow_dispatch' .github/workflows scripts/registry-census 2>/dev/null || trueRepository: Comfy-Org/ComfyUI_frontend
Length of output: 50385
Document the pull-request trigger.
Document the pull_request trigger and these path filters: scripts/registry-census/** and .github/workflows/ci-weekly-registry-census.yaml.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/registry-census/README.md` around lines 29 - 35, Update the “In CI”
section of the README to document that the workflow also runs on pull requests,
limited by changes to scripts/registry-census/** and
.github/workflows/ci-weekly-registry-census.yaml, while preserving the existing
weekly and dispatch behavior.
|
Corpus evidence after the L1 removal - the corpus job on run 31563501583 (stacked #15110 branch, which includes this tip): warm-cache restore, registry re-pin, ETag revalidation, and lockfile-keyed save in 1m42s, then feeding four matrix shards downstream. With the static idiom scan gone, this branch maintains the measurement input; #15110 is the measurement. |
…criteria (#15146) Follow-through on the maintainability audit: the weekly must not red (or hollow out) for reasons unrelated to custom nodes. Stacked on #15145 (-> #15110 -> #15081) so this diff is only the hardening. ## Plumbing (no measurement change) - **The runner is now typechecked** (`tsconfig` includes `scripts/registry-census/**/*.ts`; 6 real errors fixed, all inside the harness file). A frontend rename of `registerNodeDef`/`createReroute`/`graphToPrompt` now breaks the renaming PR loudly instead of red-ing the next weekly as a false ecosystem FAIL. Generated `src/__ecs_matrix__` stays excluded. - **Registry refresh survives the network**: curl `--retry 5`, falls back to the cached snapshot on failure, and refuses a snapshot that shrank >10% (API shape drift, not mass unpublishing). Previously one 5xx killed corpus -> shards -> verdict for the week. - **Corpus fetch refuses partial completeness**: curl retries; >5% failed fetches fails the run instead of green-shipping a shrunken denominator. - **Cache economics**: superseded ~1GB `registry-corpus-*` entries are pruned after each save (repo budget is 10GB shared; unpruned, the census would evict other workflows' caches within weeks). A Thursday cron keeps the cache inside GitHub's 7-day eviction window. - **`matrix-verdict` runs `if: always()`**: one failed shard degrades to a partial verdict, not a skipped check. - **build_matrix**: the rewrite regex required a non-empty relative prefix (it could hijack a pack's own `./scripts/utils.js` to OUR module - wrong-code-under-test); entry-cap truncation is now recorded in the manifest (the comment claimed it already was); missing corpus fails friendly. ## Measurement changes - **Real timers.** `setupFiles: []` was a mergeConfig-concatenation no-op, so all 1,879 packs executed under the unit suite's `vi.useFakeTimers()` by accident. Now assigned explicitly: `vitest.setup.ts` kept, fake timers dropped. Poison proof re-validated under real timers (14/14 channels). - **Criteria reworked against measured full-registry baselines** (1,879 packs, run 31563501583): | criterion | floor | baseline | |---|---|---| | entry JS loads (any entry per pack) | >= 95% | 97.4% | | **entry files load clean (new)** | >= 91% | 94.1% | | registerNodeDef OK | >= 99% | 100% | | every operation clean | >= 99% | 99.8% | | **entry-clean delta vs previous run (new)** | >= -1.5pp | metrics carried by actions cache | The old gated metric was only "pack had >=1 entry load" - 59-of-60 entries failing still counted as loaded. The delta gate catches gradual erosion (97.4 -> 95.9) that absolute floors ignore. - **Harness gate before any criterion**: the runner self-checks that the default workflow materialized (>=7 nodes, KSampler with widgets - the op-break poison proved a 0-node graph passes 15 if-guarded ops "clean"). If most packs fail self-check the verdict is **withheld as HARNESS FAILURE (exit 2)** - a degraded harness can no longer report a vacuously clean ecosystem or a false FAIL. ## Validated locally - `pnpm typecheck` green with the runner included - Poison corpus: `DETECTION PROOF PASSED` (14 checks incl. the two new self-check assertions), summarizer exit 1 naming three breaches - Real 1,879-row corpus: **PASS** exit 0, metrics.json written (anyPct 97.392 / entryPct 94.091 / worstOp load 99.84) - Simulated delta breach (prev entryPct 96.5): exit 1, `entry files clean dropped 2.4pp` - Simulated broad self-check failure: exit 2, verdict withheld Note: the fake-timer removal shifts measurement conditions; the first CI run of this PR re-baselines under real timers, and if entry-clean lands under the 91% floor the floor gets adjusted here before merge (that is what the review run is for). Deferred by design (discussed): shard autoscaling vs registry growth; registerNodeDef floor kept as a pure harness canary; V1 DEFS drift is now loud via the self-check. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
I have read and agree to the Contributor License Agreement |
christian-byrne
left a comment
There was a problem hiding this comment.
Superseded by #15175. Recommend closing this PR instead of rebasing it.
Full context for agent readers
Current main already contains the replacement corpus and ecosystem-matrix implementation from #15175. It preserves the same .census/corpus/registry_js contract while adding checked-in corpus pins, a readiness marker, population validation, required provenance artifacts, detection proofs, and focused tests.
This branch now conflicts with main in .gitignore and all four shared scripts/registry-census files. Rebasing it would replace the newer implementation with the older one, including its known fail-open partial-fetch behavior. The standalone weekly workflow is also absent from current main because the replacement owns the corpus lifecycle in ci-ecosystem-matrix.yaml.
No merge or auto-merge action taken.
|
Status check from the corpus pin work (FRONT-3698): @benjcooley, could you close it, or say what in here is still missing from |
Summary
Corpus infrastructure for full-registry custom-node measurement: a weekly-refreshed, cache-persisted mirror of every registry pack's frontend JS (~5,000 packs, ~1GB), ETag-revalidated so it self-updates exactly when a pack's default branch moves. This is the input for the ecosystem execution matrix (stacked: #15110).
The upstream static idiom scan (L1) is deliberately not vendored - its false-positive/negative rate was judged too high to act on. Packs are measured by executing their code (#15110) instead.
Changes
scripts/registry-census/- corpus tooling vendored fromComfy-Org/ComfyUI_ECS_Compat_Check(private, migration-phase instrument slated for archival; provenance in the README):refresh_registry.py(pin the registry snapshot - the target list),fetch_corpus.py(portable tar extraction,--revalidateper-pack tarball-ETag HEAD-check againstcorpus.lock.json, refetching only drifted packs),paths.py. Pure stdlib; all state under one gitignored, cacheable root (.census/)..github/workflows/ci-weekly-registry-census.yaml- weekly (Mon 06:17 UTC) +workflow_dispatch+ self-testing paths-filteredpull_request. Corpus restored from the actions cache, re-pinned, revalidated, re-saved under a lockfile-hash key (unchanged corpus = no duplicate save; drifted corpus = new entry).corpus.lock.jsonuploaded per run as the measurement's provenance record.Review Focus
steps.fetch.outcome == 'success' && hashFiles(lockfile) != ''- a failed fetch must not poison the cache, least of all under the empty-hash key.fetch_corpus.py: no bsdtar-only flags; extension filtering in Python with a staged swap.🤖 Generated with Claude Code