Repository navigation
ci: stop e2e blaming PRs for the base branch's specs and for cancellations - #15660
Conversation
pull_request.base.sha is fixed when the PR opens and does not follow the base branch, so a long-lived branch attributes every spec the base gained since then to itself. HEAD is refs/pull/N/merge, whose first parent is the base tip actually merged onto.
|
Important Approval pendingCodeRabbit has no unresolved comments, but it has not reviewed the latest commit. Use the checkbox below to review the latest commit. CodeRabbit will approve the changes if it finds no blocking issues.
Comment |
🎭 Playwright: ✅ 1889 passed, 0 failed · 2 flaky📊 Browser Reports
📦 Bundle: 9.66 MB gzip ⚪ 0 BDetailsSummary
Category Glance App Entry Points — 3.78 kB (baseline 3.78 kB) • ⚪ 0 BMain entry bundles and manifests Status: 1 unchanged Graph Workspace — 1.4 MB (baseline 1.4 MB) • ⚪ 0 BGraph editor runtime, canvas, workflow orchestration Status: 3 unchanged Views & Navigation — 124 kB (baseline 124 kB) • ⚪ 0 BTop-level views, pages, and routed surfaces Status: 17 unchanged Panels & Settings — 621 kB (baseline 621 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens Status: 27 unchanged User & Accounts — 27.5 kB (baseline 27.5 kB) • ⚪ 0 BAuthentication, profile, and account management bundles Status: 11 unchanged Editors & Dialogs — 126 kB (baseline 126 kB) • ⚪ 0 BModals, dialogs, drawers, and in-app editors Status: 8 unchanged UI Components — 115 kB (baseline 115 kB) • ⚪ 0 BReusable component library chunks Status: 15 unchanged Data & Services — 3.58 MB (baseline 3.58 MB) • ⚪ 0 BStores, services, APIs, and repositories Status: 17 unchanged Utilities & Hooks — 555 kB (baseline 555 kB) • ⚪ 0 BHelpers, composables, and utility bundles Status: 36 unchanged Vendor & Third-Party — 18.1 MB (baseline 18.1 MB) • ⚪ 0 BExternal libraries and shared vendor chunks Status: 18 unchanged Other — 16.3 MB (baseline 16.3 MB) • ⚪ 0 BBundles that do not match a named category Status: 294 unchanged ⚡ Performance Report
Show regressions
All metrics
Historical variance (last 15 runs)
Trend (last 15 commits on main)
Raw data{
"timestamp": "2026-09-01T08:24:06.744Z",
"gitSha": "e959bc020f45bdb29d123850229819c4965c6712",
"branch": "ci/new-spec-detection-base",
"measurements": [
{
"name": "canvas-idle",
"durationMs": 2120.342999999991,
"styleRecalcs": 10,
"styleRecalcDurationMs": 7.718000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 599.8840000000001,
"heapDeltaBytes": -1448968,
"heapUsedBytes": 67761296,
"domNodes": -277,
"jsHeapTotalBytes": 5496832,
"scriptDurationMs": 9.400000000000002,
"eventListeners": -189,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "canvas-idle",
"durationMs": 2055.676999999946,
"styleRecalcs": 10,
"styleRecalcDurationMs": 7.864000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 486.10300000000007,
"heapDeltaBytes": -8671204,
"heapUsedBytes": 61401848,
"domNodes": -278,
"jsHeapTotalBytes": 5496832,
"scriptDurationMs": 8.347,
"eventListeners": -155,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "canvas-mouse-sweep",
"durationMs": 1800.8479999999736,
"styleRecalcs": 75,
"styleRecalcDurationMs": 34.807,
"layouts": 14,
"layoutDurationMs": 4.261,
"taskDurationMs": 838.5169999999999,
"heapDeltaBytes": -7439304,
"heapUsedBytes": 62435140,
"domNodes": -281,
"jsHeapTotalBytes": 4972544,
"scriptDurationMs": 138.97400000000002,
"eventListeners": -155,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "canvas-mouse-sweep",
"durationMs": 1856.9599999999582,
"styleRecalcs": 78,
"styleRecalcDurationMs": 39.311,
"layouts": 14,
"layoutDurationMs": 4.680999999999999,
"taskDurationMs": 852.982,
"heapDeltaBytes": -6930548,
"heapUsedBytes": 63269000,
"domNodes": -279,
"jsHeapTotalBytes": 5496832,
"scriptDurationMs": 141.12199999999999,
"eventListeners": -155,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1667.6119999999628,
"styleRecalcs": 33,
"styleRecalcDurationMs": 16.786999999999995,
"layouts": 6,
"layoutDurationMs": 1.0700000000000003,
"taskDurationMs": 353.38000000000005,
"heapDeltaBytes": 4359348,
"heapUsedBytes": 74545640,
"domNodes": 80,
"jsHeapTotalBytes": 6029312,
"scriptDurationMs": 10.321,
"eventListeners": 19,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1723.1009999999287,
"styleRecalcs": 32,
"styleRecalcDurationMs": 16.308,
"layouts": 6,
"layoutDurationMs": 1.053,
"taskDurationMs": 351.371,
"heapDeltaBytes": 4404812,
"heapUsedBytes": 73877404,
"domNodes": 77,
"jsHeapTotalBytes": 4980736,
"scriptDurationMs": 11.075999999999999,
"eventListeners": 19,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "dom-widget-clipping",
"durationMs": 658.3590000000186,
"styleRecalcs": 11,
"styleRecalcDurationMs": 6.754,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 438.399,
"heapDeltaBytes": 18160936,
"heapUsedBytes": 88147496,
"domNodes": 18,
"jsHeapTotalBytes": 5242880,
"scriptDurationMs": 153.255,
"eventListeners": 2,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "dom-widget-clipping",
"durationMs": 631.1589999999114,
"styleRecalcs": 10,
"styleRecalcDurationMs": 6.107999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 432.02900000000005,
"heapDeltaBytes": 17800792,
"heapUsedBytes": 87661128,
"domNodes": 16,
"jsHeapTotalBytes": 5505024,
"scriptDurationMs": 150.848,
"eventListeners": 2,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "large-graph-idle",
"durationMs": 2019.5120000000202,
"styleRecalcs": 5,
"styleRecalcDurationMs": 4.447000000000003,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 557.762,
"heapDeltaBytes": 22158992,
"heapUsedBytes": 87044544,
"domNodes": -269,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 34.178,
"eventListeners": -155,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "large-graph-idle",
"durationMs": 2037.9840000000513,
"styleRecalcs": 5,
"styleRecalcDurationMs": 4.064000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 603.5180000000001,
"heapDeltaBytes": 10352824,
"heapUsedBytes": 85939644,
"domNodes": -271,
"jsHeapTotalBytes": -2179072,
"scriptDurationMs": 33.717999999999996,
"eventListeners": -183,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-pan",
"durationMs": 2210.3669999999624,
"styleRecalcs": 63,
"styleRecalcDurationMs": 10.884000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1551.1019999999999,
"heapDeltaBytes": 22266136,
"heapUsedBytes": 88070648,
"domNodes": 2,
"jsHeapTotalBytes": 225280,
"scriptDurationMs": 774.7120000000001,
"eventListeners": 6,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-pan",
"durationMs": 2171.3479999999663,
"styleRecalcs": 66,
"styleRecalcDurationMs": 12.248999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1575.1529999999998,
"heapDeltaBytes": 2136504,
"heapUsedBytes": 77156236,
"domNodes": -247,
"jsHeapTotalBytes": -380928,
"scriptDurationMs": 771.9449999999999,
"eventListeners": -185,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-zoom",
"durationMs": 3520.380999999986,
"styleRecalcs": 62,
"styleRecalcDurationMs": 15.681000000000001,
"layouts": 60,
"layoutDurationMs": 12.823999999999998,
"taskDurationMs": 1712.132,
"heapDeltaBytes": -2236148,
"heapUsedBytes": 71635028,
"domNodes": -254,
"jsHeapTotalBytes": -1130496,
"scriptDurationMs": 762.765,
"eventListeners": -179,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "large-graph-zoom",
"durationMs": 3427.287999999976,
"styleRecalcs": 60,
"styleRecalcDurationMs": 10.598000000000003,
"layouts": 60,
"layoutDurationMs": 12.925999999999998,
"taskDurationMs": 1627.149,
"heapDeltaBytes": 7697608,
"heapUsedBytes": 75310788,
"domNodes": 0,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 742.024,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "legacy-node-drag",
"durationMs": 3117.9610000000366,
"styleRecalcs": 40,
"styleRecalcDurationMs": 5.618999999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1911.5650000000003,
"heapDeltaBytes": 23433936,
"heapUsedBytes": 92926976,
"domNodes": 0,
"jsHeapTotalBytes": 679936,
"scriptDurationMs": 962.5039999999999,
"eventListeners": 188,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "legacy-node-drag",
"durationMs": 3070.194000000015,
"styleRecalcs": 40,
"styleRecalcDurationMs": 5.3629999999999995,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1854.002,
"heapDeltaBytes": 21642228,
"heapUsedBytes": 89267832,
"domNodes": 0,
"jsHeapTotalBytes": 417792,
"scriptDurationMs": 929.03,
"eventListeners": 188,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.699999999999818
},
{
"name": "minimap-idle",
"durationMs": 2004.8360000000116,
"styleRecalcs": 3,
"styleRecalcDurationMs": 2.309000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 536.8779999999998,
"heapDeltaBytes": 22133088,
"heapUsedBytes": 88128228,
"domNodes": -275,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 30.96,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "minimap-idle",
"durationMs": 2004.613000000063,
"styleRecalcs": 3,
"styleRecalcDurationMs": 2.2450000000000006,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 539.8779999999999,
"heapDeltaBytes": 21811700,
"heapUsedBytes": 87672968,
"domNodes": -273,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 29.932000000000006,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 572.7800000000229,
"styleRecalcs": 45,
"styleRecalcDurationMs": 8.908000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 391.806,
"heapDeltaBytes": 14057632,
"heapUsedBytes": 86105720,
"domNodes": 16,
"jsHeapTotalBytes": 5505024,
"scriptDurationMs": 137.126,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 585.6919999999946,
"styleRecalcs": 47,
"styleRecalcDurationMs": 10.281,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 389.49899999999997,
"heapDeltaBytes": 14064452,
"heapUsedBytes": 85644116,
"domNodes": 20,
"jsHeapTotalBytes": 5505024,
"scriptDurationMs": 135.39700000000002,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-idle",
"durationMs": 2020.3860000000304,
"styleRecalcs": 11,
"styleRecalcDurationMs": 8.297000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 481.626,
"heapDeltaBytes": -7658648,
"heapUsedBytes": 64280384,
"domNodes": -295,
"jsHeapTotalBytes": 5234688,
"scriptDurationMs": 7.459999999999998,
"eventListeners": -199,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-idle",
"durationMs": 2013.8820000000806,
"styleRecalcs": 9,
"styleRecalcDurationMs": 6.707000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 483.756,
"heapDeltaBytes": 3186852,
"heapUsedBytes": 75588536,
"domNodes": -296,
"jsHeapTotalBytes": 5758976,
"scriptDurationMs": 7.66,
"eventListeners": -231,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1715.6520000000341,
"styleRecalcs": 77,
"styleRecalcDurationMs": 33.052,
"layouts": 18,
"layoutDurationMs": 4.8999999999999995,
"taskDurationMs": 781.547,
"heapDeltaBytes": -10436684,
"heapUsedBytes": 61776328,
"domNodes": -297,
"jsHeapTotalBytes": 3923968,
"scriptDurationMs": 95.671,
"eventListeners": -201,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1735.433999999941,
"styleRecalcs": 77,
"styleRecalcDurationMs": 34.552,
"layouts": 18,
"layoutDurationMs": 4.915,
"taskDurationMs": 815.823,
"heapDeltaBytes": -12581904,
"heapUsedBytes": 59530916,
"domNodes": -297,
"jsHeapTotalBytes": 7069696,
"scriptDurationMs": 103.302,
"eventListeners": -201,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-transition-enter",
"durationMs": 1383.2689999999275,
"styleRecalcs": 18,
"styleRecalcDurationMs": 29.933999999999994,
"layouts": 14,
"layoutDurationMs": 14.033999999999997,
"taskDurationMs": 898.2209999999999,
"heapDeltaBytes": 26474776,
"heapUsedBytes": 99208100,
"domNodes": 13751,
"jsHeapTotalBytes": 10485760,
"scriptDurationMs": 27.567,
"eventListeners": 2381,
"totalBlockingTimeMs": 113,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "viewport-pan-sweep",
"durationMs": 8261.513999999977,
"styleRecalcs": 248,
"styleRecalcDurationMs": 39.324,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 5618.848,
"heapDeltaBytes": 15238868,
"heapUsedBytes": 90590812,
"domNodes": -268,
"jsHeapTotalBytes": -1429504,
"scriptDurationMs": 2589.3390000000004,
"eventListeners": -169,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "viewport-pan-sweep",
"durationMs": 8248.919999999998,
"styleRecalcs": 248,
"styleRecalcDurationMs": 39.196000000000005,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 5511.872,
"heapDeltaBytes": 13854080,
"heapUsedBytes": 89177048,
"domNodes": -268,
"jsHeapTotalBytes": -643072,
"scriptDurationMs": 2495.017,
"eventListeners": -169,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-idle",
"durationMs": 17766.530000000046,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 17034.412,
"heapDeltaBytes": -36779212,
"heapUsedBytes": 174110008,
"domNodes": -6548,
"jsHeapTotalBytes": 516096,
"scriptDurationMs": 315.32500000000005,
"eventListeners": -7471,
"totalBlockingTimeMs": 0,
"frameDurationMs": 17.776666666666642,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-idle",
"durationMs": 17515.415000000074,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 16815.822999999997,
"heapDeltaBytes": -34676752,
"heapUsedBytes": 175608728,
"domNodes": -6548,
"jsHeapTotalBytes": 4235264,
"scriptDurationMs": 306.94300000000004,
"eventListeners": -7475,
"totalBlockingTimeMs": 0,
"frameDurationMs": 17.780000000000047,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "vue-large-graph-pan",
"durationMs": 21780.309999999987,
"styleRecalcs": 175,
"styleRecalcDurationMs": 20.098000000000006,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 21386.132,
"heapDeltaBytes": -40507128,
"heapUsedBytes": 162437092,
"domNodes": -6548,
"jsHeapTotalBytes": 1040384,
"scriptDurationMs": 919.853,
"eventListeners": -7469,
"totalBlockingTimeMs": 215,
"frameDurationMs": 17.776666666666642,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-pan",
"durationMs": 21821.002000000022,
"styleRecalcs": 176,
"styleRecalcDurationMs": 19.603000000000037,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 21417.620999999996,
"heapDeltaBytes": -31728048,
"heapUsedBytes": 176476592,
"domNodes": -6548,
"jsHeapTotalBytes": -3805184,
"scriptDurationMs": 935.207,
"eventListeners": -7471,
"totalBlockingTimeMs": 153,
"frameDurationMs": 17.780000000000047,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "vue-node-resize-workload",
"durationMs": 6273.392000000002,
"styleRecalcs": 20,
"styleRecalcDurationMs": 74.882,
"layouts": 20,
"layoutDurationMs": 24.077,
"taskDurationMs": 6164.852,
"heapDeltaBytes": -18420180,
"heapUsedBytes": 184881432,
"domNodes": -6548,
"jsHeapTotalBytes": 10481664,
"scriptDurationMs": 116.81099999999999,
"eventListeners": -7471,
"totalBlockingTimeMs": 500,
"frameDurationMs": 17.779999999999987,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-node-resize-workload",
"durationMs": 6404.15900000005,
"styleRecalcs": 20,
"styleRecalcDurationMs": 72.68599999999998,
"layouts": 20,
"layoutDurationMs": 23.88499999999999,
"taskDurationMs": 6254.965999999999,
"heapDeltaBytes": -32560240,
"heapUsedBytes": 181098508,
"domNodes": -6548,
"jsHeapTotalBytes": 11624448,
"scriptDurationMs": 114.745,
"eventListeners": -7469,
"totalBlockingTimeMs": 415,
"frameDurationMs": 17.779999999999987,
"p95FrameDurationMs": 16.80000000000291
},
{
"name": "workflow-execution",
"durationMs": 111.60599999993792,
"styleRecalcs": 8,
"styleRecalcDurationMs": 13.761000000000001,
"layouts": 2,
"layoutDurationMs": 0.6659999999999999,
"taskDurationMs": 70.16399999999999,
"heapDeltaBytes": 3149900,
"heapUsedBytes": 71020440,
"domNodes": 110,
"jsHeapTotalBytes": 262144,
"scriptDurationMs": 5.0630000000000015,
"eventListeners": 49,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "workflow-execution",
"durationMs": 464.5989999999074,
"styleRecalcs": 10,
"styleRecalcDurationMs": 22.004,
"layouts": 3,
"layoutDurationMs": 0.9479999999999998,
"taskDurationMs": 104.523,
"heapDeltaBytes": 5045260,
"heapUsedBytes": 72650728,
"domNodes": 119,
"jsHeapTotalBytes": 0,
"scriptDurationMs": 7.001999999999998,
"eventListeners": 99,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000728
}
]
}🔬 E2E Coverage
Top 10 files by uncovered lines
|
A cancelled run lands in the failure branch, so e2e-status prints 'E2E failed' and a required check goes red for something that never ran. Still exits non-zero, since merging without e2e would be a false green. Also echoes the three job results, which the old message omitted.
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #15660 +/- ##
==========================================
+ Coverage 79.38% 82.98% +3.60%
==========================================
Files 2217 2059 -158
Lines 112241 118391 +6150
Branches 35106 36535 +1429
==========================================
+ Hits 89100 98251 +9151
+ Misses 22662 19784 -2878
+ Partials 479 356 -123
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
benjcooley
left a comment
There was a problem hiding this comment.
Root-cause analysis checks out. refs/pull/N/merge has the current base tip as parent 1, so base.sha is always an ancestor of it — three-dot degenerates to two-dot and the base's own additions get attributed to the PR. HEAD^1 is the right base. I re-ran the gate script under the runner's default shell (bash -eo pipefail) across all five outcomes and the exit codes are as claimed.
Two gaps below, both in the cancellation half. Neither blocks.
| Severity | Count |
|---|---|
| BLOCKER | 0 |
| SHOULD FIX | 2 |
| TRIVIAL | 2 |
|
@christian-byrne I saw @benjcooley give a review so I'm assigning you back to this PR, when you're done you can reassign @benjcooley |
Addresses review feedback: #15660 (comment)
Addresses review feedback: #15660 (comment)
Addresses review feedback: #15660 (comment)
Addresses review feedback: #15660 (comment)
|
@coderabbitai review |
|
benjcooley
left a comment
There was a problem hiding this comment.
Re-reviewed the four follow-up commits (3b609ed, c62b427, bc665cb, f7ac5c9). All four items are genuinely fixed, not papered over. Approving.
I re-ran the revised gate under the runner's real shell (bash --noprofile --norc -eo pipefail) across nine outcomes, and built git fixtures for the range changes rather than reasoning about them:
| scenario | result |
|---|---|
| all green | E2E passed, exit 0 |
| video skipped (push/merge_group) | E2E passed, exit 0 |
| intentional path skip | E2E skipped, exit 0 |
| real failure | E2E failed, exit 1 |
| pure cancellation | E2E cancelled, not failed, exit 1 |
| mixed failure + cancellation | E2E failed, exit 1 |
| changes cancelled | changes gate did not complete (cancelled), exit 1 |
| changes failed | changes gate did not complete (failure), exit 1 |
| setup failed (all skipped) | E2E failed, exit 1 |
The mixed row is the one that mattered — it now reports the failure instead of sending someone to burn a rerun.
On the fail-closed gate: I checked changes-filter's should-run, which is ${{ github.event_name != 'pull_request' || ... }}. That expression can only ever produce the literal "true" or "false", never empty, so gating on needs.changes.result == 'success' fully closes the false-green hole rather than narrowing it. Good.
On the diff range: I confirmed with a fixture that the round-1 two-dot fallback really did misreport — a spec file the base deleted after the branch point came back as A alongside the PR's own file. The restored three-dot form returns only the PR's file. Also verified git diff "A..B" and git diff "A...B" as single arguments behave identically to the two-argument form, and that git rev-parse --verify --quiet HEAD^2 exits 1 silently on a non-merge HEAD, so the else branch is reached cleanly.
One non-blocking note for later, not this PR: ... | grep '\.spec\.ts$' || true on the detect line still makes "git diff failed" indistinguishable from "no new specs" — if the fallback ever runs against a sha that isn't in the local object store, the video job silently no-ops. That swallow predates this PR and the failure mode is only a missing video, so it isn't worth holding the merge. Splitting the diff from the grep would close it whenever someone next touches that step.
|
Follow-up carrier for the non-blocking |
…llowing it (#16705) Follow-up to [#15660](#15660). Implements the non-blocking residual noted in [that PR's approval review](#15660 (review)): > `... | grep '\.spec\.ts$' || true` on the detect line still makes "git diff failed" indistinguishable from "no new specs" — if the fallback ever runs against a sha that isn't in the local object store, the video job silently no-ops. That swallow predates this PR and the failure mode is only a missing video, so it isn't worth holding the merge. Splitting the diff from the grep would close it whenever someone next touches that step. ## Change In `playwright-video-new-tests`'s detect step, `git diff --name-only --diff-filter=A "$DIFF_RANGE" ... | grep '\.spec\.ts$' || true` piped `git diff` straight into `grep`, with `|| true` guarding the whole pipeline. Under the runner's `bash -eo pipefail`, that `|| true` swallows a `git diff` failure identically to "grep matched nothing" — so if `$DIFF_RANGE` ever references a sha that isn't in the local object store (e.g. a shallow fetch, or a base sha that's been GC'd), the step reports zero new specs instead of erroring, and the video job silently no-ops. Split into two statements: `ADDED_FILES=$(git diff ...)` then `NEW_FILES=$(grep ... <<< "$ADDED_FILES" || true)`. Now a `git diff` failure trips its own exit code under `-e` and aborts the step with a visible error; only `grep`'s "no match" case is swallowed, which is the actually-intended no-new-specs path. ## Evidence Fixture-verified under the runner's real shell (`bash --noprofile --norc -eo pipefail`), same style as the source PR's review: | scenario | before (piped, `\|\| true`) | after (split) | | --- | --- | --- | | `git diff` fails | swallowed — reads as `has-new-tests=false` | step aborts (exit 1), error surfaced | | no new specs, diff succeeds | `has-new-tests=false` | `has-new-tests=false` (unchanged) | | new spec found | `has-new-tests=true` | `has-new-tests=true` (unchanged) | | empty diff | `has-new-tests=false` | `has-new-tests=false` (unchanged) | Commands run: ``` bash --noprofile --norc -eo pipefail -c '... git diff | grep ... || true ...' # reproduces the swallow bash --noprofile --norc -eo pipefail -c '... ADDED_FILES=$(false); NEW_FILES=$(grep <<< "$ADDED_FILES" || true) ...' # confirms split form aborts # + no-spec / spec-found / empty-diff cases re-verified against the split form actionlint .github/workflows/ci-tests-e2e.yaml # no new findings (existing SC2016/SC2086 info-level notes elsewhere, untouched by this diff) python3 -c "import yaml; yaml.safe_load(open('.github/workflows/ci-tests-e2e.yaml'))" # YAML valid ``` No behavior change on any of the five outcomes #15660 already verified (all-green, video-skipped, real failure, cancelled, should-run false) — this only changes what happens when the diff command itself errors, which was previously unreachable/silent. <!-- authored-by:agent lane-act-fe-15660-followup-8f074af2 --> --------- Co-authored-by: Amp <amp@ampcode.com>
…ellations (Comfy-Org#16682) ## Problem Merge-group runs of the required `website-e2e` check were exceeding the 15-minute job timeout, silently dequeuing approved PRs: - [Comfy-Org#15753 group 18:13Z](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/33665855796): cancelled - [Comfy-Org#15660 group 18:28Z](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/33667445294): cancelled after 15m04s during Playwright - [Passing control run](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/33669067839): 407 tests took 8.2m, with a 9m47s total job time On 4-core hosted runners, Playwright used two workers. Test retries and runner variation left too little margin for checkout, install, build, reporting, and deployment. ## Fix Shard the Playwright run four ways instead of raising the timeout or weakening coverage: - `website-e2e-shard` builds and runs one quarter of the suite, uploading a blob report. - `website-e2e-report` requires all four reports, merges HTML and JSON with Playwright 1.61.1, deploys the report, and distinguishes test failures from infrastructure/reporting failures. - `website-e2e` remains the stable required-check context and passes only when the complete report succeeds. - Focused script tests cover complete success, test failure, incomplete delivery, download failure, report-merge failure, and gate behavior. The shape follows the sharded matrix and merge-report pattern in `ci-tests-e2e.yaml`. ## Evidence At head `90a3bd08d6`, [run 33936522288, attempt 2](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/33936522288) ran all four shards, downloaded all reports, merged and deployed the report, and passed the stable `website-e2e` gate. The branch has since been updated with `main` and hardened further; current-head checks are the authoritative merge gate. Local verification on the current branch: - `pnpm test:unit scripts/cicd/website-e2e-workflow.test.ts` — 6 passed - `pnpm typecheck:scripts` — passed - `pnpm typecheck:website` — 0 errors - Full pre-push checks, including format, lint, typecheck, and knip — passed --------- Co-authored-by: Amp <amp@ampcode.com>
Two defects in
ci-tests-e2e.yamlthat both made this workflow blame a PR for something it did not do. Same file, so they are one PR.1. New-spec detection uses a base that never moves
playwright-video-new-testsruns the spec files a PR adds. On a branch whose base is stale it instead runs specs other people merged, and fails on them.github.event.pull_request.base.shais recorded when the PR opens and does not advance with the base branch.actions/checkoutleavesHEADatrefs/pull/N/merge, which already contains everything the base gained since. Three-dot diffing the merge commit against that frozen SHA therefore reports the base's own additions as added by the PR.Measured on #14073, using the exact merge commit from its run (
a580416262, logged asMerge 957bfb36d2 into b2d6e414df), branch point 2026-07-23:git diff --diff-filter=A $BASE_SHA...HEAD(before)git diff --diff-filter=A HEAD^1 HEAD(after)22 matches that run's
NEW_SPEC_FILESlist exactly, and 0 is correct — 14073 modifiespreviewAsText.spec.tsand adds no spec. The job failed onsubgraphResizePreservation.spec.ts, which came from #14461. Rebasing "fixed" it, which is why this reads as flakiness rather than a bug. It gets worse the longer a branch lives.2. A cancelled run is reported as a failed one
e2e-statusfunnelscancelledinto the same branch asfailureand printsE2E failed. Sincee2e-statusis a required check, a cancelled run leaves a red required check with a message asserting the tests failed. On 14073 this cost real time chasing a failure that never happened — run31986467016was cancelled 50s in, and the PR looked broken.Still exits non-zero: merging without e2e having run would be a false green. Only the diagnosis changes, plus the three job results are now echoed — the old message named none of them, so you could not tell which leg failed without opening the run.
upload-e2e-coveragealready special-casescancelled, so the distinction exists in this file; the status gate just did not make it.Verified across all five outcomes (all-green, video-skipped, real failure, cancelled, should-run false) — exit codes unchanged except that cancelled now says so.