Repository navigation
test: tolerate sub-pixel drift in the mobile touch-pan screenshot - #15354
christian-byrne wants to merge 1 commit into
Conversation
`@mobile Can pan with touch` asserts an exact canvas screenshot. On 2026-08-17 it failed at `40 pixels (ratio 0.01 of all image pixels)` on two merge-queue branches in a row, ejecting both PRs: - #14068, which adds CODEOWNERS entries - #15067, which changes run acknowledgement Neither can move a canvas. A third PR, #15108, passed the same assertion on the same base commit (926d78c), which is what rules out a regression on `main` and leaves flake. `maxDiffPixels: 50` matches the sibling canvas assertion in `interaction.spec.ts:234`, which carries the same tolerance for the same reason. It is deliberately just above the observed 40 and far below anything that would hide a real visual change. The cost of leaving it strict is not a red check on the PR that caused it — the assertion passes on the PR branch and only fails once the queue rebuilds it against `main`, so it ejects PRs silently and blames whichever change happens to be in the queue.
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 3 minutes Limit details: You’ve used all 4 included reviews currently available under your plan. You completed 68 included PR reviews in the past 7 days; at that activity level, included reviews refill at 4 reviews per hour. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Comment |
🎭 Playwright: ✅ 1820 passed, 0 failed · 2 flaky📊 Browser Reports
📦 Bundle: 8.85 MB gzip 🔴 +138 BDetailsSummary
Category Glance App Entry Points — 3.71 kB (baseline 3.71 kB) • ⚪ 0 BMain entry bundles and manifests
Status: 1 added / 1 removed Graph Workspace — 1.37 MB (baseline 1.37 MB) • 🔴 +875 BGraph editor runtime, canvas, workflow orchestration
Status: 1 added / 1 removed / 2 unchanged Views & Navigation — 124 kB (baseline 124 kB) • ⚪ 0 BTop-level views, pages, and routed surfaces Status: 17 unchanged Panels & Settings — 566 kB (baseline 566 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens Status: 26 unchanged User & Accounts — 27.7 kB (baseline 27.7 kB) • ⚪ 0 BAuthentication, profile, and account management bundles Status: 11 unchanged Editors & Dialogs — 125 kB (baseline 125 kB) • ⚪ 0 BModals, dialogs, drawers, and in-app editors Status: 8 unchanged UI Components — 67.1 kB (baseline 67.1 kB) • ⚪ 0 BReusable component library chunks Status: 14 unchanged Data & Services — 3.52 MB (baseline 3.52 MB) • ⚪ 0 BStores, services, APIs, and repositories Status: 17 unchanged Utilities & Hooks — 550 kB (baseline 550 kB) • ⚪ 0 BHelpers, composables, and utility bundles Status: 38 unchanged Vendor & Third-Party — 16.8 MB (baseline 16.8 MB) • ⚪ 0 BExternal libraries and shared vendor chunks Status: 18 unchanged Other — 14.2 MB (baseline 14.2 MB) • ⚪ 0 BBundles that do not match a named category
Status: 1 added / 1 removed / 284 unchanged ⚡ Performance Report
Show regressions
All metrics
Historical variance (last 15 runs)
Trend (last 15 commits on main)
Raw data{
"timestamp": "2026-08-17T03:02:49.096Z",
"gitSha": "7fe7b4b38889cb11ba82c955c5968a83c503bc04",
"branch": "fix/flaky-mobile-pan-screenshot",
"measurements": [
{
"name": "canvas-idle",
"durationMs": 2034.0040000000101,
"styleRecalcs": 9,
"styleRecalcDurationMs": 7.988,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 453.512,
"heapDeltaBytes": 4900148,
"heapUsedBytes": 69406304,
"domNodes": 18,
"jsHeapTotalBytes": 24903680,
"scriptDurationMs": 6.591999999999999,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.699999999999818
},
{
"name": "canvas-idle",
"durationMs": 2038.063999999963,
"styleRecalcs": 10,
"styleRecalcDurationMs": 8.509,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 456.97999999999996,
"heapDeltaBytes": 4769104,
"heapUsedBytes": 69381404,
"domNodes": 20,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 6.255999999999999,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "canvas-mouse-sweep",
"durationMs": 2008.340000000004,
"styleRecalcs": 79,
"styleRecalcDurationMs": 42.713,
"layouts": 12,
"layoutDurationMs": 3.7809999999999997,
"taskDurationMs": 1007.3809999999999,
"heapDeltaBytes": -17615312,
"heapUsedBytes": 46607568,
"domNodes": 6,
"jsHeapTotalBytes": 23826432,
"scriptDurationMs": 118.333,
"eventListeners": -153,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "canvas-mouse-sweep",
"durationMs": 1803.651000000002,
"styleRecalcs": 74,
"styleRecalcDurationMs": 37.641,
"layouts": 12,
"layoutDurationMs": 3.9880000000000004,
"taskDurationMs": 817.3000000000001,
"heapDeltaBytes": -664324,
"heapUsedBytes": 63995844,
"domNodes": 56,
"jsHeapTotalBytes": 25690112,
"scriptDurationMs": 109.307,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1723.2790000000477,
"styleRecalcs": 33,
"styleRecalcDurationMs": 15.571,
"layouts": 6,
"layoutDurationMs": 0.531,
"taskDurationMs": 360.205,
"heapDeltaBytes": 7995872,
"heapUsedBytes": 72493948,
"domNodes": 78,
"jsHeapTotalBytes": 25427968,
"scriptDurationMs": 8.209000000000001,
"eventListeners": 19,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.670000000000012,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "canvas-zoom-sweep",
"durationMs": 1739.496000000031,
"styleRecalcs": 33,
"styleRecalcDurationMs": 16.132,
"layouts": 6,
"layoutDurationMs": 0.576,
"taskDurationMs": 365.52799999999996,
"heapDeltaBytes": 8007936,
"heapUsedBytes": 72239100,
"domNodes": 78,
"jsHeapTotalBytes": 25427968,
"scriptDurationMs": 8.479,
"eventListeners": 19,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "dom-widget-clipping",
"durationMs": 607.3210000000131,
"styleRecalcs": 12,
"styleRecalcDurationMs": 7.914000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 392.385,
"heapDeltaBytes": -12212580,
"heapUsedBytes": 52188344,
"domNodes": 20,
"jsHeapTotalBytes": 26214400,
"scriptDurationMs": 56.766,
"eventListeners": 2,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "dom-widget-clipping",
"durationMs": 577.6319999999942,
"styleRecalcs": 13,
"styleRecalcDurationMs": 9.332,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 376.794,
"heapDeltaBytes": -11211352,
"heapUsedBytes": 53502928,
"domNodes": 22,
"jsHeapTotalBytes": 24641536,
"scriptDurationMs": 57.263,
"eventListeners": 2,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-idle",
"durationMs": 2022.5439999999821,
"styleRecalcs": 9,
"styleRecalcDurationMs": 7.363000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 595.256,
"heapDeltaBytes": 12306032,
"heapUsedBytes": 72112420,
"domNodes": -281,
"jsHeapTotalBytes": 3244032,
"scriptDurationMs": 13.165000000000003,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "large-graph-idle",
"durationMs": 2042.739000000097,
"styleRecalcs": 8,
"styleRecalcDurationMs": 6.881000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 581.99,
"heapDeltaBytes": 12931872,
"heapUsedBytes": 73149060,
"domNodes": -281,
"jsHeapTotalBytes": 2195456,
"scriptDurationMs": 11.467999999999998,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "large-graph-pan",
"durationMs": 2135.4619999999613,
"styleRecalcs": 68,
"styleRecalcDurationMs": 14.026,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1173.4740000000002,
"heapDeltaBytes": 7535668,
"heapUsedBytes": 68318804,
"domNodes": -282,
"jsHeapTotalBytes": 3436544,
"scriptDurationMs": 337.30100000000004,
"eventListeners": -147,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "large-graph-pan",
"durationMs": 2175.371999999925,
"styleRecalcs": 69,
"styleRecalcDurationMs": 15.277,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 1205.575,
"heapDeltaBytes": 590696,
"heapUsedBytes": 61153672,
"domNodes": -287,
"jsHeapTotalBytes": 4485120,
"scriptDurationMs": 345.5,
"eventListeners": -181,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "large-graph-zoom",
"durationMs": 3128.9540000000216,
"styleRecalcs": 64,
"styleRecalcDurationMs": 13.692000000000002,
"layouts": 60,
"layoutDurationMs": 7.544,
"taskDurationMs": 1308.234,
"heapDeltaBytes": 17320512,
"heapUsedBytes": 79329148,
"domNodes": 10,
"jsHeapTotalBytes": 5242880,
"scriptDurationMs": 390.89000000000004,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "large-graph-zoom",
"durationMs": 3164.718999999991,
"styleRecalcs": 65,
"styleRecalcDurationMs": 15.202,
"layouts": 60,
"layoutDurationMs": 7.664000000000001,
"taskDurationMs": 1300.47,
"heapDeltaBytes": 17351012,
"heapUsedBytes": 79255284,
"domNodes": 12,
"jsHeapTotalBytes": 5242880,
"scriptDurationMs": 390.006,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "minimap-idle",
"durationMs": 2010.6549999999856,
"styleRecalcs": 7,
"styleRecalcDurationMs": 6.142000000000001,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 588.1070000000001,
"heapDeltaBytes": 12219024,
"heapUsedBytes": 72853592,
"domNodes": -286,
"jsHeapTotalBytes": 2457600,
"scriptDurationMs": 12.626999999999999,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "minimap-idle",
"durationMs": 2016.6430000000446,
"styleRecalcs": 7,
"styleRecalcDurationMs": 6.388000000000002,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 575.669,
"heapDeltaBytes": 12904836,
"heapUsedBytes": 73872484,
"domNodes": -286,
"jsHeapTotalBytes": 3244032,
"scriptDurationMs": 12.809000000000001,
"eventListeners": -149,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 633.1870000000208,
"styleRecalcs": 47,
"styleRecalcDurationMs": 11.326,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 425.01599999999996,
"heapDeltaBytes": -10424944,
"heapUsedBytes": 54153468,
"domNodes": 20,
"jsHeapTotalBytes": 26738688,
"scriptDurationMs": 122.163,
"eventListeners": 8,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-dom-widget-clipping",
"durationMs": 622.3410000000058,
"styleRecalcs": 46,
"styleRecalcDurationMs": 10.3,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 422.7,
"heapDeltaBytes": -10352656,
"heapUsedBytes": 54126064,
"domNodes": 18,
"jsHeapTotalBytes": 25690112,
"scriptDurationMs": 126.62399999999998,
"eventListeners": 6,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333335,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "subgraph-idle",
"durationMs": 2005.0820000000158,
"styleRecalcs": 10,
"styleRecalcDurationMs": 8.256999999999998,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 440.0810000000001,
"heapDeltaBytes": 4868640,
"heapUsedBytes": 69666072,
"domNodes": 20,
"jsHeapTotalBytes": 25165824,
"scriptDurationMs": 5.789000000000001,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-idle",
"durationMs": 2004.946000000018,
"styleRecalcs": 9,
"styleRecalcDurationMs": 7.2799999999999985,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 435.93100000000004,
"heapDeltaBytes": 5017844,
"heapUsedBytes": 69612812,
"domNodes": 18,
"jsHeapTotalBytes": 25165824,
"scriptDurationMs": 5.456,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.800000000000182
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1673.5780000000204,
"styleRecalcs": 76,
"styleRecalcDurationMs": 35.005,
"layouts": 16,
"layoutDurationMs": 4.404,
"taskDurationMs": 751.009,
"heapDeltaBytes": -4061552,
"heapUsedBytes": 60583192,
"domNodes": 61,
"jsHeapTotalBytes": 25165824,
"scriptDurationMs": 83.80399999999999,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.699999999999818
},
{
"name": "subgraph-mouse-sweep",
"durationMs": 1677.3640000000114,
"styleRecalcs": 76,
"styleRecalcDurationMs": 35.598,
"layouts": 16,
"layoutDurationMs": 4.096000000000001,
"taskDurationMs": 742.433,
"heapDeltaBytes": -4318832,
"heapUsedBytes": 60197432,
"domNodes": 64,
"jsHeapTotalBytes": 25690112,
"scriptDurationMs": 83.87400000000001,
"eventListeners": 4,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.699999999999818
},
{
"name": "subgraph-transition-enter",
"durationMs": 1401.4329999999973,
"styleRecalcs": 21,
"styleRecalcDurationMs": 30.017000000000003,
"layouts": 16,
"layoutDurationMs": 13.158999999999997,
"taskDurationMs": 893.3019999999998,
"heapDeltaBytes": -6127688,
"heapUsedBytes": 81176356,
"domNodes": 13673,
"jsHeapTotalBytes": 11272192,
"scriptDurationMs": 16.523999999999997,
"eventListeners": 2375,
"totalBlockingTimeMs": 133,
"frameDurationMs": 16.666666666666636,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "viewport-pan-sweep",
"durationMs": 8229.083000000002,
"styleRecalcs": 250,
"styleRecalcDurationMs": 40.422,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 4122.63,
"heapDeltaBytes": -5062884,
"heapUsedBytes": 55576100,
"domNodes": -281,
"jsHeapTotalBytes": 4816896,
"scriptDurationMs": 1051.6760000000002,
"eventListeners": -165,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "viewport-pan-sweep",
"durationMs": 8264.598999999977,
"styleRecalcs": 249,
"styleRecalcDurationMs": 38.648,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 4293.14,
"heapDeltaBytes": -3073796,
"heapUsedBytes": 57673136,
"domNodes": -282,
"jsHeapTotalBytes": 3768320,
"scriptDurationMs": 1181.1660000000002,
"eventListeners": -165,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.666666666666668,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-idle",
"durationMs": 16928.09500000004,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 15917.378999999999,
"heapDeltaBytes": -43122568,
"heapUsedBytes": 167496320,
"domNodes": -8312,
"jsHeapTotalBytes": -11739136,
"scriptDurationMs": 114.313,
"eventListeners": -16389,
"totalBlockingTimeMs": 0,
"frameDurationMs": 17.780000000000047,
"p95FrameDurationMs": 16.80000000000291
},
{
"name": "vue-large-graph-idle",
"durationMs": 16702.481000000036,
"styleRecalcs": 0,
"styleRecalcDurationMs": 0,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 15715.919,
"heapDeltaBytes": -44203432,
"heapUsedBytes": 166868076,
"domNodes": -8312,
"jsHeapTotalBytes": -11177984,
"scriptDurationMs": 108.909,
"eventListeners": -16389,
"totalBlockingTimeMs": 0,
"frameDurationMs": 17.776666666666763,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "vue-large-graph-pan",
"durationMs": 21588.424999999974,
"styleRecalcs": 176,
"styleRecalcDurationMs": 19.833999999999964,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 21005.051000000003,
"heapDeltaBytes": -27923788,
"heapUsedBytes": 169201916,
"domNodes": -8312,
"jsHeapTotalBytes": -10231808,
"scriptDurationMs": 444.251,
"eventListeners": -16381,
"totalBlockingTimeMs": 106,
"frameDurationMs": 17.776666666666642,
"p95FrameDurationMs": 16.80000000000291
},
{
"name": "vue-large-graph-pan",
"durationMs": 20923.857999999997,
"styleRecalcs": 170,
"styleRecalcDurationMs": 17.58599999999999,
"layouts": 0,
"layoutDurationMs": 0,
"taskDurationMs": 20280.805999999997,
"heapDeltaBytes": -27797756,
"heapUsedBytes": 182740940,
"domNodes": -8316,
"jsHeapTotalBytes": -12595200,
"scriptDurationMs": 416.16200000000003,
"eventListeners": -16383,
"totalBlockingTimeMs": 40,
"frameDurationMs": 17.779999999999927,
"p95FrameDurationMs": 16.799999999999272
},
{
"name": "workflow-execution",
"durationMs": 107.36099999996895,
"styleRecalcs": 6,
"styleRecalcDurationMs": 13.690000000000001,
"layouts": 3,
"layoutDurationMs": 0.707,
"taskDurationMs": 77.548,
"heapDeltaBytes": 3109380,
"heapUsedBytes": 66720832,
"domNodes": 109,
"jsHeapTotalBytes": 2883584,
"scriptDurationMs": 5.804,
"eventListeners": 49,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.66333333333332,
"p95FrameDurationMs": 16.700000000000728
},
{
"name": "workflow-execution",
"durationMs": 110.37099999998645,
"styleRecalcs": 7,
"styleRecalcDurationMs": 13.623,
"layouts": 2,
"layoutDurationMs": 0.6330000000000001,
"taskDurationMs": 78.896,
"heapDeltaBytes": 3037816,
"heapUsedBytes": 66517072,
"domNodes": 108,
"jsHeapTotalBytes": 3145728,
"scriptDurationMs": 4.486000000000001,
"eventListeners": 49,
"totalBlockingTimeMs": 0,
"frameDurationMs": 16.663333333333338,
"p95FrameDurationMs": 16.799999999999727
}
]
} |
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #15354 +/- ##
==========================================
+ Coverage 79.06% 81.40% +2.34%
==========================================
Files 2209 1882 -327
Lines 122868 117972 -4896
Branches 36466 35788 -678
==========================================
- Hits 97140 96035 -1105
+ Misses 25161 21470 -3691
+ Partials 567 467 -100
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
|
Superseded by #15356, closing this one so there is a single PR to review. Same defect, same conclusion that a bound is the honest fix, but that PR pins the cause rather than inferring it. It pulled the artifacts from two failing runs and found the actual and diff PNGs byte-identical across different runners, then localised the 40 pixels: they are anti-aliasing inside the glyphs of the It also adds what this PR does not: a poll asserting the pan actually applied before any pixels are compared, so a pan that silently never happens fails with a readable error instead of masquerading as a screenshot diff — which a tolerance alone would hide. The merge-queue evidence here is the better impact story, so I have carried it over to 15356: this flake does not just redden PR checks, it ejects unrelated PRs from the merge queue. |
## The bug
`.github/workflows/ci-binary-size-validation.yaml` subscribes to the
`labeled` and `unlabeled`
pull_request event types:
```yaml
on:
pull_request:
types: [opened, synchronize, reopened, labeled, unlabeled]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
```
So **any** label write starts a second binary-size run on the identical
commit, and
`cancel-in-progress` kills the one already in flight. A label change
cannot alter a bundle's
size, so there is nothing for the job to re-evaluate — the second run
just recomputes the same
answer and leaves a permanent `cancelled` check run behind on the
commit.
## Why it matters beyond wasted minutes
The cancelled check does not go away when the retry succeeds. Both check
runs stay attached to
the commit, so the commit's check rollup is `FAILURE` forever.
The PR risk grader reads that rollup. Its reversibility axis treats a
non-`SUCCESS` rollup as
`no_green_checks_tier`, which `.github/risk.json` sets to **R2** — so a
PR gets bumped to R2
because a label bot wrote a `size:*` label two seconds after CI started.
The chain, end to end:
1. `dosubot[bot]` writes a `size:*` label seconds after the PR opens or
is pushed to.
2. That `labeled` event re-triggers binary-size on the same commit;
`cancel-in-progress` cancels
the in-flight run.
3. The commit now permanently carries a `binary-size / CANCELLED` check
run.
4. The grader counts `CANCELLED` as a failing check, so the rollup is
not `SUCCESS`, so
reversibility is pinned at R2.
## Measured (re-verified for this PR, 2026-08-18)
| measure | value |
|---|---|
| binary-size runs, 30 days (created >= 2026-07-19) | 4,307 |
| ...concluded `cancelled` | **1,128 = 26.2%** |
| ...concluded `failure` | 39 = 0.9% |
| complete 2-day window (created >= 2026-08-16) | 819 runs |
| ...cancelled | 164 = 20.0% |
| cancels superseded by a later run on the **same head SHA** | **163 of
164 = 99.4%** |
| median gap to the superseding run | **2 s** (p25 0 s, p75 4 s) |
| distinct head SHAs in that window carrying **both** a `cancelled` and
a `success` run | 110 of 613 = **17.9%** |
Cancellations outrun real findings by roughly 29 to 1.
**Timed to the second on three live PRs** (label event → new run → prior
run cancelled):
- Comfy-Org#15413 — `size:XS` at 21:31:18Z → new run 21:31:20Z, prior run
(started 21:31:17Z) cancelled
- Comfy-Org#15414 — `size:L` at 21:35:47Z → new run 21:35:49Z, prior run
(21:35:45Z) cancelled
- Comfy-Org#15412 — `size:L` at 21:02:03Z → new run 21:02:06Z, prior run
(21:02:01Z) cancelled
**A settled rollup showing the consequence.** Commit `ce9ec9b4d0`
(Comfy-Org#15414, a `browser_tests/`-only
PR) — rollup state `FAILURE` over 85 contexts: 57 `SUCCESS`, 26
`SKIPPED`, and exactly one
failing-class context, `binary-size / CANCELLED`. That PR carries
`risk:R2`.
**Across the whole label population:** of the 112 PRs currently carrying
`risk:R2`, **17 (15.2%)
have a check rollup whose only failing-class context is `binary-size /
CANCELLED`** — Comfy-Org#14546,
Comfy-Org#15138, Comfy-Org#15173, Comfy-Org#15178, Comfy-Org#15261, Comfy-Org#15280, Comfy-Org#15307, Comfy-Org#15354, Comfy-Org#15371, Comfy-Org#15376,
Comfy-Org#15380, Comfy-Org#15388, Comfy-Org#15402,
Comfy-Org#15404, Comfy-Org#15410, Comfy-Org#15411, Comfy-Org#15414.
## Why removing the trigger is the right fix, and not the concurrency
group
Keying the concurrency group on `github.event.pull_request.head.sha`
does **not** help: a label
write happens on the identical commit, so it lands in the same group and
still cancels. The
re-trigger itself is what has to go.
`binary-size` is **not** a required check for merge — the `ProtectMain`
ruleset requires `test`,
`lint-and-format`, `e2e-status` and `website-e2e`, and `CLA Check`
requires `cla-assistant`.
## Trade-off, stated plainly
The `allow-large-binaries` bypass is read from
`github.event.pull_request.labels.*.name` at the
job `if:`. With `labeled` removed, applying that label no longer
re-evaluates the check
instantly — it takes effect on the next `synchronize`/`reopened`. That
costs nothing today:
**`allow-large-binaries` has been applied to 0 PRs, ever**, and the
check does not gate merge, so
a stale red never blocks anyone.
`scripts/cicd/check-binary-size.sh:165` prints "add the
'allow-large-binaries' label to the PR"
on failure; that advice still works, it just lands on the next push
rather than instantly. I have
deliberately not touched that file — happy to reword it in a follow-up
if a reviewer wants it.
## This PR demonstrates the fix on itself
`pull_request` workflow definitions are read from the PR head, so the
change is already live
here. Timeline:
- 22:50:13Z — PR opened
- 22:50:17Z — binary-size run `32194526247` starts
- 22:50:19Z — `dosubot[bot]` applies `size:XS`, **two seconds into the
run** — the exact pattern
that cancels the run on every other PR
Result: **one** binary-size run on this commit, no second trigger, no
cancellation,
`binary-size pass 35s`. Compare Comfy-Org#15413 / Comfy-Org#15414 / Comfy-Org#15412 above, where
the same two-second label
write killed the in-flight run and left a permanent `CANCELLED` on the
commit.
## Scope
This is half the loop. The other half is grader-side: `CANCELLED` is
counted as a *failing*
check in `Comfy-Org/github-workflows`
(`scripts/pr-risk/grade-pr-risk.sh`), which is what turns a
cancelled run into an R2. That fix has **not shipped** and is not in
this PR. So this PR removes
the largest source of the cancellations but does not fully close the
loop — a binary-size run
cancelled for any other reason would still red the rollup.
No other change to the file.
Summary
@mobile Can pan with touch(pan.spec.ts:212) asserts an exact canvas screenshot with no tolerance. It flakes, and because it only flakes once the merge queue rebuilds a PR againstmain, it ejects PRs silently.Evidence
Three merge-queue branches on 2026-08-17, all built on the same base commit
926d78c8bf:CI: Tests E2Epr-14068-926d78c8bf40 pixels (ratio 0.01)pr-15067-fb633f46b340 pixels (ratio 0.01)pr-15108-926d78c8bfNeither failing PR can move a canvas. The third passing on the same base is what rules out a regression on
mainand leaves flake. Both failures reported the identical40 pixels, and both PRs were removed from the queue bygithub-merge-queue[bot]with no reason recorded.The change
maxDiffPixels: 50is the same tolerance, for the same reason, as the sibling canvas assertion atinteraction.spec.ts:234("Litegraph renders edge with a slight offset"). Other canvas-heavy specs already carry tolerances —canvasSettings.spec.tsusesmaxDiffPixels: 100,load3d.spec.tsusesmaxDiffPixelRatioup to0.1.50 is deliberately just above the observed 40 and far below anything that would hide a real visual change: the canvas is 1081×1999, so this is 0.0019% of the image.
Why this is worth fixing rather than retrying
The assertion passes on the PR branch and fails only on the queue branch. So the author sees a green PR, the queue ejects it without a reason, and
mergeStateStatusgoes back toCLEAN— indistinguishable from a PR that is simply waiting. Whoever is in the queue at the time absorbs the blame.Review focus