Skip to content

feat(benchmarks): group runs by workflow and hardware, one comparison per group - #2005

Closed
mariana953 wants to merge 20 commits into
Comfy-Org:PabloWiedemann/sidebar-tabs-assetsfrom
mariana953:mar/benchmarks-layout
Closed

mariana953 wants to merge 20 commits into
Comfy-Org:PabloWiedemann/sidebar-tabs-assetsfrom
mariana953:mar/benchmarks-layout

Conversation

@mariana953

@mariana953 mariana953 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

ELI5

Benchmarks used to be a big spreadsheet of runs plus a second table you had to fill by ticking rows. Now each workflow on each machine gets its own card with one table and one bar per session, and the fastest build is marked right in the row.

Summary

  • Runs are grouped by workflow + hardware, because only runs of the same workflow on the same machine are comparable. Each group has its title on the page (the workflow name in sentence case, since it is content and not a section label) and its table in a card.
  • Row controls (drag handle, checkbox) sit in the card gutter and show on hover or when selected, so session names line up with the title.
  • No axis row under the bars; the tooltip shows every duration. The fastest bar is a lighter gray instead of white.
  • Each row shows the session, build, a bar with a min to max whisker for the chosen metric, the value, and a Vs fastest column (or Vs baseline once you pick one).
  • Compare by (Fastest / Average / Slowest) replaces the separate metric select and sort select.
  • One columns picker replaces the two Properties pickers.
  • Row ⋯ menu: Set as baseline, Rename session, Delete files. Delete keeps the existing confirm dialog. Row and group menus use the same design system menu as the filters, with no custom styling.
  • Selecting rows opens a floating selection bar (white, so it stands out over the dark tables) with Export results and Clear. The 20-run export limit still applies, with the same tooltip.
  • On narrow windows the "Median and range" header wraps to two lines and the bar column narrows, so tables fit without a sideways scrollbar. If a window is narrower still, the table scrolls with the app's slim scrollbar.
  • Each group can also be exported on its own from its ⋯ menu, and collapsed. A collapsed group becomes a slim card, so a page of collapsed groups reads as a compact list.
  • States: skeleton while loading, error with Refresh, empty with a link to Performance Test, and "no matches" for filters. Empty, error and no-matches use the same centered pattern as the Apps empty state, with no card.
  • Toolbar follows Apps and Builds: Workspace, Build, Hardware, Workflow and Compare by on the left; refresh, search and columns on the right. Every dropdown is the design system Menu with radio items, the same one Apps uses for Filter and View, so size, spacing and margins match. The title has no subtitle, like the other pages.
  • The table follows the shared table criteria used elsewhere in the app: 12px sentence-case header with a hairline under it, hairlines between rows that hide on hover and selection, rounded row fill.

No functionality removed

Before After
Workspace, build, hardware, workflow filters Same filters, same toolbar
Search runs Same
Refresh, Select folder Same (Select folder in the header)
Library columns picker + comparison columns picker One columns picker
Sort by date and columns Compare by + manual order (drag, or Alt + arrow keys)
Comparison metric select Compare by
Duration / Speed toggle Vs column, switches to Vs baseline when a baseline is set
Set baseline, Reset baseline Row ⋯ menu, Reset in the card footer
Best value highlight Fastest pill + green value in the extra columns
Rename session, Delete files Row ⋯ menu
Collapse comparison Collapse per group
Export results (max 20) Selection bar and group ⋯ menu, same limit

Judgement calls

  • The first 3 runs are no longer preselected on load. Every group already shows its comparison, so selection is only for export.
  • The exported image includes relative speed only when a group has a custom baseline, matching what the page shows. Without one, it exports durations as before.
  • Runs from different workflows or machines are never compared in the same table, even if both are selected. Export still accepts a mixed selection, as before.

Before / after

Before is the base branch (#1981). After is this PR. Seeded data, 1320 px window. Screenshots use main's gray palette (#1996) applied locally: this branch is stacked on #1981, which predates it. The branch does not change the palette, so on main it renders gray.

Before After
Default
Scrolled
After: interactions
Rows selected, selection bar
Row ⋯ menu
Custom baseline
Bar hover
Narrow window (1024 px)
Empty
No matches
Filter menu (same Menu as Apps)
Compare by menu
All groups collapsed
One group open

Test plan

  • Full renderer suite passes (1722 tests), eslint, prettier, typecheck
  • Captured in the built app with seeded runs (default, selection, menu, baseline, hover, narrow, empty)
  • Select folder changes the folder and reloads
  • Set a baseline, then reset it from the card footer
  • Rename and delete a session from the row menu
  • Export a selection and a single group

🤖 Generated with Claude Code

The Comparison title moves inside its card, styled like the other
section labels, and the two Properties pickers become icon buttons with
the label kept for screen readers, so the comparison toolbar fits one
row. Matches the New UI proposal's Benchmarks page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 10, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@PabloWiedemann
PabloWiedemann force-pushed the PabloWiedemann/sidebar-tabs-assets branch from a3c501f to 569fb3a Compare November 10, 2026 21:01
Drop the card around the runs library and the Comparison section: filters,
refresh, search and columns sit in one toolbar row, the table is a plain
hairline list without the forced horizontal scroll, and Select folder moves
to the page header. Empty states get a quiet dashed placeholder and row
delete buttons stay muted until hovered. BrandedPageHeader gains an
optional actions slot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mariana953 mariana953 changed the title style(benchmarks): Comparison as one card, icon-only column pickers style(benchmarks): open library list, Select folder in the header Nov 10, 2026
mariana953 and others added 8 commits November 10, 2026 22:41
Series colours follow the platform proto palette (no yellow), the baseline
marker and drop target use the neutral foreground, checked rows use a white
check and a soft fill instead of a coloured stripe, and tables, the chart and
empty states use fills instead of strokes. Tests updated for the new hex
values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… panel

Use the display title with no subtitle and the same page column as Builds
and Apps. The comparison title and its controls share one row in a single
panel, the chart comes before the metric table, and the best cell is a
neutral fill instead of green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use the design system card surface (bg-card) so Benchmarks matches the
other pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One page, one card per workflow and hardware. Each card names the fastest
build, shows a bar per session for the selected metric with its range, and
a Vs column against the fastest run or a chosen baseline. Row menu sets the
baseline, renames or deletes; selected rows export through a floating bar.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mariana953 pushed a commit to mariana953/Comfy-Desktop that referenced this pull request Nov 11, 2026
@mariana953 mariana953 changed the title style(benchmarks): open library list, Select folder in the header feat(benchmarks): group runs by workflow and hardware, one comparison per group Nov 11, 2026
Workflow name as a sentence-case title, a smaller summary with the lead as a percentage, row controls in the card gutter so rows align with the title, no axis row, softer bars.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mariana953 pushed a commit to mariana953/Comfy-Desktop that referenced this pull request Nov 11, 2026
mariana953 pushed a commit to mariana953/Comfy-Desktop that referenced this pull request Nov 11, 2026
mariana953 and others added 2 commits November 11, 2026 13:09
Filters on the left; refresh, Compare by, columns and search on the right, search last at 220px. Title without a subtitle, as on the other pages. The per-group summary sentence is gone (the Fastest pill and the Vs column carry it). Wider gutter for the row controls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mariana953 pushed a commit to mariana953/Comfy-Desktop that referenced this pull request Nov 11, 2026
…er app tables

Filters and Compare by use BaseSelect with a new optional label prefix, so
every dropdown is one component and options never wrap. Table head, row
hairlines and hover follow the shared table criteria. Select folder is md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mariana953 pushed a commit to mariana953/Comfy-Desktop that referenced this pull request Nov 11, 2026
mariana953 and others added 2 commits November 11, 2026 14:24
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The five toolbar dropdowns use the design system Menu with radio items, the
same one Apps uses for Filter and View, so size, spacing and margins match.
BaseSelect goes back to its original shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mariana953 pushed a commit to mariana953/Comfy-Desktop that referenced this pull request Nov 11, 2026
Collapsed groups show as compact rows in a card with tight spacing, and
every group header shares the same inset so titles line up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mariana953 pushed a commit to mariana953/Comfy-Desktop that referenced this pull request Nov 11, 2026
Same centered pattern as the Apps empty state: heading, one muted line and
at most one white action, with no card around it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mariana953 pushed a commit to mariana953/Comfy-Desktop that referenced this pull request Nov 11, 2026
mariana953 and others added 2 commits November 11, 2026 18:17
…windows

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the custom radius, padding and item overrides so every menu on the
page renders with the design system menu, like the filter dropdowns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mariana953

Copy link
Copy Markdown
Contributor Author

Moved to #2015, based on main now that #1981 is closed. Same change.

@mariana953 mariana953 closed this Nov 11, 2026
Chroma01 pushed a commit to Chroma01/Comfy-Desktop that referenced this pull request Nov 13, 2026
… per group (Comfy-Org#2015)

* feat(benchmarks): group runs by workflow and hardware, one comparison per group

Same change as Comfy-Org#2005, now based on main instead of the closed Comfy-Org#1981.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* style: page title matches the Builds title on main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(benchmarks): compare against the whole group, split machines, fix reordering and exports

Keeps Mar's grouped layout and rebuilds the logic behind it.

- Runs group by workflow and machine profile (GPU, CPU, memory, OS), read from each
  run's results.json in main. Two PCs with the same GPU no longer share a table; the
  label adds the CPU (then memory, then OS) only when needed to tell machines apart.
- Fastest, baseline, bar scale and best-value highlights come from every run in the
  group, not just the rows a filter leaves visible. When the reference row is hidden,
  the card says which session it is compared with. A chosen baseline survives filters.
- "Only one build ran this workflow here" now counts builds in the whole group, so it
  no longer appears on every card when a Build filter or search is active.
- Manual order is stored per group over all of its runs, so rows hidden by a filter,
  runs added after a Performance Test and runs from a newly selected folder can all be
  moved. Dropping onto a lower row places the row after it, so the bottom slot is
  reachable. Alt+Up/Down join Alt+Left/Right. Compare by no longer has a Manual option;
  a header click goes back to sorted order.
- Selection count and export only include runs that pass the filters. Relative speed is
  exported only when the export is one group with a chosen baseline. Series colours
  follow export order. Reloading clears a stale export error.
- Switching folders resets baselines, manual order and collapsed cards.
- Session names rename in place with the shared inline-edit composable; the row menu's
  Rename focuses the name after the menu closes.
- The columns picker is the design-system Menu with checkbox items, replacing the
  details element and its document-level listener.
- Bars are focusable with an aria-label listing all four durations, so the tooltip
  works from the keyboard.
- Grouping and comparison live in lib/benchmarkGroups.ts; each card is
  views/benchmarks/BenchmarkGroupCard.vue.

* feat(benchmarks): group by choice, responsive table, column reorder and baseline flag

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(benchmarks): keep the range header on one line in narrow windows

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(benchmarks): keep the toolbar on one row and fold search into an icon when narrow

Filters sit on the left and wrap as a group; refresh, search and columns stay
on the right. Below 720px of toolbar width the search becomes an icon button
that opens the field in place, focused, and folds back on Escape or blur when
empty. Select folder stays beside the page title.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(benchmarks): list every table column in the columns picker, uppercase its title

The picker now shows the columns the table always has. Session, the Compare by
value and Vs baseline are checked and locked; Workflow and the range bar can be
hidden like any other column, and the Compare by metric no longer shows twice.
The menu title uses the uppercase section label style.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(benchmarks): even toolbar rows at every width

One row when everything fits. Below that, the data filters (workspace, build,
hardware, workflow) take the first row and the view options (group by, compare
by) share the second with refresh, search and columns. In narrow windows the
search row goes on top and the six menus form an even two-column grid, so no
menu is left alone on a row. Search is always visible, so the fold-to-icon
toggle and its string are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(benchmarks): announce Alt+Left/Right on column headers and keep their focus

Column headers reused the row hint, which tells you to press Alt+Up or
Alt+Down, in their tooltip and screen reader description. Columns move with
Alt+Left/Right, so they now get their own hint.

Moving a column to the left with the keyboard moved the focused header in
the DOM, and Chromium dropped focus to the page, so the next Alt+arrow did
nothing. The header button gets focus back after the move, as the row grips
already do.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Benjamin Lu <benjaminlu1107@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant