Skip to content

test: add privacy-safe perf workload identity - #16023

Merged
christian-byrne merged 3 commits into
test/perf-cdp-task-accountingfrom
test/perf-workload-identity
Aug 26, 2026
Merged

christian-byrne merged 3 commits into
test/perf-cdp-task-accountingfrom
test/perf-workload-identity

Conversation

@christian-byrne

@christian-byrne christian-byrne commented Aug 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Stacked on #16016/#15997. Add schema-v4, per-measurement workload identity so results are comparable only when topology and environment denominators match.

Identity and privacy

Includes anonymized topology hash/counts, renderer/settings, viewport/DPR, frontend/build/browser identity, coarse GPU classification, and explicit nullable activity fields. Raw node IDs, types, labels, widget values, prompts, paths, workflow content, and raw GPU strings are excluded. Dirty-reason keys are allowlisted and counts sanitized.

Verification

Identity + CDP tests 9/9; Knip; browser/scripts typechecks; formatting; hooks and diff checks pass.

Performance proof evidence

  • Role: privacy-safe performance-harness identity/schema infrastructure, not a runtime optimization.
  • Local proof recorded at 52f3f5d82f29a01bb6efb6179e95c5a7d3d61e6d:
    PATH=/home/c_byrne/.nvm/versions/node/v25.9.0/bin:$PATH \
    NODE_OPTIONS=--localstorage-file=/tmp/codex-proof-wave-b-localstorage \
    node_modules/.bin/vitest run \
      scripts/perf-workload-identity.test.ts \
      src/utils/cdpPerformanceMetrics.test.ts
    Result: 2 files, 9/9 tests passed; Vitest 285 ms, wall 0.89 s.
  • Controls covered: deterministic serialization/hash, topology-order independence, privacy exclusions, unavailable activity markers, and inherited CDP accounting.
  • Current-head caveat: the head has advanced since that local run.
  • Browser/screenshots: product-browser visual QA and screenshots are not applicable. A future harness smoke should attach a sanitized schema-v4 artifact to prove real measurement identity propagation.
  • Final GitHub state: merged at final head 469632ad225f2facb59ad174aa842fc1d6796f33; the fresh audit found no red or pending checks. The exact local command above was run on the cited earlier revision.

@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Aug 26, 2026
@coderabbitai

coderabbitai Bot commented Aug 26, 2026 •

Copy link
Copy Markdown
Contributor

Important

Approval pending

CodeRabbit has no unresolved comments, but it has not reviewed the latest commit.

Use the checkbox below to review the latest commit. CodeRabbit will approve the changes if it finds no blocking issues.

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 26, 2026 •

Copy link
Copy Markdown
Contributor

🎨 Storybook: ✅ Built — View Storybook

Details

⏰ Completed at: 08/26/2026, 11:02:07 PM UTC

Links

🎭 Playwright: ✅ 2013 passed, 0 failed

📊 Browser Reports
  • chromium: View Report (✅ 1992 / ❌ 0 / ⚠️ 0 / ⏭️ 5)
  • chromium-2x: View Report (✅ 2 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • chromium-0.5x: View Report (✅ 1 / ❌ 0 / ⚠️ 0 / ⏭️ 0)
  • mobile-chrome: View Report (✅ 18 / ❌ 0 / ⚠️ 0 / ⏭️ 0)

📦 Bundle Size

⏳ Size data collection in progress…

⚡ Performance

⚠️ Performance tests failed. Check the CI workflow logs.

@github-actions github-actions Bot added the risk:R2 PR risk grade (advisory shadow check; grader-owned) label Aug 26, 2026
@codecov

codecov Bot commented Aug 26, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@                        Coverage Diff                        @@
##           test/perf-cdp-task-accounting   #16023      +/-   ##
=================================================================
+ Coverage                          82.61%   82.62%   +0.01%     
=================================================================
  Files                               1942     1942              
  Lines                             111756   111759       +3     
  Branches                           32668    32182     -486     
=================================================================
+ Hits                               92323    92337      +14     
+ Misses                             19095    19072      -23     
- Partials                             338      350      +12     
Flag Coverage Δ
e2e 66.90% <100.00%> (+<0.01%) ⬆️
unit 74.83% <33.33%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/App.vue 30.23% <100.00%> (+3.40%) ⬆️
src/utils/cdpPerformanceMetrics.ts 100.00% <100.00%> (ø)

... and 16 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@christian-byrne

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 26, 2026 •

Copy link
Copy Markdown
Contributor
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@christian-byrne

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 26, 2026 •

Copy link
Copy Markdown
Contributor
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@benjcooley benjcooley left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: the real perf collection path is broken, and the new identity is not used to gate comparisons.

Severity Count
BLOCKER 2
SHOULD FIX 0
TRIVIAL 0

The perf CI timed out after every measurement-backed test failed; no perf artifact was produced.

Comment thread scripts/perf-report.ts Resolved
@christian-byrne christian-byrne self-assigned this Aug 26, 2026
@vercel

vercel Bot commented Aug 26, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
comfy-ui-frontend-desktop-ui Ready Ready Preview Aug 26, 2026 10:57pm

Request Review

@christian-byrne
christian-byrne dismissed benjcooley’s stale review August 26, 2026 23:30

Stale after both blockers were verified fixed.

Full context for agent readers

Exact remote head 469632ad2 reads Vite-provided page globals in Playwright and filters baseline/history/trend samples by comparable workload identity. Fresh-clone focused tests pass 42/42; bypassing identity filtering fails the mismatch detector. Current-head CI has no failed checks; this stacked branch reports no separately required checks.

@christian-byrne
christian-byrne merged commit 2b1535c into test/perf-cdp-task-accounting Aug 26, 2026
87 checks passed
@christian-byrne
christian-byrne deleted the test/perf-workload-identity branch August 26, 2026 23:31
christian-byrne added a commit that referenced this pull request Aug 27, 2026
Stacked on #16016/#15997. Add schema-v4, per-measurement workload
identity so results are comparable only when topology and environment
denominators match.

Includes anonymized topology hash/counts, renderer/settings,
viewport/DPR, frontend/build/browser identity, coarse GPU
classification, and explicit nullable activity fields. Raw node IDs,
types, labels, widget values, prompts, paths, workflow content, and raw
GPU strings are excluded. Dirty-reason keys are allowlisted and counts
sanitized.

Identity + CDP tests 9/9; Knip; browser/scripts typechecks; formatting;
hooks and diff checks pass.

Addresses review feedback:
#16016 (comment)
@huang47 huang47 added the risk:high PR risk grade (advisory shadow check; grader-owned) label Sep 2, 2026 — with ChatGPT Codex Connector
@huang47 huang47 removed the risk:R2 PR risk grade (advisory shadow check; grader-owned) label Sep 2, 2026

This branch was successfully deployed

1 active deployment
Preview – comfy-ui-frontend-desktop-ui — 469632ad Deployed Aug 26, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high PR risk grade (advisory shadow check; grader-owned) size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants