Repository navigation
test(website): pin randomized seeds in the SDK snippet upload test - #19464
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (1)
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 2 remain after this review. 📝 WalkthroughWalkthroughThe workshop workflow snippet tests now use a shared input builder. It combines URL-upload fields with values for randomized inputs in both test cases. ChangesWorkshop workflow snippet tests
Priority: ⬇️ Low Estimated code review effort: 2 (Simple) | ~10 minutes Change: Other Suggested reviewers: Merge Risk: ⚪ Minimal · up to This change stabilizes the workflow snippet test inputs without changing production behavior. No material merge risk is evident from the supplied review context. 🚥 Pre-merge checks | ✅ 7✅ Passed checks (7 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
benjcooley
left a comment
There was a problem hiding this comment.
👨💻 BILL: review of 06ce8b601f, ready
This is a test-only fix for main's red snippet suite.
- The cause: #19323's "sets the inputs the browser uploads and saves every output" test built its inputs without pinning
randomizeinputs. After #19391,prepareWorkflowRenderdraws a fresh seed, so all 26 cases failed. On main25c268595dI reproduced 26 failed / 66 passed. - The fix: both tests share a
browserInputs(model)helper that pins everyrandomizeinput to 7, the same way the first test already did. No assertions changed. - Verified:
workshop-workflow-snippet.test.tspasses 92/92 on 3 consecutive runs, and the wholesrc/configsuite passes 3,417/3,417.
| Severity | Count |
|---|---|
| BLOCKER | 0 |
| SHOULD FIX | 0 |
| TRIVIAL | 0 |
🌾 Fallow: ✅ PassedFallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened. No new findings in the changed files. Reproduce locally# the base the gate compared against
npx fallow audit --root . --quiet --format json \
--changed-since 25c268595d612e2ee0aa1749191eee6a022c601f
Full docs: ADR-DEVEX-LINT-0015 🌐 Website E2ETip All tests passed.
🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measureExecutes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight This candidate changes nothing the matrix measures, so every expensive stage was skipped. Reproduce locally# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
npx vitest run --config vitest.matrix.config.mts
Full docs: scripts/registry-census/README.md |
## Summary Workshop apps (Cinematic Studio and Re-shoot, at `/models/apps/<slug>`) sent no Workshop events, so their use and failures never reached PostHog or the Datadog run logs. Follows #19451, which added `page_type: 'app'`, `app_slug` and the Datadog `feature: apps` split. ## Changes - **What**: - **App views:** `model_viewed` with `page_type: 'app'` and `app_slug` fires once per app shown, only when Workshop and the apps flag let the page render. The app switcher counts each app once. - **Studio takes:** each Cinematic Studio take (one Router request) records `run_started` and `run_finished`: - succeeded, with `request_id` and `output_count`; - failed, with `workshopFailureAnalytics`: the reason, HTTP status, Router error type and request ID; - or cancelled. A take whose model page fails to load records a failed run (`unavailable`, `input_preparation`); a take cancelled while its model loads records nothing. These carry the same model fields as model-page runs (`model_slug`, `router_id`, `provider`, `render_engine: router`) plus `page_type: 'app'` and `app_slug`, so they also land in the Datadog run logs under `feature: apps`. - **Studio downloads:** `output_download_clicked` as an app event. - Also carries the SDK snippet seed-test fix, identical to #19464 (already on `main`), so it adds nothing when squashed. - `workshopModelAnalytics` now files `APP` models as `page_type: 'app'` with `app_slug`, instead of `workflow` (Bill's trivial on #19451). ## Review Focus - **Scope:** Re-shoot is still a demo with no paid runs, so only its view is recorded. - **Studio slug:** `CINEMATIC_STUDIO_APP_SLUG` is a constant. A test asserts it matches the app content config, which keeps that config out of the studio's client bundle. - **Tests:** new table and page tests cover succeeded and failed takes, downloads, app views and the helper. All fail on the old source. The download test cancels the link's navigation so happy-dom doesn't leave the page. The website suite passes (9777).
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #19464 +/- ##
==========================================
- Coverage 85.10% 84.14% -0.97%
==========================================
Files 2406 3201 +795
Lines 135477 155751 +20274
Branches 36591 42188 +5597
==========================================
+ Hits 115296 131052 +15756
- Misses 19713 22984 +3271
- Partials 468 1715 +1247
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Summary
Fixes the website unit tests on main: #19391 made empty
randomizeseeds draw a fresh value per run, and #19323 added an SDK snippet test that compares two separately prepared graphs without pinning the seed, so all 26 workflow cases fail.Changes
browserInputshelper that pins randomized seeds, as the first test already did. Test-only.Review Focus
Passes 3/3 repeated runs; website src suite passes.