Repository navigation
test: document QwenVL dropdown only reaching the first registered model directory - #14311
claude[bot] wants to merge 4 commits into
Conversation
🔬 E2E Coverage
🎭 Playwright: ✅ 1822 passed, 0 failed · 1 flaky📊 Browser Reports
🎨 Storybook: ✅ Built — View Storybook🌾 Fallow: ✅ PassedFallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened. No new findings in the changed files. Reproduce locally# the base the gate compared against
npx fallow audit --root . --quiet --format json \
--changed-since 4d28112ea0855fa80c07338d9fd85a9d58dd4e69
Full docs: ADR-DEVEX-LINT-0015 |
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #14311 +/- ##
==========================================
+ Coverage 78.95% 81.57% +2.62%
==========================================
Files 2209 1882 -327
Lines 112281 109185 -3096
Branches 32992 31287 -1705
==========================================
+ Hits 88647 89066 +419
+ Misses 23178 19747 -3431
+ Partials 456 372 -84
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
📄 Knowledge reviewDosu skipped reviewing this PR because your organization has used its |
aaab428 to
b1b9d65
Compare
|
What this pins: the model dropdown only ever reads the first of a node's registered directories, so models in the second and later directories are invisible in the picker. Real fix: FE-1177. Why Merge order. #14311, #14313 and #14314 all insert into On the green checks. Worth knowing before you trust them: on 2026-08-11 all five of these were 262 commits behind main, so their passing CI proved nothing about current main. They were rebased onto Note for anyone running these locally: use |
#14489 de-registered the QwenVL family from MODEL_NODE_MAPPINGS, which fixed the reported symptom but left the underlying bug class untouched: the asset browser still resolves a node type to exactly one category and queries only that directory. Retargets the it.fails case onto LS_LoadSegformerModel (3 registered directories, the worst remaining case) so the test dies honestly when the bug is actually fixed, rather than documenting an example that no longer exists in the product. Verified failing for the right reason: include_tags resolves to 'models,segformer_b2_clothes', so segformer_b3_clothes and segformer_b3_fashion are unreachable. Still live on main as of 2026-08-03 for 5 node/input pairs; see the comment block for the full list.
b1b9d65 to
2667936
Compare
christian-byrne
left a comment
There was a problem hiding this comment.
Request changes: this detector fabricates a picker path removed by #14529.
Full context for agent readers
That merged fix is already an ancestor of this exact head. Production intentionally leaves all three Segformer directories unmapped because the node expects IDs and falls back to /api/object_info, while the test alone inserts LS_LoadSegformerModel -> segformer_b2_clothes. The it.fails case therefore exercises only mocked reachability, not a live product path. The inline finding identifies the required change.
Exact-head R2 review: bec2de1d5c01e565cc420f3358250aa99e600a2a.
| buildAssetListResponse([validAsset({ id: 'segformer-b3-fashion' })]) | ||
| ) | ||
|
|
||
| await assetService.getAssetsPageForNodeType('LS_LoadSegformerModel') |
There was a problem hiding this comment.
Blocker: this picker path exists only in the test mock.
Full context for agent readers
#14529 intentionally removed all LS_LoadSegformerModel mappings because the widget accepts IDs rather than asset filenames and should fall back to /api/object_info; that merged change is an ancestor of this exact head. The only category registration now exists in this test's mock at line 50, so getAssetsPageForNodeType() can exercise only a fake path and cannot detect a live dropdown regression. The five-pair table is stale too: #14529 removed onnx and clip, the Ultralytics rows remain disabled, and FlashVSR's empty input key does not expose an eligible widget picker. Please remove this case or retarget it only if a current reachable production mapping exposes the behavior.
There was a problem hiding this comment.
Confirmed and removed — agreed on all points.
Re-verified against current main: LS_LoadSegformerModel has no segformer_* rows left in MODEL_NODE_MAPPINGS at all, so the only category registration was in this test's own mock at line 50, exactly as you said. Checked the rest of the five-pair table too: UpscaleModelLoader/CLIPVisionLoader are down to one directory each now that #14529 removed onnx and clip, UltralyticsDetectorProvider's rows are still commented out/disabled, and FlashVSRNode's two directories sit behind an empty input key so no widget picker exists for it either. Searched the current mapping table for any other node registered against multiple directories under one non-empty key to retarget onto instead — there isn't one, so there's no reachable production analog left for this bug shape right now.
Removed the it.fails case and its supporting mock entry (commit fe4f25b), consistent with how #14312 and #14314 resolved the same #14489/#14529-driven staleness elsewhere in this series. Also added a 2026-08-25 status note to the PR description that reconciles the now-stale 2026-08-03 five-pair table with this outcome, rather than leaving it dangling.
pnpm test:unit src/platform/assets/services/assetService.test.ts passes (65/65), and pnpm lint/pnpm format are clean.
Leaving this thread for you to re-review/resolve per the repo's review-comment-resolution convention.
Generated by Claude Code
#14529 (already an ancestor of this branch) removed all three segformer_* rows from MODEL_NODE_MAPPINGS, along with onnx and clip, because those widgets accept ids rather than asset filenames and fall back to /api/object_info. The only remaining LS_LoadSegformerModel category registration lived in this test's own mock, so the it.fails case exercised a picker path that no longer exists in production. Checked the rest of MODEL_NODE_MAPPINGS for a still-reachable node registered against multiple directories under one non-empty input key: none exists. UltralyticsDetectorProvider's rows remain disabled, and FlashVSRNode's two directories sit behind an empty key, so it never exposes a widget picker. Removing rather than retargeting, matching how #14312 and #14314 resolved the same staleness elsewhere in this test series.
…nd bucket key (Comfy-Org#14313) <!-- ccr-slack-attribution --> _Requested by **Christian Byrne** · [Slack thread](https://comfy-organization.slack.com/archives/C0A4XMHANP3/p1785357549411849?thread_ts=1785296212.634389&cid=C0A4XMHANP3)_ This adds a failing test documenting that the node-widget dropdown's category identifier, a full directory path like LLM/Qwen-VL/Qwen2.5-VL-3B-Instruct, and the model-library sidebar's bucket key, the top-level folder only such as LLM, are incompatible, so querying the bucket-based model list with the widget's own category string returns nothing. This mismatch is a known blocker to migrating the node-dropdown query onto the newer bucket-based asset listing, since the two systems need to agree on category identifier shape first. This test exists so the mismatch has permanent, executable documentation rather than living only in investigation notes. Root cause: getCategoryForNodeType, the forward/widget path, returns full hierarchical paths, while modelFolderFromTag, the bucket path, collapses bare tags to only their first path segment, two different identifier spaces for what's conceptually the same category. Path to a real fix: normalizing all tag-as-identifier usages onto one consistent scheme, follow-up work that hasn't been scheduled yet, would resolve this. Once that lands, convert this test from it.fails to a real assertion. How to verify: pnpm test:unit src/platform/assets/services/assetService.test.ts --- _Generated by [Claude Code](https://claude.ai/code/session_01P6mHd4AJXxneJFqt8y3ozA)_ --- ### Status note — 2026-08-03 Re-verified against `main` on this date. `AILab_QwenVL` and eight other model-id-based node families were **de-registered from `MODEL_NODE_MAPPINGS` by Comfy-Org#14489** (merged 2026-08-01, backported to `cloud/1.48` via Comfy-Org#14527). That fixed the *reported symptom* in BE-5071 — those widgets now fall back to the node's own `/api/object_info` combo. **The bug class documented here is not fixed.** The asset browser still resolves a node type to exactly one category. As of 2026-08-03 it remains live on `main` for five node/input pairs: | Node | Input | Directories (only the first is reachable) | |---|---|---| | `LS_LoadSegformerModel` | `model_name` | `segformer_b2_clothes`, `segformer_b3_clothes`, `segformer_b3_fashion` | | `UpscaleModelLoader` | `model_name` | `upscale_models`, `onnx` | | `CLIPVisionLoader` | `clip_name` | `clip_vision`, `clip` | | `FlashVSRNode` | *(auto)* | `FlashVSR`, `FlashVSR-v1.1` | | `UltralyticsDetectorProvider` | `model_name` | `ultralytics/bbox`, `ultralytics/segm` | Tracked by FE-1177 (query `model_type:` rather than a single directory), FE-1181 (generate the mapping from `object_info`), FE-1080 (child-directory tags) and BE-5255 (backend subtype-carrying tag). **Why this is a merged test rather than a Slack thread:** `it.fails` inverts the report — the assertion genuinely fails (proving the bug is real) but CI stays green. It turns **red only when someone fixes the bug without updating the test**, which is exactly the alarm we want. As an unmerged draft it provides none of that, which is why these are being taken out of draft. ### Status note — 2026-08-25 Per [review](Comfy-Org#14313 (review)), `AILab_QwenVL` / `LLM/Qwen-VL/*` was an invalid target: Comfy-Org#14489 (merged 2026-08-01, confirmed an ancestor of this PR's exact head via `git merge-base --is-ancestor`) intentionally de-registered it from `MODEL_NODE_MAPPINGS` because that widget accepts a model or repository ID and now falls back to `/api/object_info`, so the mock's `LLM/Qwen-VL/*` category never reaches a live product path. Unlike the sibling PRs in this series, this PR's bug shape (a full hierarchical category path vs. a bucket key truncated to its first path segment) has a still-reachable production analog, so the case was **retargeted, not removed**. Checked current `MODEL_NODE_MAPPINGS` for a live entry that is (a) a multi-segment directory path and (b) registered with a non-empty input key, since an empty key means the node auto-loads without a widget and can't expose the bug. The `chatterbox/*` and `CogVideo/ControlNet` entries are also multi-segment but all use an empty key, so they're not eligible either. `['LLM/checkpoints', 'LoadChatGLM3', 'chatglm3_checkpoint']` is: it's live on `main`, uses the real `chatglm3_checkpoint` widget key, and `getAssetCategories`'s bare-tag mode (`asset.tags.map(tag => tag.split('/')[0])`) collapses an `LLM/checkpoints`-tagged asset to bucket key `LLM`, while `getCategoryForNodeType('LoadChatGLM3')` still returns the full `LLM/checkpoints` string — the exact same key-space mismatch the test documents, on a mapping nothing has removed. Also independently re-checked the 2026-08-03 five-pair table per the review's flag: `Comfy-Org#14529` (also confirmed an ancestor of this head) removed the `segformer_*`, `onnx`, and `clip` mappings entirely; `UltralyticsDetectorProvider`'s two rows remain commented out/disabled; and `FlashVSRNode` registers two directories under an empty input key, so it never exposes a widget picker. None of the five pairs still exhibit a live, widget-reachable symptom — consistent with the same conclusion reached independently on Comfy-Org#14311, Comfy-Org#14312, and Comfy-Org#14314 today, though that table was never this test's own case and doesn't otherwise affect this PR's fix. Mock and assertion in `src/platform/assets/services/assetService.test.ts` now use `LoadChatGLM3` / `chatglm3_checkpoint` / `LLM/checkpoints` in place of `AILab_QwenVL` / `model_name` / `LLM/Qwen-VL/Qwen2.5-VL-3B-Instruct`. `pnpm test:unit src/platform/assets/services/assetService.test.ts` passes (65 passed, 1 expected fail). How to verify: `pnpm test:unit src/platform/assets/services/assetService.test.ts` --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Christian Byrne <cbyrne@comfy.org>
fe4f25b to
2cbde48
Compare
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
❌ ErrorsYour PR has failed checks. Please review the issues below and take necessary action before merging. 🚦 6 Pipeline jobs failed
Useful? React with 👍 / 👎 This comment will be updated automatically if new data arrives.🔗 Commit SHA: 2cbde48 | Docs | View more details | Give us feedback! |
Requested by Christian Byrne · Slack thread
This adds a failing test documenting that the AILab_QwenVL model dropdown on Cloud only ever lists models from the first of its 10 registered directories, making most of them (e.g. Qwen3-VL-8B-Instruct) unselectable and breaking published workflows that reference them.
This is a known, currently-triaged bug rather than something fixed here. A separate, already-merged model-tagging/asset-browser migration (PRs #12948, #12972, #13302, #13464, #13574) reworked how the model-library sidebar lists models, but it does not touch the node-dropdown query path this bug lives in, so it does not fix this bug on its own. An earlier, unmerged attempt at a direct fix exists in git history at fix/qwenvl-model-dropdown-category, and is confirmed broken, since it registers a bare parent tag that no asset actually carries, which would return an empty list instead of a partial one, due to exact (non-hierarchical) tag matching.
Root cause: getCategoryForNodeType in src/stores/modelToNodeStore.ts only keeps the first registered category per node type, and getAssetsPageForNodeType in src/platform/assets/services/assetService.ts queries only that one category, exact match, no fallback.
Path to a real fix: the long-term direction is migrating this query onto the bucket-based asset listing added by the migration above, and replacing the hand-maintained MODEL_NODE_MAPPINGS list with one generated from node definitions. Neither of those follow-up efforts has been scheduled yet. Once either lands, convert this test from it.fails to a normal assertion.
How to verify: pnpm test:unit src/platform/assets/services/assetService.test.ts
Status note — 2026-08-03
Re-verified against
mainon this date.AILab_QwenVLand eight other model-id-based node families were de-registered fromMODEL_NODE_MAPPINGSby #14489 (merged 2026-08-01, backported tocloud/1.48via #14527). That fixed the reported symptom in BE-5071 — those widgets now fall back to the node's own/api/object_infocombo.The bug class documented here is not fixed. The asset browser still resolves a node type to exactly one category. As of 2026-08-03 it remains live on
mainfor five node/input pairs:LS_LoadSegformerModelmodel_namesegformer_b2_clothes,segformer_b3_clothes,segformer_b3_fashionUpscaleModelLoadermodel_nameupscale_models,onnxCLIPVisionLoaderclip_nameclip_vision,clipFlashVSRNodeFlashVSR,FlashVSR-v1.1UltralyticsDetectorProvidermodel_nameultralytics/bbox,ultralytics/segmTracked by FE-1177 (query
model_type:rather than a single directory), FE-1181 (generate the mapping fromobject_info), FE-1080 (child-directory tags) and BE-5255 (backend subtype-carrying tag).Why this is a merged test rather than a Slack thread:
it.failsinverts the report — the assertion genuinely fails (proving the bug is real) but CI stays green. It turns red only when someone fixes the bug without updating the test, which is exactly the alarm we want. As an unmerged draft it provides none of that, which is why these are being taken out of draft.Following that note, the case was retargeted from
AILab_QwenVLontoLS_LoadSegformerModel(the worst remaining case at three directories), since QwenVL itself was gone fromMODEL_NODE_MAPPINGSby that point.Status note — 2026-08-25
The table above is now stale, and the retargeted case has been removed rather than fixed further.
Per review, #14529 (merged 2026-08-01, an ancestor of this PR's head) removed all three
segformer_*rows along withonnxandclip— the same "id widget, not an asset filename" / "unreachable directory" reasoning #14489 applied to QwenVL. Checking each row in the 2026-08-03 table above against currentMODEL_NODE_MAPPINGS:LS_LoadSegformerModelis no longer mapped at all, so the only remaining category registration lived in this test's own mock;UpscaleModelLoaderandCLIPVisionLoaderare each down to a single directory now thatonnxandclipare gone, so neither is a multi-directory case anymore;UltralyticsDetectorProvider's two rows remain commented out/disabled (per #12075/BE-689), so neither directory is reachable at all; andFlashVSRNodestill spans two directories but behind an empty input key, meaning it never exposes a widget picker for the bug to manifest in.So none of the five original pairs still exhibit a live, widget-reachable "reads only the first directory" symptom. Searched the rest of current
MODEL_NODE_MAPPINGSfor any other node registered against multiple directories under one non-empty input key, and there isn't one —FlashVSRNode's empty-key rows are the only remaining multi-directory registration in the table.Per Christian's explicit fallback (remove the case unless it can be retargeted onto a still-reachable production mapping with a decided contract), and consistent with how #14312 and #14314 resolved the identical #14489-driven staleness elsewhere in this test series, the
it.failscase and its supporting mock entry have been removed. The underlying bug class (getCategoryForNodeTypekeeping only the first registered category per node type) is real and still present insrc/stores/modelToNodeStore.ts, but there is no reachable production node/input pair left to pin it against without inventing a mapping that doesn't exist onmain. If a futureMODEL_NODE_MAPPINGSaddition reintroduces a genuine multi-directory, non-empty-key case, this test should be restored against that node.How to verify:
pnpm test:unit src/platform/assets/services/assetService.test.ts