Repository navigation
feat(website): one switch decides model page robots and sitemap listing - #19399
MaanilVerma wants to merge 5 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
QAThree local builds of
Sample entries with the switch on: <url><loc>https://comfy.org/models/</loc></url>
<url><loc>https://comfy.org/models/luma_2--uni-1-max-image-edit--edit-images/</loc></url>
<url><loc>https://comfy.org/models/xai--grok-imagine-video-1.5-reference--animate-images/</loc></url>Heads-up for FE-2922: twins follow No sitemap XSD check exists in the repo, so none was run. Update (fb67009, merged #19386 Update (6b6ded7, merged #19386 |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
🌐 Website E2ETip All tests passed.
🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measureExecutes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight This candidate changes nothing the matrix measures, so every expensive stage was skipped. Reproduce locally# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
npx vitest run --config vitest.matrix.config.mts
Full docs: scripts/registry-census/README.md 📦 Bundle Size
🌾 Fallow: ✅ PassedFallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened. No new findings in the changed files. Reproduce locally# the base the gate compared against
npx fallow audit --root . --quiet --format json \
--changed-since 8a695a9ec190da414b1c29a162101fdb2c0da857
Full docs: ADR-DEVEX-LINT-0015 🔗 Website PreviewWebsite Preview: https://comfy-website-preview-pr-19399.vercel.app Models is built and gated by PostHog (workshop-enabled), using staging Cloud. This commit: https://website-frontend-ke7js42n3-comfyui.vercel.app Last updated: 2026-09-29T20:31:13Z for |
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## maanil/fe-2986-models-url-registry #19399 +/- ##
======================================================================
+ Coverage 84.14% 84.23% +0.09%
======================================================================
Files 3202 3210 +8
Lines 155760 156384 +624
Branches 42186 40608 -1578
======================================================================
+ Hits 131069 131735 +666
+ Misses 22982 22949 -33
+ Partials 1709 1700 -9
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 103 files with indirect coverage changes 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
🔍 Cursor Review — Consolidated panel
Triggered by @MaanilVerma.
Found 5 finding(s).
| Severity | Count |
|---|---|
| 🟡 Medium | 3 |
| 🟢 Low | 2 |
Panel: 8/8 reviewers contributed findings.
…ry' into tmp-cascade-2921 # Conflicts: # apps/website/src/config/indexing.test.ts
benjcooley
left a comment
There was a problem hiding this comment.
Review
The off-state is genuinely inert — MODEL_PAGES_INDEXABLE && … collapses to false, isWorkshopRoute(p) && !false is the old expression, and noindex={!false} is the old attribute. Nothing ships changed. The registry lookup is the right key, and modelsUrlKind strips the trailing slash itself so the raw Astro.url.pathname is fine.
My concern is the premise rather than the code. The PR is titled "one switch decides", and the Review Focus says that with the switch on "exactly the 148 model pages join the sitemap and lose noindex; nothing else moves." That isn't accurate — isExcludedFromSitemap is also the gate for markdown-twin generation, so a third surface moves, and the pages the switch admits have no server-rendered content behind them yet. Details inline.
| Severity | Count |
|---|---|
| BLOCKER | 0 |
| SHOULD FIX | 2 |
| TRIVIAL | 1 |
TRIVIAL: the explicit : boolean on MODEL_PAGES_INDEXABLE is load-bearing. Drop it and the constant narrows to the literal false, isIndexableModelPage becomes provably constant, and a future "simplification" pass can delete the registry lookup as dead. Worth one line saying so, since the file is otherwise a bare export.
…ry' into maanil/fe-2921-models-sitemap
benjcooley
left a comment
There was a problem hiding this comment.
Re-review (scoped to d162b02 + the two base merges)
Approving. The two SHOULD FIX items from my last pass are handled, and the code in this diff is correct.
What changed
| Last round | This round |
|---|---|
| SHOULD FIX — switch not safe to flip alone | Documented at the flip site; the build-time HTML assertion I wanted turns out to already exist in #18899 (scripts/check-models-html.ts + CI wiring). Reasonable split across the stack. |
| SHOULD FIX — twin coupling untested | Real test in markdown-twin.test.ts. Verified it fails under a decoupled gate, and the includes(modelTwin) assertion keeps it from passing vacuously. |
TRIVIAL — bare : boolean |
One line explaining it. |
The new test is the good kind: it pins writeMarkdownTwins' gate to isExcludedFromSitemap rather than re-asserting what indexing.test.ts already covers, and the two switch states share one case table.
One correction, left open in the thread on model-page-launch.ts — #19404 does not add an empty-body guard. It only moves HEADER out of DROPPED_TAGS and drops role="status"; src/integrations/markdown-twins.ts is untouched and its writeFile is unconditional. So an early flip publishes an empty twin rather than a "Loading" one, and BaseLayout still emits the text/markdown alternate link because that hangs off !shouldNoindex, not off twin content. Not blocking — the switch is off and the merge order holds — but worth either building (~3 lines) or dropping from the rationale.
Housekeeping: #19404 is CONFLICTING now; its markdown-twin.test.ts block collides with the one just added here. CI on this head is still re-running after the base merges.
The switch-off path remains inert: MODEL_PAGES_INDEXABLE && … collapses to false, isWorkshopRoute(p) && !false is the original expression, and noindex={!false} is the original attribute.
…ry' into maanil/fe-2921-models-sitemap
benjcooley
left a comment
There was a problem hiding this comment.
Re-review (6b6ded7)
Still approving. The only commit since my last pass is another merge from maanil/fe-2986-models-url-registry — this PR's own diff is byte-identical at +119/-6 across the same 5 files.
I checked the merge for anything load-bearing rather than taking it as noise. Two files it touched are in this PR's dependency path:
config/workshop-release.ts— onlyassertWorkshopCloudEnvForBuildchanged (dropped theisWorkshopInBuild()early return, reworded an error).isWorkshopRoute, whichisExcludedFromSitemapactually calls, is untouched.integrations/workshop-release-gate.ts— the no-Workshop leak check moved fromexistsSyncto theassetsmap and gained anunbuiltassertion. Doesn't touchunregisteredModelsPathsor anything indexing reads.
Everything else in the merge is base drift (cinematic-studio, workshop-model-balance, workflow-render). No change to indexing.ts, markdown-twins.ts, markdown-twin.ts, BaseLayout.astro or [slug].astro.
Open item from last round is closed. I'd flagged that #19404 had no empty-body guard despite being cited as one. The author corrected the record and built it — d47506c adds a page.body === '' check before the write plus unlinkTwin to strip the text/markdown alternate from the built HTML. Verified against the diff. Both nits on that guard's implementation belong on #19404 and are noted in the thread.
Both review threads resolved. Worth re-confirming CI is green on this head before merge — it was mid-run when I last looked.
## Summary **Deploy gate: comfy-router#46 is merged and live (2026-09-30). Its follow-up comfy-router#48 (serves `/hub/models.md` instead of 301ing it to a /workflows 404) must be deployed before this merges. After the merge, purge Cloudflare for `/hub/models*`, `/hub/workflows*` and the 30 workflow names in `hub-workflow-names.json`. The redirects ship as 307; FE-3083 flips them to 308 later.** This branch combines every Workshop /models SEO PR, so merging it to main replaces merging them one by one. It also launches: Comfy-Org#19405 commits `launchedModelPages = 'all'`, so all 148 model pages become indexable in the same deploy as the URL move. Part of FE-2912, FE-2942 (PR 2 is FE-3086), FE-2974 (/minimax 308 is FE-3087) Fixes FE-2978, FE-2989, FE-2991, FE-2990, FE-2959, FE-2981, FE-2977, FE-2979, FE-3014, FE-2936, FE-2980, FE-2975, FE-3085, FE-2972, FE-2935, FE-2950, FE-2940, FE-2930, FE-2985, FE-2957, FE-2961, FE-2986, FE-2921, FE-2922, FE-2947 ## Changes Router follow-up: comfy-router#48 (`/hub/models.md` routing, 324/324 router tests), deploy before this merges. The included PRs, in merge order. Every head was checked as contained at the final tip. | PR | Ticket | What | Ben | |---|---|---|---| | Comfy-Org#18887 | FE-2978 | Pricing Product markup and full video `uploadDate` | approved | | Comfy-Org#18889 | FE-2989 | A unique meta description on every model page | approved | | Comfy-Org#18890 | FE-2991 | Model page JSON-LD, emitted once noindex lifts | approved | | Comfy-Org#18892 | FE-2990 | Cleaner tag pills, guard against duplicate summaries | approved | | Comfy-Org#18893 | FE-2959, 2981, 2977, 2979 | Social tags, hub title, Organization data. The FE-2977 title was confirmed by the owner | approved | | Comfy-Org#19362 | FE-3014 | Every non-production build is noindex | approved | | Comfy-Org#19366 | FE-2936 | Model titles lead with "API & Playground" | approved | | Comfy-Org#19402 | FE-2980 | Every internal link ends with a slash, plus a CI check | approved | | Comfy-Org#19419 | FE-2975 | Footer links the catalogue on every page and locale | approved | | Comfy-Org#19476 | FE-3085 | `NOINDEX_ROUTES` decides noindex for serverless-animation | approved; re-approval pending on a one-line follow-up (CodeRabbit approved) | | Comfy-Org#19359 | FE-2974 | One typed redirect list that generates `vercel.json` | approved | | Comfy-Org#19369 | FE-2972 A | The old to new model URL map | approved | | Comfy-Org#18891 | FE-2935 | Model content shows whatever the flag says | approved | | Comfy-Org#18899 | FE-2942 | Model pages render in the static HTML | approved | | Comfy-Org#18898 | FE-2950 | Catalogue h1 and A to Z links in the static HTML | approved | | Comfy-Org#19375 | FE-2940 | Related models leave no page unlinked | approved | | Comfy-Org#19385 | FE-2930 | Examples show the prompt that made them | approved | | Comfy-Org#19394 | FE-2985 | Render guard e2e: no loader, no layout shift | approved | | Comfy-Org#19357 | FE-2957 | robots.txt stops blocking our scripts and styles | approved | | Comfy-Org#19360 | FE-2961 | Turning Workshop off keeps every Models page | approved (a CodeRabbit changes-requested review is still open) | | Comfy-Org#19386 | FE-2986 | One registry of every Models address | approved | | Comfy-Org#19421 | FE-2972 B1 | Model pages served at `/hub/models` | approved | | Comfy-Org#19422 | FE-2972 B2 | Every old model URL redirects (307) | approved | | Comfy-Org#19424 | FE-2972 B3 | Workflow pages at `/hub/workflows` with redirects | approved | | Comfy-Org#19399 | FE-2921 | One switch decides model robots and sitemap listing | approved | | Comfy-Org#19404 | FE-2922 | Clean markdown twins for model pages | approved | | Comfy-Org#19405 | FE-2947 | Lifts noindex on every canonical model page | approved | | comfy-router#46 | FE-3075 | Serves `/hub/models*` from the website and routes `/hub/workflows` by manifest. Merged and live | merged | These fixes exist only on this branch, because they only break once two PRs meet: - `scripts/check-models-html.ts` reads each page at `model.href` and skips models without a page (f1df1e4). - `e2e/fixtures/modelsCatalogue.ts` reads `dist/hub/models` (f87bb4c). - `routerIdsByModelPage` accepts an optional `href`, since Comfy-Org#19421 made it optional and Comfy-Org#19405 no longer typechecked (9783819). - `ModelStatus` links a successor only when it has a page, for the same reason on Comfy-Org#18899 (c4761d8). - The model breadcrumb in `modelJsonLd.ts` points at `/hub/models/` instead of the `/models/` redirect (eb985b6). Its label still says "Models" while the nav says "Hub". - Twin, meta-description and entry tests use the hub pages and the new tag set (86ac3b0, d4ffc0f). - The render guard finds the catalogue through `getRoutes().workshop`, and its `test.fail` is gone now that Comfy-Org#18898 renders the h1 (831499f). - A new test renders a real model page and checks its markdown twin has the model's h1 (9a6c5cb). - Main's newer catalogue and workflow e2e cases use the hub addresses (2aeda77). - `resetWorkshopAccountSource` resets `decision`; a clean auto-merge of main's Comfy-Org#19454 into Comfy-Org#18891 left it assigning a variable that no longer exists (c28c7a7). The conflict resolutions, briefly: `ALTERNATE_TWIN_SOURCES` stays deleted as Comfy-Org#18898 wants, so `/hub/models.md` comes from the real catalogue. The hub keeps Comfy-Org#18898's single static h1 instead of the `WorkshopHero` h1 that main's Comfy-Org#19320 added. `ExamplesTab` keeps Comfy-Org#19385's figure and caption with main's video poster. The Comfy-Org#19399/Comfy-Org#19405 indexing tests were rewritten for hub paths. The Comfy-Org#19359 redirect test no longer builds `/enterprise//` from slash-ended routes. The Comfy-Org#19369 URL-map test imports `PROVIDER_NAMES` from its new home. The full table of 47 resolutions is in the integration report. ## Review Focus - Launch rides along with the URL move. If launch should stay a separate step, set `launchedModelPages` in `apps/website/src/config/model-page-launch.ts` to `new Set()` before merging. - Main's Comfy-Org#19320 gives the catalogue island its own h1 through `WorkshopHero`. This branch keeps Comfy-Org#18898's static h1 and renders only the subtitle in `ModelsCatalogue.vue`, so the hub has exactly one h1. - Comfy-Org#19476 overlaps the open community PR Comfy-Org#18591 (same sitemap exclusion for serverless-animation). One of them should be closed once this lands. ## QA - [x] Gates: `pnpm typecheck` and website typecheck clean, `pnpm lint` 0 errors, eslint on `apps/website` clean, `pnpm knip` clean - [x] Website unit: 547 files, 10347 passed, 4 skipped - [x] Build A (`build:e2e`, CI env): 1193 pages; release gate, `check:hreflang`, `validate:llms-txt-links` (225 links), `validate:jsonld` (947 pages), `check:link-slashes`, `check:models-html` (148 pages) all pass - [x] Build B (`WORKSHOP_IN_BUILD=0`): same 182 Models pages as the on build, no `dist/workshop`, `/` is noindex; with the flag stubbed on, no Run, no Hub nav and no Firebase requests - [x] Build C (preview shape): 945 of 945 pages noindex, no canonicals except redirect stubs, no hreflang in the sitemap - [x] E2E on Build A: 644 of 645 passed; the one mobile failure passed on rerun. Earlier load flakes (navigation, demos, fdct, wan-3.0, network isolation, render guard) all passed when rerun alone - [x] Launch: 148 model pages indexable with canonicals, 149 hub entries in the sitemap, 148 non-empty twins with the model h1, 148 in `llms-full.txt`, 0 titles over 60, 30 workflow pages still noindex - [x] Redirects: 853 rules, 782 to /hub all 307, the MiniMax rows 307, every /hub destination built - [x] comfy-router#46 under `wrangler dev` in front of this build: 312 router tests pass; 148 old model URLs (both slash forms) 307 then 200; 30 workflow names served by the website and their old URLs 307; a non-manifest workflow 302 with no-store; `manifest.json` passes through; `/workflows/` and bare `/hub` unchanged - [x] After the router deploys and this merges: purge Cloudflare, then curl `https://comfy.org/hub/models/` for a 200 and three old `/models/<slug>/` URLs for one 307 each - [x] Rich Results Test on `/pricing/` and `/learning/animations/character-sheet/`, then "Validate fix" on Merchant listings and Videos - [x] Rich Results Test on two live model pages: breadcrumbs valid and the breadcrumb URL returns 200 with no redirect - [x] GPTBot and Googlebot curls on the hub and one model page, then a GSC live test on both - [x] Resubmit the sitemap in Search Console and URL-inspect about 20 model pages on day 0 and day 14 - Not covered: Rich Results and GSC need production; the redirect matrix used a local shim for `vercel.json`, since `astro preview` ignores it --------- Co-authored-by: GitHub Action <action@github.com> Co-authored-by: benjcooley <benjcooley@gmail.com> Co-authored-by: Deep Mehta <42841935+deepme987@users.noreply.github.com>
|
Superseded by the integration branch #19494, which combined this PR with every other Workshop SEO slice and has been merged to main. Closing this one. |
Summary
Model pages now get their robots meta and their sitemap listing from one function,
isIndexableModelPage(), so the launch (FE-2947) flips one switch and both change together. The switch stays off here, so today's output is unchanged.Changes
isIndexableModelPage(pathname)inconfig/indexing.tsis true only whenMODEL_PAGES_INDEXABLEis on and the registry calls the address amodel.[slug].astroreads it fornoindex, andisExcludedFromSitemapreads it to let those pages through. Aliases, showcase,catalogue.json,page.json, workflow and app pages stay out either way. Sitemap URLs come from the built pages, so they already match each page's canonical (trailing slash) and follow the registry when the base path moves.<lastmod>yet. The model data has no per-page date, and build time orgit logon the shallow CI checkout would stamp every page the same. The ticket allows shipping inclusion without lastmod; the date source is raised on the ticket.Review Focus
sitemap-0.xml, every page's robots meta and the twin list are byte-identical to the base.modelpages join the sitemap and lose noindex; nothing else moves.vi.mock.Fixes FE-2921
https://linear.app/comfyorg/issue/FE-2921/add-passing-model-pages-to-the-sitemap-with-real-update-dates
Stacked on #19386