Skip to content

feat(website): index the Models pages and describe them to search engines - #18480

Closed
benjcooley wants to merge 1 commit into
bencooley/fe-2770-models-static-contentfrom
bencooley/fe-2770-models-indexable
Closed

benjcooley wants to merge 1 commit into
bencooley/fe-2770-models-static-contentfrom
bencooley/fe-2770-models-indexable

Conversation

@benjcooley

Copy link
Copy Markdown
Collaborator

Summary

Every canonical /models/<slug> is now indexable, in the sitemap, and carries structured data; stacked on #18462 so the first crawl of each URL sees the static content, not the old showcase.

Changes

  • What:
    • routes/models/[slug].astro: noindex removed. Adds breadcrumbs (Home › Models › name), mainEntityId, and a SoftwareApplication node with the provider as an Organization and an Offer priced from the estimate the page shows, converted at the published CREDITS_PER_USD rate, with the credits label as its description. No offer when the estimate is not a single number (variable-priced models) — 32 of 148 pages get one today. The catalogue carries no priceUsdFrom/creditsPerRun for any model, so those fields were not usable.
    • routes/models/index.astro: CollectionPage with breadcrumbs and an ItemList of all 148 models.
    • config/indexing.ts: replaces the blanket /models/* sitemap exclusion with an explicit set of alias redirect stubs (routerModelSlugAliases); /models/showcase and the checkout routes stay excluded via NOINDEX_PATHNAMES. isWorkshopRoute() is gone with its only caller. Built sitemap: 148 canonical model pages + /models/ + /zh-CN/models/; zero aliases, showcase, checkout or JSON routes. Alias stubs keep Astro's own noindex.
    • utils/jsonLd.ts: softwareApplicationNode accepts providerName and a paid offer.
    • lib/workshop/price-label.ts: offerForEstimate() (pure, tested).
    • lib/markdown-twin.ts: drops role="status" regions so the playground's loading frame never reads as page content; the model hero is a <section> rather than a <header> so the twin keeps it. Each model page now gets a .md twin with the hero, price, tags and related models.
  • Breaking: none for visitors. isWorkshopRoute removed (no other callers).

Review Focus

  • hreflang: /models is already in LOCALE_INVARIANT_ROUTE_KEYS, so detail pages emit no zh-CN alternate — no new guard needed. (Pre-existing and out of scope: that also leaves /models/ and /zh-CN/models/ unclustered.)
  • The Offer's price is derived, not authored: 14.8 credits/Run → $0.07 at 211 credits/USD. If that rate ever moves, creditsUtil is the single source and the offer follows it.
  • indexing.ts now imports workshop-browse-content (loaded by astro.config.ts at config time). It is JSON + zod only; the build ran clean.

Refs FE-2770 (https://linear.app/comfyorg/issue/FE-2770) · closes FE-2026 (https://linear.app/comfyorg/issue/FE-2026)

Screenshots (if applicable)

No visible change. Crawler-visible additions on /models/bfl--flux-2-max--generate-images/:

<meta name="robots">                       (gone)
BreadcrumbList: Home › Models › FLUX 2 Max Text-to-Image
SoftwareApplication: provider Black Forest Labs,
  offers { price: 0.07, priceCurrency: USD, description: "14.8 credits/Run" }
sitemap-0.xml: <loc>https://comfy.org/models/bfl--flux-2-max--generate-images/</loc>

…ines

The model pages were noindex and kept out of the sitemap as the
pre-launch gate, and nothing turned that off at launch. Every canonical
/models/<slug> is now indexable and in the sitemap; the alias stubs,
showcase and checkout routes stay out.

Each model page declares a SoftwareApplication with its provider, the
breadcrumb to the catalogue, and an Offer priced from the estimate the
page shows at the published credit rate, only when that estimate is one
number. The catalogue declares an ItemList of every model.

The markdown twin now drops status regions, so a loading frame never
reads as page content, and the model hero is a section rather than a
header so the twin keeps it.

Refs FE-2770, closes FE-2026
@benjcooley
benjcooley requested review from a team and dante01yoon as code owners September 22, 2026 23:59
@vercel

vercel Bot commented Sep 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
billing-web Ignored Ignored Sep 22, 2026 11:59pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 156053a6-af5d-4565-88e2-2d4a529e2bda

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

🌐 Website E2E

Tip

All tests passed.

Status ✅ Passed
Report View Report

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

This candidate changes nothing the matrix measures, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

📦 Bundle Size

⏳ Size data collection in progress…

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since 1c989a236d7f13b54a3566962af63c1bdf2be351

Full docs: ADR-DEVEX-LINT-0015

🔗 Website Preview

Website Preview: https://comfy-website-preview-pr-18480.vercel.app

Models is built and gated by PostHog (workshop-enabled), using staging Cloud.

This commit: https://website-frontend-fm68wqjgb-comfyui.vercel.app

Last updated: 2026-09-23T00:01:18Z for a8cc7fe

@codecov

codecov Bot commented Sep 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@                           Coverage Diff                            @@
##           bencooley/fe-2770-models-static-content   #18480   +/-   ##
========================================================================
  Coverage                                    83.35%   83.35%           
========================================================================
  Files                                         2957     2957           
  Lines                                       146822   146828    +6     
  Branches                                     39796    39799    +3     
========================================================================
+ Hits                                        122380   122386    +6     
  Misses                                       23201    23201           
  Partials                                      1241     1241           
Flag Coverage Δ
website-unit 75.85% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
apps/website/src/config/indexing.ts 100.00% <100.00%> (ø)
apps/website/src/config/workshop-release.ts 100.00% <ø> (ø)
apps/website/src/lib/markdown-twin.ts 81.90% <ø> (ø)
apps/website/src/lib/workshop/price-label.ts 100.00% <100.00%> (ø)
apps/website/src/utils/jsonLd.ts 95.89% <100.00%> (+0.11%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Sep 23, 2026
if (!estimate) return undefined
const match = /^~?\s*(\d+(?:\.\d+)?)\s*credits(?:\/|$)/.exec(estimate)
if (!match) return undefined
return { price: creditsToUsd(Number(match[1])), description: estimate }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: creditsToUsd rounds to whole cents, so anything under about 1.06 credits prices at $0. Two pages ship that today: luma--photon-flash-1-image-generation--generate-images and luma--photon-flash-1-image-modify--edit-images show "0.57 credits/Run" and publish "offers":{"price":0,...}, which Google shows as Free. Eleven of the 32 offers are also off by more than 1%, up to 11%: bria--remove-image-background at 3.8 credits publishes $0.02 for a true $0.018, and the three seedream-5-pro pages publish $0.05 for ~9.5 credits ($0.045). The '~9.5 credits/Image' → 0.05 row in price-label.test.ts encodes that rounding as correct. schema.org price takes decimals, so could offerForEstimate divide by CREDITS_PER_USD and round to 4 places instead, with a '0.57 credits/Run' row asserting a non-zero price?

title="Models - Comfy"
title={title}
description="Run the world's leading AI models in ComfyUI. Browse every supported model with community workflow templates ready to run."
pageType="CollectionPage"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: this turns /models/ into a CollectionPage with an ItemList of all 148 models, but its markdown twin /models.md (and its section of llms-full.txt) is still built from the showcase: it's headed "Grok Imagine in ComfyUI" and links no models. The cause is ALTERNATE_TWIN_SOURCES in integrations/markdown-twins.ts:39 (['models', 'models/showcase']), which predates this PR and made sense while /models/ was the showcase. Now an answer engine reading the twin gets a different page from the one we describe to Google. Could that entry go, so /models.md is built from /models/ itself?

Comment on lines +66 to +71
expect(graph.find((node) => node['@type'] === 'SoftwareApplication')).toEqual(
expect.objectContaining({
name,
offers: expect.objectContaining({ priceCurrency: 'USD' })
})
)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick (non-blocking): offers: expect.objectContaining({ priceCurrency: 'USD' }) still passes with price: 0, the exact case above, and the toEqual on the five @types (line 59) fails whenever a node is added even if nothing regressed. Could this assert offers.price is above 0, and use arrayContaining for the types?

@MaanilVerma

Copy link
Copy Markdown
Collaborator

Tested on a build with #18462 and current main merged in, on this PR's Vercel preview, and on a local build with #18484 on top. The probe walks every model in the build's own catalogue.json, not just the URLs in the description.

The indexing side is solid. All 148 model pages are indexable with the right canonical, the sitemap holds exactly those 148 plus both indexes (no alias, showcase, checkout, .json or .md, and nothing prod serves went missing), all 212 alias stubs stay noindex and point at the right page, and no hreflang points at a 404. JSON-LD parses on every page with correct breadcrumbs and SoftwareApplication fields, the ItemList has all 148, and a </script> in a model name can't break out of the script tag.

Check Result
148 pages indexable, canonical correct pass
Sitemap contents, nothing dropped vs prod pass
212 alias stubs pass
hreflang pass
JSON-LD structure and fields pass
Offer only on single-number prices pass, 32 offers
Offer price never $0 fail, 2 pages, see inline (price-label.ts)
Offer price precision 11 of 32 off by up to 11%, same inline
e2e would catch a $0 offer no, see inline (spec)
Markdown twins, 148 pages pass
/models.md describes the catalogue fail, see inline (index.astro)
Unit / touched e2e pass / 16 of 16

Not tied to a line: apps/website/README.md:206 still says these pages stay out of sitemaps and markdown exports. And once these pages are indexed, a WORKSHOP_IN_BUILD=0 build would 404 all 148 (the release gate refuses any /models/ entry but index.html). Prod builds with 1, so it's not live, but FE-2770 says a kill switch never unpublishes. Worth changing the gate, or is that switch retired?

@dante01yoon dante01yoon left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two crawler-visible outputs do not match the intended indexed model catalogue yet. I confirmed these against the current head and the catalogue examples in the PR description.

if (!estimate) return undefined
const match = /^~?\s*(\d+(?:\.\d+)?)\s*credits(?:\/|$)/.exec(estimate)
if (!match) return undefined
return { price: creditsToUsd(Number(match[1])), description: estimate }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: creditsToUsd rounds to cents before this Offer is emitted, so low-cost models publish price: 0 and are represented to search engines as free. For example, the catalogue's 0.57-credit runs convert below half a cent. Could the structured-data price retain enough precision to remain non-zero (and add that case to the test) rather than reuse the display-price rounding?

title="Models - Comfy"
title={title}
description="Run the world's leading AI models in ComfyUI. Browse every supported model with community workflow templates ready to run."
pageType="CollectionPage"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: this describes /models/ as a 148-item catalogue, but the existing ALTERNATE_TWIN_SOURCES mapping still builds /models.md from /models/showcase. Answer engines therefore receive the Grok showcase instead of this catalogue. Could this change also remove/update that alternate twin mapping so the HTML, JSON-LD, and markdown representations agree?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants