Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .fallowrc.jsonc
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
"index.html",
"scripts/**/*.{js,ts}",
"apps/website/src/scripts/**/*.ts",
// Astro injects these routes when WORKSHOP_IN_BUILD is enabled.
"apps/website/src/routes/models/*.{astro,ts}",
"**/.storybook/{main,preview}.ts",
"packages/ingest-types/openapi-ts.config.ts"
],
Expand All @@ -29,7 +31,9 @@
// Loaded by config, not imported: oxlint jsPlugin (.oxlintrc.json) and
// stylelint customSyntax (.stylelintrc.json) respectively.
"eslint-plugin-playwright",
"postcss-html"
"postcss-html",
// Executed by models-snippets.test.ts inside the generated Node examples.
"mime-types"
],

"ignoreUnresolvedImports": [
Expand Down
7 changes: 7 additions & 0 deletions apps/website/MODELS_CONTENT_FORMAT.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,13 @@ preserved in the packed file, not added to the UI.
Old model-only URLs redirect to a deterministic current record. Detail lookup,
sample ownership and saved drafts use the content slug, not only the Router ID.

Corrections to a published page's catalogue category live in
`src/data/workshop-use-case-overrides.json`, keyed by verified Router ID. They
change its search/category/task labels while preserving the published URL,
example ownership and production-test ledger. The OpenAI GPT Image routes
accept text-to-image generation, so their imported `edit-images` categorization
is corrected to `generate-images`. The build rejects unknown IDs or categories.

## Media policy

A thumbnail or output sample URL may not be published in two different use
Expand Down
2 changes: 1 addition & 1 deletion apps/website/MODELS_INPUT_SANITY_REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Prefer a small, obvious form over complete exposure of the provider schema.
## Evidence and false-positive rules

- Review the current worktree, not the older PR preview or previous conversation.
Start with `routerWorkshopModels`, `getRouterWorkshopModelDetail`,
Start with `workshopModels`, `getRouterWorkshopModelDetail`,
`schemaForModel`, `defaultValues`, `validateForm` and
`prepareWorkshopRouterInput`. Inspect actual per-use-case slugs.
- Check both the curated contract and the pinned native OpenAPI snapshot.
Expand Down
43 changes: 37 additions & 6 deletions apps/website/MODELS_INPUT_SCHEMA.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ HeyGen's curated language choices correspond to languages in the
Language and locale remain unset for automatic detection; a selected locale
overrides language, as described in the
[HeyGen speech contract](https://developers.heygen.com/docs/voices/speech).
Voice ID stays editable in Advanced with the existing tested voice.
Voice ID is hidden and fixed to the existing tested account voice in the
input-presentation data; creators cannot choose an unavailable account voice.

Model price estimates prefer the generated node pricing rules. Models whose
nodes have no price badge can use verified flat rates from
Expand Down Expand Up @@ -332,11 +333,24 @@ with `pnpm --filter @comfyorg/website generate:workshop-router-contracts`.
- Multiple Qwen reference inputs become ordered native image entries; Base64
arrays and first/last-frame mappings remain unchanged. This does not add every
optional provider media mode or override provider-specific limits.
- Python/TypeScript examples use the same storage handshake for URL files and
local Base64 encoding for Base64 files. cURL continues to omit local files and
marks the request incomplete. For script retries, reuse the prepared URLs and
key; rerunning upload setup creates different request URLs. Signed URLs and
keys must not be logged, checked in, or put into the content pack.
- Python/TypeScript examples use `comfy-sdk==0.2.0` / `@comfyorg/sdk@0.2.0`.
URL-capable local inputs use SDK assets; native inline-only inputs retain
Base64 with MIME inferred from the supplied file or HTTP response. Default
source URLs remain runnable without inventing local filenames. Preparing the
API tab does not read private file bytes or download example media.
Embedded default media retains its original data URI: inline requests use
those known bytes, and URL inputs use the SDK's `assets.from_bytes` /
`assets.fromBytes`. User-selected files still refer to local paths.
- JSON endpoints use the SDK's `models.run`, which owns request identity and
retries. SDK 0.2.0 cannot decode binary responses, so binary/unknown-output
endpoints retain HTTP examples that check errors and save non-JSON bytes.
cURL generates a fresh key on execution, preserves usable source URLs, and
warns when it omits file inputs. Retrying a request requires its original
prepared body and key. Credentials are checked before accessing inputs.
- SDK examples use the same environment family as the page, with separate
`COMFY_ROUTER_BASE_URL` and `COMFY_BASE_URL` overrides. SDK dependencies are
development-only: tests execute generated code; they add no browser imports.
Signed URLs and keys must not be checked in or put into the content pack.

**Live verification is blocked, not certified.** On September 10, both
`api-nodes-prod` and `api-nodes-staging` returned HTTP 200 without any
Expand Down Expand Up @@ -442,6 +456,23 @@ Content identity and unfinished per-use-case widget customization are documented
in [Models content format](MODELS_CONTENT_FORMAT.md). Live render tests remain
paused while these input definitions are tuned.

## Sign-in drafts

Signed-out visitors can select local media before signing in. The same
pre-navigation hook serves the header and Playground: scalar edits flush to
session storage, while media is saved to IndexedDB before same-tab navigation.
No file is uploaded until the authenticated Run path prepares the request.

Each tab stores an opaque draft key in session storage. Media drafts are consumed
on return, validated against the current input schema, and usable for one hour.
Writes prune expired entries and retain at most four drafts, each capped at
100 MiB. Entries left behind by an abandoned login are pruned on a later write;
there is no background deletion timer. Document-owned Blob URLs are never stored.
The storage operation has a five-second deadline and aborts with its component.
Repeated sign-in clicks share one save; completion cannot redirect a departed
page. If storage fails, sign-in proceeds and the return page asks the visitor to
reselect missing files instead of silently submitting default media.

## Verification boundaries

During input tuning, use a short change → focused test/browser check → fix loop.
Expand Down
38 changes: 38 additions & 0 deletions apps/website/e2e/models-catalogue-corrections.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import { expect } from '@playwright/test'

import { test } from './fixtures/modelsAccount'

test('GPT Image generation pages are discoverable as text-to-image while published URLs remain valid', async ({
page
}) => {
const hydrated = Promise.withResolvers<void>()
const moduleRequested = page.waitForRequest(
'**/_website/ModelsCatalogue.*.js'
)
await page.route('**/_website/ModelsCatalogue.*.js', async (route) => {
await hydrated.promise
await route.continue()
})
try {
await page.goto('/models/?useCase=generate-images', { waitUntil: 'commit' })
await moduleRequested
await expect(page.getByTestId('workshop-search')).toBeDisabled()
} finally {
hydrated.resolve()
}
await page.getByTestId('workshop-search').fill('gpt image')
const cards = page
.getByTestId('workshop-models-grid')
.getByTestId('workshop-model-card')
await expect(cards).toHaveCount(3)
await page.getByRole('link', { name: /GPT Image 1\.5/ }).click()
await expect(page).toHaveURL(
/\/models\/openai--gpt-image-1\.5--edit-images\/$/
)
await expect(
page.getByRole('textbox', { name: 'Prompt', exact: true })
).toBeVisible()
await page.goto('/models/?useCase=edit-images')
await page.getByTestId('workshop-search').fill('gpt image')
await expect(cards).toHaveCount(0)
})
101 changes: 101 additions & 0 deletions apps/website/e2e/models-performance.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
import type { Page } from '@playwright/test'
import { expect } from '@playwright/test'

import { test } from './fixtures/modelsAccount'

/** Same-origin script resources the page has requested so far, by URL. */
function requestedScripts(page: Page) {
return page.evaluate(() =>
performance
.getEntriesByType('resource')
.filter(
(entry): entry is PerformanceResourceTiming =>
entry instanceof PerformanceResourceTiming
)
.flatMap((entry) => {
const url = new URL(entry.name)
return url.origin === location.origin && url.pathname.endsWith('.js')
? [[entry.name, entry.decodedBodySize] as const]
: []
})
)
}

test('catalogue browsing stays within its JavaScript budget', async ({
page
}) => {
// Every page pays for the layout islands (header, footer, translations,
// analytics). Charge the catalogue only for what it adds on top of those,
// so a header change cannot fail this test and a catalogue regression
// cannot hide behind a shrink elsewhere.
// The default resource-timing buffer holds 250 entries and drops the rest
// silently; a late catalogue chunk must not fall off the end.
await page.addInitScript(() => performance.setResourceTimingBufferSize(2000))
await page.goto('/')
const shared = new Set((await requestedScripts(page)).map(([name]) => name))

await page.goto('/models/')
await page.getByTestId('workshop-search').fill('kling')
await page.getByRole('heading', { level: 1 }).click()
await expect(
page
.getByTestId('workshop-models-grid')
.getByTestId('workshop-model-card')
.first()
).toContainText('Kling')
const scripts = await requestedScripts(page)
const catalogueBytes = scripts
.filter(([name]) => !shared.has(name))
.reduce((total, [, bytes]) => total + bytes, 0)
expect(catalogueBytes).toBeGreaterThan(0)
expect(catalogueBytes).toBeLessThan(900_000)
// The whole-route ceiling stays as well: catalogue code that migrated into
// a chunk the homepage also loads would pass the delta and still be paid
// for by every visitor to /models/.
const totalBytes = scripts.reduce((total, [, bytes]) => total + bytes, 0)
expect(totalBytes).toBeLessThan(1_800_000)
})

test('video cards load on screen and stop playing when scrolled away', async ({
page
}) => {
await page.goto('/models/')
await page.getByTestId('workshop-filter').click()
const videos = page.getByTestId('section-generate-videos').locator('video')
await expect.poll(() => videos.count()).toBeGreaterThan(0)
await expect
.poll(() =>
videos.evaluateAll((elements: HTMLVideoElement[]) =>
elements.every((video) => video.currentSrc === '' && video.paused)
)
)
.toBe(true)

const first = videos.first()
await first.scrollIntoViewIfNeeded()
await expect
.poll(() =>
first.evaluate(
(video: HTMLVideoElement) => video.readyState >= 2 && !video.paused
)
)
.toBe(true)
await page.getByRole('heading', { level: 1 }).scrollIntoViewIfNeeded()
await expect
.poll(() =>
videos.evaluateAll((elements: HTMLVideoElement[]) =>
elements.every((video) => video.paused)
)
)
.toBe(true)
await page.emulateMedia({ reducedMotion: 'reduce' })
Comment thread
benjcooley marked this conversation as resolved.
Resolved
await first.scrollIntoViewIfNeeded()
// The source is attached (so the frame can show) before playback would have
// started; asserting `paused` earlier than that proves nothing.
await expect
.poll(() => first.evaluate((video: HTMLVideoElement) => video.readyState))
.toBeGreaterThanOrEqual(1)
await expect
.poll(() => first.evaluate((video: HTMLVideoElement) => video.paused))
.toBe(true)
})
Loading
Loading