Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
93da152
feat(litegraph): prove a node's widget form rather than deriving it
claude Oct 26, 2026
8cfcda4
fix(litegraph): refuse when the node's class can still rewrite the pa…
claude Oct 27, 2026
3adac83
refactor(litegraph): split the payload preconditions out of provenValues
claude Oct 27, 2026
cc6195e
docs(adr): stop claiming test coverage this PR does not have
claude Oct 29, 2026
4005a86
feat(census): project a real object_info into a node-shape fixture
christian-byrne Oct 29, 2026
8fd25bd
feat(census): prove widget forms over every registered class
christian-byrne Oct 29, 2026
ec1badb
feat(census): report the widget-form population from the rows
christian-byrne Oct 29, 2026
6b904af
fix(deps): exclude the generated matrix tree from depcruise
christian-byrne Oct 29, 2026
790b7b6
chore(tooling): register the census config as an entry point
christian-byrne Oct 29, 2026
302cf2e
docs(census): document the widget-form census and its three populations
christian-byrne Oct 29, 2026
d26c12b
fix(census): keep the baseline row out of the gated verdict's population
christian-byrne Oct 29, 2026
74643fc
fix(census): roll the pack delta up per pack, and report needsFormBut…
christian-byrne Oct 29, 2026
a5c4c1b
[automated] Apply ESLint and Oxfmt fixes
actions-user Oct 29, 2026
ccc56c5
fix(census): keep the baseline census out of test:unit
christian-byrne Oct 29, 2026
d3e233b
fix(census): exclude the generated object_info artefacts from oxfmt
christian-byrne Oct 29, 2026
38e64bf
perf(census): give the matrix shards headroom for the census, and say…
christian-byrne Oct 29, 2026
0557840
fix(census): satisfy the two repo-checks the census broke
christian-byrne Oct 29, 2026
01d923f
fix(litegraph): refresh widget form proof checks
christian-byrne Oct 30, 2026
aa73dff
refactor(litegraph): strip the rationale out of the code and into the…
christian-byrne Oct 30, 2026
6ad64a7
docs(litegraph): keep widget proof contracts concise
christian-byrne Oct 30, 2026
a5b081e
Merge remote-tracking branch 'origin/main' into pr22007-fe22007
christian-byrne Oct 30, 2026
2101005
chore(architecture): register widget form proof test
christian-byrne Oct 30, 2026
e28fafe
Merge current widget-form predicate into registry census
christian-byrne Oct 31, 2026
b253b7c
docs(registry-census): align proof reporting schema
christian-byrne Oct 31, 2026
bbb9b99
fix(registry-census): adapt proof channel to current predicate
christian-byrne Oct 31, 2026
db87b4c
chore(registry-census): register census support files
christian-byrne Oct 31, 2026
d5a0f37
chore(registry-census): keep census config concise
christian-byrne Oct 31, 2026
73cf472
Merge remote-tracking branch 'origin/main' into wfp
christian-byrne Nov 2, 2026
e4521af
fix(litegraph): type the widget form proof as a union and refuse an e…
christian-byrne Nov 2, 2026
652d799
Merge remote-tracking branch 'origin/widget-form-proof-predicate' int…
christian-byrne Nov 2, 2026
83a2f2c
refactor(census): read the refusal straight off the union proof
christian-byrne Nov 2, 2026
f3149fd
fix(litegraph): report length_mismatch before the empty-order refusal
christian-byrne Nov 2, 2026
59c7af8
fix(census): record formAgreement when serialize throws
christian-byrne Nov 2, 2026
1f5804a
fix(litegraph): require replayed, drop the payload has_errors read, t…
christian-byrne Nov 2, 2026
7103ad0
Merge branch 'wfp' into census
christian-byrne Nov 2, 2026
6d4ef61
fix(census): make registry sweep opt in
christian-byrne Nov 3, 2026
d5d9d82
fix(census): gate per-pack registry forms
christian-byrne Nov 3, 2026
fac9110
fix(repo-checks): remove census rationale comment
christian-byrne Nov 3, 2026
515b336
chore: resolve merge conflicts with main
christian-byrne Nov 4, 2026
63bc7be
fix(ci): give the census shards the opt-in budget, not the corpus job
christian-byrne Nov 4, 2026
60142bf
refactor(census): drop the double assertion on the probe's custom widget
christian-byrne Nov 4, 2026
09adef0
test(census): compare the fixture against the projection in one asser…
christian-byrne Nov 4, 2026
edecb8c
test(census): assert the subgraph probes as one collection
christian-byrne Nov 4, 2026
a2b75d9
fix(census): move the census config rationale into the README
christian-byrne Nov 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .dependency-cruiser.json
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,12 @@
"tsPreCompilationDeps": "specify",
"doNotFollow": { "path": "node_modules" },
"exclude": {
"path": ["\\.(test|spec|stories)\\.ts$", "/__tests?__/", "/test/"]
"path": [
"\\.(test|spec|stories)\\.ts$",
"/__tests?__/",
"/test/",
"^src/__ecs_matrix__/"
]
},
"enhancedResolveOptions": {
"extensions": [".ts", ".mts", ".js", ".vue", ".json"],
Expand Down
3 changes: 2 additions & 1 deletion .fallowrc.jsonc
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@
"**/.storybook/{main.ts,preview.ts,style.css}",
"packages/comfyui-api-types/openapi-ts.config.ts",
"packages/ingest-types/openapi-ts.config.ts",
"scripts/**/*.{js,ts}"
"scripts/**/*.{js,ts}",
"vitest.*.config.mts"
],
"entryPointRole": "support"
},
Expand Down
12 changes: 10 additions & 2 deletions .github/workflows/ci-ecosystem-matrix.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,12 @@ on:
- '!.storybook/**'
- '!**/*.md'
workflow_dispatch:
inputs:
registry_forms:
description: Run the opt-in per-pack widget-form census
required: false
default: false
type: boolean

permissions:
contents: read
Expand Down Expand Up @@ -253,7 +259,7 @@ jobs:
needs: corpus
if: github.repository == 'Comfy-Org/ComfyUI_frontend'
runs-on: ubuntu-latest
timeout-minutes: 20
timeout-minutes: ${{ inputs.registry_forms && 40 || 20 }}
strategy:
fail-fast: false
matrix:
Expand All @@ -262,6 +268,7 @@ jobs:
env:
MATRIX_RENDERER: ${{ matrix.renderer }}
MATRIX_VUE: ${{ matrix.renderer == 'vue' && '1' || '' }}
MATRIX_REGISTRY_FORMS: ${{ inputs.registry_forms && '1' || '' }}
steps:
# This job EXECUTES unreviewed third-party pack code under vitest;
# the checkout must not leave a usable git credential behind for it.
Expand Down Expand Up @@ -343,7 +350,7 @@ jobs:
row_names=()
for f in matrix-rows/*.json; do
name=$(basename "$f" .json)
case "$name" in _manifest*) continue ;; esac
case "$name" in _manifest*|_registry-baseline) continue ;; esac
row_names+=("$name")
done
echo "specs=$built rows=${#row_names[@]}"
Expand Down Expand Up @@ -448,6 +455,7 @@ jobs:
env:
MATRIX_RUN_ID: ${{ github.run_id }}
MATRIX_EXPECT_SHARDS: '4'
MATRIX_EXPECT_REGISTRY_FORMS: ${{ inputs.registry_forms && '1' || '' }}
run: |
set +e
MATRIX_OUT="$PWD/matrix-rows" \
Expand Down
2 changes: 2 additions & 0 deletions .oxfmtrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@
"public/materialdesignicons.min.css",
"scripts/registry-census/corpus.pins.json",
"scripts/registry-census/detection-proof/**",
"scripts/registry-census/object_info.fixture.json",
"scripts/registry-census/object_info.shape.json",
"src/__ecs_matrix__/**",
"src/types/generatedManagerTypes.ts",
"src/locales/**/*.json",
Expand Down
1 change: 1 addition & 0 deletions knip.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ const config: KnipConfig = {
entry: [
'{build,scripts}/**/*.{js,ts}',
'vitest.matrix.config.mts',
'vitest.census.config.mts',
'src/assets/css/style.css',
'src/scripts/ui/menu/index.ts',
'src/types/index.ts',
Expand Down
35 changes: 27 additions & 8 deletions scripts/cicd/ecosystem-matrix-eligibility.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ const workflowSchema = z.object({
jobs: z.record(
z.string(),
z.object({
'timeout-minutes': z.union([z.number(), z.string()]).optional(),
outputs: z.record(z.string(), z.string()).optional(),
steps: z
.array(
Expand Down Expand Up @@ -50,17 +51,19 @@ const bumpBranch = z
.min(1)
.parse(createPullRequestStep().with?.['branch'])

function evaluate(expression: string, context: object) {
function run(expression: string, context: object): unknown {
const javascript = expression
.slice(3, -2)
.replace(/\.([a-zA-Z_][\w-]*)/g, '?.["$1"]')
return z.boolean().parse(
runInNewContext(javascript, {
...context,
startsWith: (haystack: unknown, needle: unknown) =>
String(haystack ?? '').startsWith(String(needle ?? ''))
})
)
return runInNewContext(javascript, {
...context,
startsWith: (haystack: unknown, needle: unknown) =>
String(haystack ?? '').startsWith(String(needle ?? ''))
})
}

function evaluate(expression: string, context: object) {
return z.boolean().parse(run(expression, context))
}

interface Candidate {
Expand Down Expand Up @@ -140,6 +143,22 @@ describe('ecosystem matrix candidate eligibility', () => {
})
})

describe('ecosystem matrix shard budget', () => {
it.for([
['an ordinary run', {}, 20],
['a manual dispatch without the census', { registry_forms: false }, 20],
['a manual registry-forms census', { registry_forms: true }, 40]
] satisfies [string, object, number][])(
'budgets the pack shards for %s',
([, inputs, minutes]) => {
const budget = z
.string()
.parse(matrix.jobs['ecosystem-matrix']['timeout-minutes'])
expect(run(budget, { inputs })).toBe(minutes)
}
)
})

describe('corpus pin bump PR creation', () => {
it('opens the PR with PR_GH_TOKEN', () => {
expect(createPullRequestStep().with?.['token']).toBe(
Expand Down
189 changes: 177 additions & 12 deletions scripts/registry-census/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,13 @@ python3 scripts/registry-census/fetch_corpus.py --write-pins # bump the pins
python3 -m unittest discover -s scripts/registry-census # verdict unit tests
```

**`deps:audit` is `depcruise src`, and `build_matrix.py` writes ~1.1GB of
third-party pack JS into `src/__ecs_matrix__/`.** `tsconfig.json`,
`.oxfmtrc.json`, `.oxlintrc.json` and `knip.config.ts` already exclude that
path; `.dependency-cruiser.json` does too, as of the widget-form census.
Without it, following the instructions above makes the repo unpushable with an
OOM that `--max-old-space-size` does not save.

Pure stdlib; needs `curl` and `tar` on PATH, plus `git` for `--write-pins`.

## In CI
Expand Down Expand Up @@ -146,6 +153,156 @@ is `setup(app)` — `api.addEventListener`, patching `LGraphCanvas.prototype`,
wrapping `app.graphToPrompt`, adding canvas menus — scores as a fully clean
row. A green matrix did not catch a `setup`-dispatch regression, and cannot.

## The widget-form census (telemetry rung)

`proveWidgetForm` (`src/lib/litegraph/src/widgets/widgetFormProof.ts`) gates
emitting `widgets_values_form` — a per-node declaration of which widgets
serialize and in what order. A node carrying one is addressed through that form
and consults no widget catalog on any leg, so a **wrong form is authoritative**
and silently misplaces values. It therefore proves rather than derives: emit
only where core can prove the final `widgets_values` is the array it itself
produced from `node.widgets`.

This instrument is the only thing that can give that rule a population number,
because deriving an order breaks on precisely the mutations the battery already
performs (`wSplice`, `wReorder`, `wConvert`, `wPushPojo`,
`wPushForeignClass`) — which is why the predicate checks array **identity**, not
deep equality.

**None of it gates.** A refusal is the predicate working as designed on a shape
it was built to decline, so a floor would fail the run for correct behaviour.

### Three populations. Do not quote one as another.

| channel | population | where |
| --------------------------------------- | ------------------------------------------------------------------------------------- | ------------------------------------ |
| `registryForms` on `_registry-baseline` | every class in `object_info.shape.json`, no pack code | `vitest.census.config.mts`, run once |
| `registryForms` on a pack row | the same sweep inside that pack's process; read as a **delta** against the baseline | opt-in pack row |
| `formProof` on a pack row | the battery's standard-workflow nodes, and the pack's own `registerCustomNodes` types | every pack row |

```bash
# 1. the baseline: every registry class, no pack code (~8s)
CENSUS_FORMS_OUT=.census/results/rows/_registry-baseline.json \
pnpm exec vitest run --config vitest.census.config.mts
# 2. the pack rows (opt in because the 3,857-class sweep is expensive)
MATRIX_REGISTRY_FORMS=1 MATRIX_OUT=.census/results/rows \
pnpm exec vitest run --config vitest.matrix.config.mts
# 3. the report
python3 scripts/registry-census/census_forms.py .census/results/rows
```

### Where the node definitions come from

`object_info.shape.json` is a **node-shape projection** of a real `/object_info`
dump — 3,857 classes, 998 from core and 2,859 from 97 packs. It exists because
`matrix_runner.ts` carries six hardcoded defs for its battery, and six classes
is not a registry: without an `object_info`, every Python-defined pack node
class is invisible to the census, which is where widget-shape diversity
actually lives.

The source dump lives in `Comfy-Org/cloud`
(`services/ingest/data/object_info.json`) — the same dump the agent image bakes.
That repo is **private** and this one is **public**, and a pack-executing job
may hold no credential, so the projection is committed rather than fetched.
`object_info.pin.json` records the source commit's sha256 and the population;
`project_object_info.py` regenerates both and documents what the projection
drops (prose and presentation) and what it keeps (everything that changes a
widget). Verify a bump two ways: `--verify` for byte-reproducibility from the
pinned dump, and a digest comparison between a census run against the full dump
and one against the projection for verdict fidelity.

`object_info.fixture.json` is a seven-class slice of the projection, used so the
runner's own tests stay fast while still registering real definitions. Both test
suites assert it is byte-identical to its entries in the projection; a drifted
fixture would make every verdict measured through it a claim about a node shape
that does not exist.

### Why the census runs last, after every gated channel

When `MATRIX_REGISTRY_FORMS=1`, registering 3,857 defs runs each pack's
`beforeRegisterNodeDef` 3,857 times
instead of six. **`packs free of contained hook errors` is a gated criterion
with a measured baseline**, so folding those firings into it would move a
required gate for a reason no pull request caused. So:

- the battery still registers its own six defs, in the same place, and every
gated channel's denominator and baseline are unchanged;
- the census registers the real projection afterwards, on a fresh graph, and
keeps its own `hookErrors` — a finding, not a floor;
- `row.hookErrors` is read off **before** the census phase, so nothing it does
can reach a gated channel;
- both census functions are total. An escaping throw would leave the
write-ahead stub in place, and **stub rate is itself gated** — a telemetry
channel must not be able to cost a row.

The cost of that placement, stated because it is a real fidelity loss: the app's
order is `addCustomNodeDefs` → `registerNodeDef` → `registerCustomNodes`, and
the census registers _after_ `registerCustomNodes`. The harness already deviates
from that order (see the hook table above); the census deviates by one step
further.

### What it costs, because it is not free

Registering 3,857 classes and instantiating each one costs **~5.3s of CPU per
pack** on top of the battery's ~1.5s — measured over the detection-proof corpus
at four workers: 16.03s user CPU for 11 packs with the 7-class fixture, 74.44s
with the full projection. At ~485 packs that is ~43 CPU-minutes added per shard.

The ordinary required matrix does not pay that cost. Per-pack registry forms run
only when a manual workflow dispatch sets `registry_forms`, or when a local run
sets `MATRIX_REGISTRY_FORMS=1`. The manual run asserts that every pack row wrote
the channel and receives the 40-minute shard budget; ordinary required runs keep
their prior 20-minute budget. A manual timeout cannot block a push or merge.

Note also where the matrix actually executes packs: `should-run` restricts that
on pull requests to branches named `version-bump-*`, so an ordinary PR does not
execute packs and the aggregate check passes without them. The per-pack census
therefore produces its numbers only on an opted-in `workflow_dispatch` or
local run.

### The seam, and why it is faithful in both directions

`proveWidgetForm` needs the array `serialiseWidgetValues` produced **before**
the extension serialize hook, because identity is the whole proof — and core
hands that array to exactly one thing outside `LGraphNode`: the node's own
`onSerialize`, which `runExtensionSerializeHook` calls with the payload itself
(`const view = canonical`). So the runner chains one on.

- **It cannot manufacture a pass.** Post-hook identity is read from the
_returned_ payload, so a pack hook running after ours, or a `serialize`
override writing after `super`, still refuses.
- **It cannot manufacture a refusal.** Installing a hook does force the hook
path for a node that would otherwise have returned early, but that path
leaves a JSON-cloneable `widgets_values` alone: the writeback skips cloneable
values, and the one `Object.assign` that could overwrite the field draws from
`readExtensionFields`, which excludes every canonical field by construction.

It is a seam, not the production call site. A producer landing inside
`serializeFromStoreState` should be measured again from there.

### `coreIsLastWriter`, and the blind spot the census measures instead of hiding

`coreIsLastWriter` is a **required** option with no default, because assuming it
is how the predicate once accepted a subgraph host. The runner answers it the
way the predicate's docstring prescribes — comparing both serialization entry
points against `LGraphNode.prototype`.

That comparison has one hole: a pack that reassigns
`LGraphNode.prototype.serialize` leaves every node still identical to the
prototype, so the check reads `true` while core is no longer the last writer.
The census cannot close it from the outside, so it records
`registryForms.prototypePatched` — captured from references taken **before** any
pack entry runs. A provable rate is only as good as that flag is false.

### `SubgraphNode`

Probed through `LGraph.convertToSubgraph` with the production
`subgraph-created` → `registerNewSubgraph` listener, not the litegraph test
fixture's `TestSubgraphNode` subclass — measuring that would be measuring a
class the product never makes. Two topologies, and each records what the
verdict would have been had `coreIsLastWriter` been answered wrongly, at this
seam and at the call site the predicate's own docstring prescribed.

### PASS criteria

Applied independently by the legacy and Vue `matrix-verdict` jobs over each
Expand Down Expand Up @@ -206,21 +363,29 @@ than reading as zero omissions.
Packs execute under real timers: the matrix config assigns `setupFiles`
explicitly, keeping `vitest.setup.ts` (environment globals) and dropping
the unit suite's `vitest.timer.setup.ts` fake timers that `mergeConfig`
array-concatenation would otherwise smuggle in.
array-concatenation would otherwise smuggle in. `vitest.census.config.mts`
assigns `include`, `exclude` and `setupFiles` after the merge for the same
reason. It collects only `scripts/registry-census/*.census.test.ts`, the glob
`vite.config.mts` excludes from the `tooling` project, so a census spec runs
under exactly one of the two configs.

### Environment

| var | used by | meaning |
| ---------------------- | ------------------ | -------------------------------------------------- |
| `CENSUS_ROOT` | all | state root, default `.census/` |
| `MATRIX_OUT` | runner, summarizer | directory of per-pack row JSON |
| `MATRIX_PREV` | summarizer | baseline metrics to compare against |
| `MATRIX_METRICS_OUT` | summarizer | where to write metrics on PASS |
| `MATRIX_RUN_ID` | summarizer | skips the delta when the baseline is this same run |
| `MATRIX_EXPECT_SHARDS` | summarizer | required manifest count; unset disables the check |
| `MATRIX_STALE_MARKER` | summarizer | path to `registry-stale.json`, if present |
| `MATRIX_RENDERER` | summarizer | expected `legacy` or `vue` renderer arm |
| `MATRIX_VUE` | runner | enables Nodes 2.0 when set to `1` |
| var | used by | meaning |
| ------------------------------ | ------------------ | -------------------------------------------------------------- |
| `CENSUS_ROOT` | all | state root, default `.census/` |
| `MATRIX_OUT` | runner, summarizer | directory of per-pack row JSON |
| `MATRIX_PREV` | summarizer | baseline metrics to compare against |
| `MATRIX_METRICS_OUT` | summarizer | where to write metrics on PASS |
| `MATRIX_RUN_ID` | summarizer | skips the delta when the baseline is this same run |
| `MATRIX_EXPECT_SHARDS` | summarizer | required manifest count; unset disables the check |
| `MATRIX_STALE_MARKER` | summarizer | path to `registry-stale.json`, if present |
| `MATRIX_RENDERER` | summarizer | expected `legacy` or `vue` renderer arm |
| `MATRIX_VUE` | runner | enables Nodes 2.0 when set to `1` |
| `MATRIX_REGISTRY_FORMS` | runner | enables the expensive per-pack registry-form sweep |
| `MATRIX_EXPECT_REGISTRY_FORMS` | summarizer | requires the opt-in sweep on every completed pack row |
| `MATRIX_OBJECT_INFO` | runner | node defs for the census; defaults to the committed projection |
| `CENSUS_FORMS_OUT` | baseline census | where the `_registry-baseline` row is written |

## Widget form/values agreement — a measurement, not a gate

Expand Down
Loading
Loading