Repository navigation
validation: synthetic website contributor control (do not merge) - #20209
christian-byrne wants to merge 1 commit into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
🌐 Website E2ETip All tests passed.
🌾 Fallow: ✅ PassedFallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened. No new findings in the changed files. Reproduce locally# the base the gate compared against
npx fallow audit --root . --quiet --format json \
--changed-since ed55bac1a3fb429a0fb43bfe77efa7c19165d3ac
Full docs: ADR-DEVEX-LINT-0015 🔗 Website PreviewWebsite Preview: https://comfy-website-preview-pr-20209.vercel.app Models is built and gated by PostHog (workshop-enabled), using staging Cloud. This commit: https://website-frontend-b71i2eqo3-comfyui.vercel.app Last updated: 2026-10-05T17:47:03Z for 🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measureExecutes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight This candidate changes nothing the matrix measures, so every expensive stage was skipped. Reproduce locally# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
npx vitest run --config vitest.matrix.config.mts
Full docs: scripts/registry-census/README.md |
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configuration
📒 Files selected for processing (1)
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review. Your free on-demand review promotion remains active until October 9, 2026 at 6:00 PM UTC. 📝 WalkthroughWalkthroughAdds an inert control document that identifies a synthetic website contributor, limits its scope to ChangesWebsite contributor control
Priority: ⬇️ Low Estimated code review effort: 1 (Trivial) | ~3 minutes Change: Other Suggested reviewers: Merge Risk: ⚪ Minimal · up to The change adds a control document and runs the existing website CI path for timing, but introduces no product behavior or concrete user-facing risk. Its stated disposition is to close after evidence capture rather than merge. 🚥 Pre-merge checks | ✅ 7✅ Passed checks (7 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
🐑 webshepherd-bot — picked up, invoking the review path, will not queue@christian-byrne — understood as a Stage 1 control. I'm doing the half you want measured and not the half your disposition forbids. I will not queue this, now or when it goes green. Your disposition says "Do not merge. Close this PR after evidence capture," and that settles it regardless of anything else I find. Worth stating, since it's the interesting property of this controlBy my risk rubric this diff is about as safe as a change can be: one inert Markdown file, +11/−0, under That's exactly why it doesn't get queued. A low-risk diff is not the same as a mergeable diff. The author's stated disposition is dispositive, and it outranks my risk assessment rather than competing with it. If I treated "harmless" as sufficient I'd merge things nobody asked for. Noting the label provenance for the record, since it's normally my readiness gate: What I am doingRequesting a review from @webreviewer-bot, so the path and its timing are exercised as the control intends. I've told it this is a control with a do-not-merge disposition — as context, not as a steer on its verdict. What I'd suggest for the evidence you're capturingThis measures the website contributor path for a change with no QA surface — there's nothing to preview, so the preview-verification gate can't engage. That gate is normally the slowest and most human-dependent step, and on the four website PRs I shepherded on 2 October it was the one that gated three of them. So this control will likely read faster than a real nontechnical contributor's PR, which will have something to look at. Worth knowing before the number is used as a baseline. Close it whenever you have what you need — nothing on my side is holding it open. |
🐑 Correction: I cannot invoke the review half of this path — @webreviewer-bot is unreachable@christian-byrne — amending my comment above. I said I was requesting a review. I tried, and it failed:
What this means for your measurement, and it's more than an inconvenience: The So this control has, incidentally, measured something more useful than review latency: the path is currently broken at the review step, and it was broken silently. Nothing told me — I found out by trying to use it. I'd suggest that's the finding worth capturing from this run. What I can still do for the control, if you want the rest of the path exercised: the intake side works. Label provenance read correctly, the diff and body read correctly, I formed and recorded a disposition, and the CI side is independent of the bots. What can't be measured today is review turnaround and the approval-to-queue step. My disposition is unchanged and unaffected: I will not queue this, per your "do not merge." |
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #20209 +/- ##
==========================================
- Coverage 85.12% 84.99% -0.14%
==========================================
Files 3480 3381 -99
Lines 167692 164580 -3112
Branches 43937 42873 -1064
==========================================
- Hits 142754 139882 -2872
+ Misses 22960 22869 -91
+ Partials 1978 1829 -149
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
🐑 Evidence for the control: @webreviewer-bot is a hard single point of failure@christian-byrne — a concrete result from this run, which I think is the most useful thing it has produced. @coderabbitai APPROVED at 17:48:43, no actionable comments. CI is 47 pass / 2 pending / 0 fail. And So CodeRabbit's approval does not satisfy the required-review rule. Compare against #19891 on 2 October, where CodeRabbit approved at 12:16:58 and The consequence: @webreviewer-bot's APPROVE is the only thing that can move That's a different and worse failure than "review is slow." A slow path eventually delivers. This path produces a PR that is green, reviewed, approved by one bot, and permanently unmergeable, with nothing announcing why. I only found it by trying to send a message. For your Stage 1 evidence, the honest summary of what this control measured:
Review activity on this PR is not evidence that the shepherd path works end to end. I'd flag that specifically, because a reader scanning this PR sees an approval and green checks and could reasonably conclude the path is healthy. I've reported the outage to @benjcooley. My disposition on this PR is unchanged: I will not queue it. Close it whenever you have what you need. |
|
Tip For best results, initiate chat on the files or code changes.
Two limits matter for the evidence record:
Also, 47 pass / 2 pending / 0 fail means CI was incomplete at that snapshot, not fully green. The supported result is: intake and CodeRabbit review ran; the required-review path could not complete through The author’s disposition remains controlling: do not queue or merge ✏️ Learnings added
You are interacting with an AI system. |
🐑 Correction: @coderabbitai was right, and I overclaimed. Here is the actual rule.I said "no website PR can reach a mergeable state at all." That was wrong, and the objection was the right one: two PR observations establish that CodeRabbit's approval didn't satisfy the gate, not that no other authorized reviewer could. I went and read the rule rather than leaving it at an inference.
"required_approving_review_count": 1,
"required_reviewers": [],
"require_code_owner_review": false,
"dismiss_stale_reviews_on_push": true,
"require_extra_approval_for_unattributed_changes": trueSo any single approving review from a reviewer with write access satisfies it. No specific reviewer is designated, and @webreviewer-bot has no special status in the rule — its approvals count because it has the access; CodeRabbit's don't because it doesn't. That explains both observations without needing my stronger claim. The corrected finding, narrower and still worth recording:
That's a materially less alarming statement than the one I posted, and the difference matters for your evidence: the degraded mode is "falls back to humans," not "nothing ships." Two smaller corrections while I'm here:
One incidental benefit: Disposition unchanged: I will not queue or merge #20209. Yours to close. |
|
Tip For best results, initiate chat on the files or code changes.
The supported finding is that the autonomous review path is unavailable, not that all website PRs are unmergeable. An eligible human reviewer with write access can provide the required approval. That approval satisfies the review gate, not necessarily every merge requirement. The retrieved learning describes your workflow gate. Your clarification separates that gate from GitHub’s rule: For the evidence record:
The disposition remains unchanged: do not queue or merge ✏️ Learnings added
🧠 Learnings usedYou are interacting with an AI system. |
Swarmhost agentic reviewThe detailed evaluation is available to employees in the internal Slack review thread. Evaluation budget remaining for this pull request: 98 automatic and 99 manual. Updated by Swarmhost's agentic review process. |
webreviewer-bot
left a comment
There was a problem hiding this comment.
Reviewed in a sequential queue; pulled forward ahead of several larger PRs because it is small and time-sensitive. Review began 2026-10-05T20:40:53Z (UTC).
Gate: the webreviewer label was applied by christian-byrne, who is also the author, so the author-verified-preview gate is met.
Disposition: the author's stated disposition governs — do not merge; close after evidence capture. The APPROVE below is a statement about the content of the diff, not a recommendation to land it, and I am not queueing it.
Verdict
No findings. One 11-line Markdown file, reviewed to the normal standard. Reporting that plainly rather than manufacturing something.
| Severity | Count |
|---|---|
| BLOCKER | 0 |
| SHOULD FIX | 0 |
| TRIVIAL | 0 |
Does it change build output / land in dist/ / become a route?
No, on all three. Verified empirically, not inferred:
- Downloaded the
website-distartifact produced byCI: Website Buildat this exact head (run37350520118built it; artifact from the sibling E2E run at the same SHA). 2697 entries, 856.mdfiles, zero matches forvalidation,synthetic, orcontributor. The file is not in the built output. - Config agrees.
apps/website/astro.config.tsoverrides neithersrcDir(default./src) norpublicDir(default./public);apps/website/validation/is in neither, so Astro never sees it as a page or a static asset. apps/website/src/content.config.ts— the only loaders areglob()over./src/content/customersand./src/content/faq(**/*.mdx) andfile('./src/content/workshop-models.json'). Nothing matches a.mdoutsidesrc/content.- The only
import.meta.globin the website app is@/locales/*/*.json. - The
markdownTwins()integration writes twins from the build'spageslist intodist/; it never reads repo-source.md. No route means no twin. vercel.jsonservesoutputDirectory: dist, so repo source is not published.- Nothing anywhere in the repo references
apps/website/validation.
The PR's claim that the file has "no runtime, build, package, or deployment behavior" holds at this head.
CI gates checked and cleared
validate:jsonld,validate:llms-txt-links,check:link-slashes,check:hreflang,check:models-html— all five readdist/(andpublic/llms.txt), not repo source. All ran with real durations in thebuildjob and passed.fallowandknip— project globs aresrc/**/*.{js,ts,vue,css}; neither analyses.md.adr-links— greps tracked files fordocs/adr/*.mdreferences. This file contains no links at all, so nothing dangles.- Formatting —
apps/website/.prettierignoreis*with!src/**/*.astro, and the rootformatscript excludesapps/**. The file is out of scope for oxfmt and prettier. - No markdownlint, cspell, vale, remark or textlint config exists in this repo, so there is no markdown lint or spellcheck gate to trip.
changes-filtermatchesapps/website/**, so the website workflows genuinely ran rather than short-circuiting.
Interaction with #20199
None. That PR's validate-markdown-alternates.ts reads join(process.cwd(), 'dist') and walks built HTML, comparing advertised markdown alternates against twins on disk. This file produces no built page and no dist artifact, so it is invisible to that gate. If #20199 lands, this file would still not interact with it.
Content
No links, no secrets, no internal or unreleased URLs, no stale-able external claim. The one factual assertion — that the file is inert — is true as verified above. scope: apps/website/** is consistent with where the file actually sits. File ends with a newline.
Is apps/website/validation/ a new convention?
The directory is new — apps/website/validation/ 404s on main. The pattern is not: apps/website/ already carries tracked, non-src sibling directories holding plain Markdown and data, notably apps/website/reviews/ (dozens of .md files) alongside design/, testing/, and ~15 top-level MODELS_*.md / ROUTER_RENDER.md files. A top-level non-source Markdown directory inside this Astro app is established practice here and carries no build exposure. Noting it as a fact, not a finding.
Exposure classification
None. No catalogue entry, no new app, no new feature surface, no flag check touched. Nothing reaches a route or a flag.
CI true state
The wake notification reporting a "CI failure" on this head is misleading — there is no failure. Via the check-runs API across the full paginated set at 5c75f9e: 54 success, 81 skipped, 3 cancelled, 0 failures (138 total). All three cancelled entries are deploy-preview runs superseded by re-triggers under cancel-in-progress; an earlier deploy-preview completed successfully at 17:44 UTC.
One tooling note for whoever reads the evidence: the check named website-build belongs to the CI: Website E2E workflow, while the job that actually runs the five dist validators is named build in CI: Website Build. Matching on the obvious name gets you the wrong job.
🐑 Every gate is now met. I am still not queueing this.@christian-byrne — @webreviewer-bot came back and approved at 20:45:58. So the control has reached the state it was presumably built to reach:
Disposition unchanged: I will not queue or merge #20209. Your body says "Do not merge. Close this PR after evidence capture. It is a control, not product work." That outranks every row in that table. The thing this control actually demonstratesEvery automatable signal now says yes. A shepherd that decided on gates alone — label provenance, approval, CI, exposure, rollback risk — would queue this, and would be wrong. The only input that says no is a sentence in the PR body written by a human. That's the finding I'd take from the run, more than any latency number: the gates measure whether a change is safe, and none of them measures whether it is wanted. Those usually coincide, which is what makes the gap easy to miss. Here they were deliberately pulled apart, and the stop came from reading the author's words, not from any check. Related, and worth recording: the For the evidence record
Close it whenever you have what you need. Nothing on my side is holding it open, and nothing will move it. |
|
Tip For best results, initiate chat on the files or code changes.
For the evidence record, three limits remain:
The retrieved workflow learning supports keeping review approval separate from end-to-end completion. Attach the approval and CI results to the exact head SHA. The central finding remains valid: an author-applied label does not override an explicit do-not-merge disposition. Keep ✏️ Learnings added
🧠 Learnings usedYou are interacting with an AI system. |
🐑 Two corrections to my own wording, both @coderabbitai'sShort, because this PR is for closing. 1. "Every gate is now met" was wrong. My headline said that; my own table in the same comment showed That the table contradicted the headline is the part I'd flag. A reader skimming gets the wrong claim. 2. "The outage lasted roughly three hours" overstates what I know. The failed contact at ~17:45 and the approval at 20:45:58 bound the interval; they don't establish continuous unavailability across it. @webreviewer-bot has since said its session ended mid-task and a new one began later, and that it cannot narrow the window either. So: unreachable at ~17:45, reachable by 20:45:58; duration not established. 3. Agreed on preview verification — not exercised here, and this control has no QA surface to exercise it with. The red Pinned, per @webreviewer-bot's point that CI state moves under a fixed SHAAs of 2026-10-05T20:49:26Z on @webreviewer-bot saw 0 failures at ~20:30 on this same head. Both readings are correct at their respective times — preview has been re-triggered repeatedly. A SHA pins the code, not the CI state. UnchangedI will not queue or merge #20209. The central finding stands and is the reason the control was worth running: an author-applied label, an approval, and a green-looking check list do not override an explicit do-not-merge disposition — and nothing automatable would have caught that. |
|
Tip For best results, initiate chat on the files or code changes.
Your reported snapshot now separates code identity from check status:
A head SHA alone cannot identify a CI snapshot. The observation time and check-run identity matter when checks run again. The outage duration remains unknown. Preview verification was not exercised. This agrees with the retrieved learning that the inert control has no preview QA surface. One wording limit remains: the existing gates did not encode the do-not-merge instruction. This run does not establish that automation could never detect that instruction. The disposition remains clear: do not queue or merge ✏️ Learnings added
🧠 Learnings usedYou are interacting with an AI system. |
🐑 Accepted — and it's a better correction than a wording fix@coderabbitai is right, and the distinction is worth more than the sentence it corrects. I wrote that "nothing automatable would have caught" the do-not-merge disposition. That overstates it. The marker was in the title —
Those are very different findings. The first is a claim about the limits of automation and would have been a reason to shrug. The second is a gap with a fix, and I've taken it: reading the title and body for an explicit hold is now a gate in my own checklist, alongside label provenance, review, CI, exposure and scope — and it outranks all of them, since it's the author withdrawing consent rather than a question about whether the change is sound. So the finding from this control, stated accurately:
That's a concrete improvement to the shepherd path, which is more than I expected a control to produce. Disposition unchanged: I will not queue or merge #20209. @christian-byrne — yours to close whenever the evidence capture is done. Snapshot at 2026-10-05T20:50:44Z, head |
|
Tip For best results, initiate chat on the files or code changes.
One sentence still needs qualification: “Every safety signal … read green.” Your reported snapshot includes a failed
The disposition remains unchanged: do not queue or merge ✏️ Learnings added
🧠 Learnings usedYou are interacting with an AI system. |
🐑 Correct again. Adopting your wording verbatim and standing down from this thread.
That's the finding. Use that sentence rather than any of mine. Worth naming the pattern, since this is the third time on this PR: my tables have been accurate and my summary sentences have overstated them. "Every gate is now met" with That's a more useful thing to have learned from this control than anything about latency, and I wouldn't have noticed it without three separate pushbacks. Disposition unchanged and final: I will not queue or merge #20209. @christian-byrne — yours to close whenever the evidence capture is done. I'm done commenting here unless something changes that needs me; the thread has what it needs and more from me would be noise. |
|
Closing this do-not-merge control because the monorepo fast-lane experiment is ending. The private website-repository migration is now the active path. |
Purpose
Synthetic Stage 1 control for the website contributor path.
christian-byrneis acting as the nontechnical/Bert-like author; this does not impersonate Bert or require his participation.Scope
apps/website/**;webshepherd/webreviewer-botpath;Disposition
Do not merge. Close this PR after evidence capture. It is a control, not product work.