Skip to content

docs(skills): add verify-sandbox-deployed-app skill - #1379

Open
gabrielseco wants to merge 28 commits into
mainfrom
docs-skills-verify-sandbox-deployed-app
Open

gabrielseco wants to merge 28 commits into
mainfrom
docs-skills-verify-sandbox-deployed-app

Conversation

@gabrielseco

@gabrielseco gabrielseco commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds a Claude Code skill that captures the manual sandbox-verification loop used to check onboarding behavior on the deployed sandbox demo.

Why

This loop (seed via --env=sandbox, log past the Vercel password gate, drive the flow, check the claim) was being re-derived by hand each time it was needed. Extracted out of #1371 so it can land and be reviewed independently of the seed-onboarding env flag work.

What changed

Toggle details
  • Adds .claude/skills/verify-sandbox-deployed-app/SKILL.md, a repeatable skill for driving the deployed sandbox demo (https://remote-flows-eight.vercel.app) to verify onboarding behavior without the user doing it by hand.
  • Documents seed:onboarding --env=review in CLAUDE.md: .env.review holds the deployed app's credentials, so employments seeded with it belong to the same company the deployed app uses (.env.sandbox points at the same gateway but a different company, which makes the deployed app return 404 Company not found).
  • Rewrites the scripts/seed-onboarding.ts usage comment to explain that --env=<name> only picks the .env.<name> credentials file, while VITE_REMOTE_GATEWAY inside it picks the gateway.

No library or runtime code changes.

Screenshots

N/A

Related Resources

Testing

  • N/A — docs-only skill definition
  • Feature flag: N/A

Note

Low Risk
Documentation and agent skill only; no changes to library code, APIs, or CI behavior.

Overview
Adds a Claude Code skill (verify-sandbox-deployed-app) that standardizes checking onboarding on the deployed Vercel sandbox demo: confirm .env.review (deployed-app credentials, not .env.sandbox), seed with npm run seed:onboarding -- --env=review, pass the Vercel password gate, drive the flow with a one-off Playwright script, and report pass/fail.

Docs updates spell out why .env.review vs .env.sandbox matters (same sandbox gateway, different company → 404 on the deployed app) and document --env=review in CLAUDE.md. scripts/seed-onboarding.ts only gets clearer usage comments for generic --env=<name> (file picks credentials; VITE_REMOTE_GATEWAY inside picks the gateway).

No application or package runtime changes.

Reviewed by Cursor Bugbot for commit fdd426c. Bugbot is set up for automated code reviews on this repo. Configure here.

gabrielseco and others added 5 commits September 23, 2026 09:38
Lets seed-onboarding.mjs bypass the local example dev server via --env=<name>,
reusing example/api/{utils,get_token,proxy}.js for auth against .env.<env> at
the repo root. Prints a ready-to-click app link (?employmentId= prefilled,
via VITE_APP_URL) when set. Onboarding.tsx now reads employmentId from the
URL query string so that link actually prefills the intro form.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Captures the manual sandbox-verification loop (seed via --env=sandbox, log
past the Vercel password gate with VITE_APP_PASSWORD, drive the flow, check
the claim) as a repeatable skill instead of re-deriving it each time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Split into its own PR (#1379) so it can be reviewed independently of
the seed-onboarding env flag.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Deploy preview for adp-cost-calculator ready!

Project:adp-cost-calculator
Status: ✅  Deploy successful!
Preview URL:https://adp-cost-calculator-amq86xz9e-remotecom.vercel.app
Latest Commit:fdd426c

Deployed with vercel-action

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Deploy preview for remote-flows ready!

Project:remote-flows
Status: ✅  Deploy successful!
Preview URL:https://remote-flows-nfnb9qroj-remotecom.vercel.app
Latest Commit:fdd426c

Deployed with vercel-action

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 9920f32. Configure here.

Comment thread .claude/skills/verify-sandbox-deployed-app/SKILL.md
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

📊 Coverage Report

⚪ Coverage unchanged

Metric Current Previous Change Status
Lines 86.82% 86.82% 0% ⚪
Statements 86.39% 86.39% 0% ⚪
Functions 85.42% 85.42% 0% ⚪
Branches 78.19% 78.19% 0% ⚪

Detailed Breakdown

Lines Coverage
  • Covered: 5006 / 5766
  • Coverage: 86.82%
  • Change: 0% (0 lines)
Statements Coverage
  • Covered: 5095 / 5898
  • Coverage: 86.39%
  • Change: 0% (0 statements)
Functions Coverage
  • Covered: 1330 / 1557
  • Coverage: 85.42%
  • Change: 0% (0 functions)
Branches Coverage
  • Covered: 3093 / 3956
  • Coverage: 78.19%
  • Change: 0% (0 branches)

✅ Coverage check passed

gabrielseco and others added 8 commits September 23, 2026 12:58
The basic and Germany onboarding specs create real employments against
the sandbox and ran against Playwright's 30s test / 5s expect defaults.
Traces from PR #1371 and main show nothing hanging: Germany reached the
final ILA sign click at 27.2s, leaving <3s for the sign POST, and Spain
failed step-title assertions while ~3s employment writes were in flight.

- test.slow() on both onboarding specs (90s budget)
- expectOnboardingStep helper with a 15s timeout for step transitions,
  which are gated on sandbox writes rather than rendering
- wait on the ILA sign POST and assert it succeeded before checking the
  dialog closed, so a backend rejection reports as such

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ally

Replaces the per-spec test.slow() and the 15s step-title timeout with:

- submitAndWaitForSave / clickAndWaitForSave helpers that wait for the
  request each step's submit fires (POST /v1/employments, POST
  engagement-agreement-details, PATCH /v1/employments/{id}, PUT
  benefit-offers, POST .../sign) and assert it succeeded, instead of
  waiting for the "Loading..." text to disappear. A backend failure now
  reports method, path and status.
- Global timeout (60s) and expect timeout (10s) in playwright.config.ts,
  matching Dragon's defaults. Response waits make failures precise but do
  not make the sandbox faster; the Germany flow alone takes ~30s on a
  good day, so the 30s default budget has to go up regardless.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
fakeValueFor() hardcoded [true] for checkbox fields, mimicking a
jsonType-array shape that doesn't match how the real form submits
const-based checkboxes (e.g. FRA's ack_non_eligible_job_titles, a
jsonType string with const: "acknowledged"). parseFormValuesToAPI
swaps the RHF boolean for field.const at submit time; mirror that
here instead of sending the raw fill value, or the API 422s with a
type mismatch.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Runs via tsx like the other scripts in scripts/, and is now covered by the
project type-check. API failures throw a typed ApiError; the CommonJS auth
helpers in example/api are required with typed casts so the example app is
untouched.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
gabrielseco added a commit that referenced this pull request Sep 25, 2026
* feat(scripts): seed onboarding directly against a gateway env

Lets seed-onboarding.mjs bypass the local example dev server via --env=<name>,
reusing example/api/{utils,get_token,proxy}.js for auth against .env.<env> at
the repo root. Prints a ready-to-click app link (?employmentId= prefilled,
via VITE_APP_URL) when set. Onboarding.tsx now reads employmentId from the
URL query string so that link actually prefills the intro form.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(skills): add verify-sandbox-deployed-app skill

Captures the manual sandbox-verification loop (seed via --env=sandbox, log
past the Vercel password gate with VITE_APP_PASSWORD, drive the flow, check
the claim) as a repeatable skill instead of re-deriving it each time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* style: fix oxfmt formatting in verify-sandbox-deployed-app skill

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(skills): remove verify-sandbox-deployed-app skill

Split into its own PR (#1379) so it can be reviewed independently of
the seed-onboarding env flag.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(e2e): give sandbox onboarding specs room for slow backend writes

The basic and Germany onboarding specs create real employments against
the sandbox and ran against Playwright's 30s test / 5s expect defaults.
Traces from PR #1371 and main show nothing hanging: Germany reached the
final ILA sign click at 27.2s, leaving <3s for the sign POST, and Spain
failed step-title assertions while ~3s employment writes were in flight.

- test.slow() on both onboarding specs (90s budget)
- expectOnboardingStep helper with a 15s timeout for step transitions,
  which are gated on sandbox writes rather than rendering
- wait on the ILA sign POST and assert it succeeded before checking the
  dialog closed, so a backend rejection reports as such

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(e2e): gate onboarding steps on save responses, set timeouts globally

Replaces the per-spec test.slow() and the 15s step-title timeout with:

- submitAndWaitForSave / clickAndWaitForSave helpers that wait for the
  request each step's submit fires (POST /v1/employments, POST
  engagement-agreement-details, PATCH /v1/employments/{id}, PUT
  benefit-offers, POST .../sign) and assert it succeeded, instead of
  waiting for the "Loading..." text to disappear. A backend failure now
  reports method, path and status.
- Global timeout (60s) and expect timeout (10s) in playwright.config.ts,
  matching Dragon's defaults. Response waits make failures precise but do
  not make the sandbox faster; the Germany flow alone takes ~30s on a
  good day, so the 30s default budget has to go up regardless.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
gabrielseco and others added 4 commits September 25, 2026 10:28
# Conflicts:
#	CLAUDE.md
#	scripts/seed-onboarding.ts
stringArg('env') returned undefined for a bare --env (no =value), which
made ENV falsy and silently fell through to the local-dev-server branch
instead of erroring - the old .mjs kept the bare true value truthy and
hit the existing "Unknown --env=..." guard. A fat-fingered --env (e.g.
--env sandbox with a space instead of =) now seeds through whichever
environment example/.env happens to point at, with no indication the
flag was ignored - the exact failure mode #1371 was written to prevent.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Captures the manual sandbox-verification loop (seed via --env=sandbox, log
past the Vercel password gate with VITE_APP_PASSWORD, drive the flow, check
the claim) as a repeatable skill instead of re-deriving it each time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@gabrielseco
gabrielseco force-pushed the docs-skills-verify-sandbox-deployed-app branch from 9920f32 to 421145a Compare September 28, 2026 08:11
gabrielseco and others added 2 commits September 28, 2026 10:20
The xargs-into-printf one-liner used the password as printf's format
string, so a password containing % or \ failed or got mangled (and
wrote a partial line). It also appended without guarding against a
missing trailing newline, corrupting the file's last variable. Pass the
password via %s and prefix a newline instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without the guard, a failed op read (e.g. 1Password desktop app not
running) still appended an empty VITE_APP_PASSWORD= line, which Step 1
would then treat as present.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gabrielseco added a commit that referenced this pull request Sep 28, 2026
* feat(scripts): seed onboarding directly against a gateway env

Lets seed-onboarding.mjs bypass the local example dev server via --env=<name>,
reusing example/api/{utils,get_token,proxy}.js for auth against .env.<env> at
the repo root. Prints a ready-to-click app link (?employmentId= prefilled,
via VITE_APP_URL) when set. Onboarding.tsx now reads employmentId from the
URL query string so that link actually prefills the intro form.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(skills): add verify-sandbox-deployed-app skill

Captures the manual sandbox-verification loop (seed via --env=sandbox, log
past the Vercel password gate with VITE_APP_PASSWORD, drive the flow, check
the claim) as a repeatable skill instead of re-deriving it each time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* style: fix oxfmt formatting in verify-sandbox-deployed-app skill

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(skills): remove verify-sandbox-deployed-app skill

Split into its own PR (#1379) so it can be reviewed independently of
the seed-onboarding env flag.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(e2e): give sandbox onboarding specs room for slow backend writes

The basic and Germany onboarding specs create real employments against
the sandbox and ran against Playwright's 30s test / 5s expect defaults.
Traces from PR #1371 and main show nothing hanging: Germany reached the
final ILA sign click at 27.2s, leaving <3s for the sign POST, and Spain
failed step-title assertions while ~3s employment writes were in flight.

- test.slow() on both onboarding specs (90s budget)
- expectOnboardingStep helper with a 15s timeout for step transitions,
  which are gated on sandbox writes rather than rendering
- wait on the ILA sign POST and assert it succeeded before checking the
  dialog closed, so a backend rejection reports as such

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(e2e): gate onboarding steps on save responses, set timeouts globally

Replaces the per-spec test.slow() and the 15s step-title timeout with:

- submitAndWaitForSave / clickAndWaitForSave helpers that wait for the
  request each step's submit fires (POST /v1/employments, POST
  engagement-agreement-details, PATCH /v1/employments/{id}, PUT
  benefit-offers, POST .../sign) and assert it succeeded, instead of
  waiting for the "Loading..." text to disappear. A backend failure now
  reports method, path and status.
- Global timeout (60s) and expect timeout (10s) in playwright.config.ts,
  matching Dragon's defaults. Response waits make failures precise but do
  not make the sandbox faster; the Germany flow alone takes ~30s on a
  good day, so the 30s default budget has to go up regardless.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📦 Bundle Size Report

Metric Current Previous Change Status
Total (gzip) 167.25 kB 167.25 kB 0 B (0%) 🟢
Total (raw) 559.22 kB 559.22 kB 0 B (0%) 🟢
CSS (gzip) 21.94 kB 21.94 kB 0 B (0%) 🟢
CSS (raw) 114.43 kB 114.43 kB 0 B (0%) 🟢

Size Limits

  • ✅ Total gzipped: 167.25 kB / 350 kB (47.8%)
  • ✅ Total raw: 559.22 kB / 850 kB (65.8%)
  • ✅ CSS gzipped: 21.94 kB / 25 kB (87.8%)

Largest Files (Top 5)

  1. CheckBoxField-CBXcXbix.js - 24.61 kB (0 B (0%))
  2. ContractorOnboarding-CZBU8CGm.js - 13.54 kB (0 B (0%))
  3. Onboarding-dS9AgOSV.js - 11.27 kB (0 B (0%))
  4. Termination-zUqQ758Y.js - 11.08 kB (0 B (0%))
  5. styles.css - 10.97 kB (0 B (0%))
View All Files (75 total)
File Size (gzip) Change
CheckBoxField-CBXcXbix.js 24.61 kB 0 B (0%)
ContractorOnboarding-CZBU8CGm.js 13.54 kB 0 B (0%)
Onboarding-dS9AgOSV.js 11.27 kB 0 B (0%)
Termination-zUqQ758Y.js 11.08 kB 0 B (0%)
styles.css 10.97 kB 0 B (0%)
index.css 10.97 kB 0 B (0%)
CostCalculator-DKTQMrVy.js 10.68 kB 0 B (0%)
internals-CjKQthVT.js 8.23 kB 0 B (0%)
index.js 5.05 kB 0 B (0%)
JSONSchemaForm-I1mWdIp7.js 4.92 kB 0 B (0%)

✅ Bundle size check passed

gabrielseco and others added 2 commits September 28, 2026 15:06
.env.sandbox holds the local-dev sandbox client, which belongs to a
different company than the deployed demo app's client. Employments seeded
with it make every employment call on the deployed app return 404
"Company not found". Add .env.review (the deployed app's own Vercel
credentials, still VITE_REMOTE_GATEWAY=sandbox) and point the
verify-sandbox-deployed-app skill, the seed script docs and CLAUDE.md at
it. No code change needed: --env=<name> already loads .env.<name> and the
gateway comes from VITE_REMOTE_GATEWAY inside the file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gabrielseco added a commit that referenced this pull request Sep 29, 2026
…#1393)

* feat(scripts): seed onboarding directly against a gateway env

Lets seed-onboarding.mjs bypass the local example dev server via --env=<name>,
reusing example/api/{utils,get_token,proxy}.js for auth against .env.<env> at
the repo root. Prints a ready-to-click app link (?employmentId= prefilled,
via VITE_APP_URL) when set. Onboarding.tsx now reads employmentId from the
URL query string so that link actually prefills the intro form.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(skills): add verify-sandbox-deployed-app skill

Captures the manual sandbox-verification loop (seed via --env=sandbox, log
past the Vercel password gate with VITE_APP_PASSWORD, drive the flow, check
the claim) as a repeatable skill instead of re-deriving it each time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* style: fix oxfmt formatting in verify-sandbox-deployed-app skill

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(skills): remove verify-sandbox-deployed-app skill

Split into its own PR (#1379) so it can be reviewed independently of
the seed-onboarding env flag.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(e2e): give sandbox onboarding specs room for slow backend writes

The basic and Germany onboarding specs create real employments against
the sandbox and ran against Playwright's 30s test / 5s expect defaults.
Traces from PR #1371 and main show nothing hanging: Germany reached the
final ILA sign click at 27.2s, leaving <3s for the sign POST, and Spain
failed step-title assertions while ~3s employment writes were in flight.

- test.slow() on both onboarding specs (90s budget)
- expectOnboardingStep helper with a 15s timeout for step transitions,
  which are gated on sandbox writes rather than rendering
- wait on the ILA sign POST and assert it succeeded before checking the
  dialog closed, so a backend rejection reports as such

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(e2e): gate onboarding steps on save responses, set timeouts globally

Replaces the per-spec test.slow() and the 15s step-title timeout with:

- submitAndWaitForSave / clickAndWaitForSave helpers that wait for the
  request each step's submit fires (POST /v1/employments, POST
  engagement-agreement-details, PATCH /v1/employments/{id}, PUT
  benefit-offers, POST .../sign) and assert it succeeded, instead of
  waiting for the "Loading..." text to disappear. A backend failure now
  reports method, path and status.
- Global timeout (60s) and expect timeout (10s) in playwright.config.ts,
  matching Dragon's defaults. Response waits make failures precise but do
  not make the sandbox faster; the Germany flow alone takes ~30s on a
  good day, so the 30s default budget has to go up regardless.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(scripts): send checkbox const value, not [true], in seed-onboarding

fakeValueFor() hardcoded [true] for checkbox fields, mimicking a
jsonType-array shape that doesn't match how the real form submits
const-based checkboxes (e.g. FRA's ack_non_eligible_job_titles, a
jsonType string with const: "acknowledged"). parseFormValuesToAPI
swaps the RHF boolean for field.const at submit time; mirror that
here instead of sending the raw fill value, or the API 422s with a
type mismatch.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* refactor(scripts): port seed-onboarding to TypeScript

Runs via tsx like the other scripts in scripts/, and is now covered by the
project type-check. API failures throw a typed ApiError; the CommonJS auth
helpers in example/api are required with typed casts so the example app is
untouched.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(scripts): error loudly on a bare --env flag instead of ignoring it

stringArg('env') returned undefined for a bare --env (no =value), which
made ENV falsy and silently fell through to the local-dev-server branch
instead of erroring - the old .mjs kept the bare true value truthy and
hit the existing "Unknown --env=..." guard. A fat-fingered --env (e.g.
--env sandbox with a space instead of =) now seeds through whichever
environment example/.env happens to point at, with no indication the
flag was ignored - the exact failure mode #1371 was written to prevent.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
gabrielseco and others added 2 commits September 29, 2026 12:15
…ools

The skill tells the agent never to read the 1Password secret itself, so
pre-approving op read contradicted that and let it read any secret silently.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant