Repository navigation
fix(onboarding): arm the first-run nudge on the tour's ending, not on a start - #14672
christian-byrne wants to merge 3 commits into
Conversation
… a start `tourWasShown` was set from whether `startTour` returned true, so every ending that followed a started tour got the `ran` copy — including a tour the user skipped on step 1 and a tour torn down by a target that never mounted, which raises a `loadError` toast at the same moment. `finish()` already computes `outcome` and `skipReason`; the store now records them as `lastEnding` and the controller reads the ending rather than the start: - completed -> nudge, `ran` copy - skipped by user -> nudge, `noTour` copy (no congratulation) - target_timeout -> no nudge (the error toast owns that moment) - postponed -> no nudge - trigger_lost -> no nudge - never resolved -> nudge, `noTour` copy (unchanged) `nudge_shown` and `explore_templates_clicked` now carry `tour_outcome`, so the population the tour never reached is separable from the one that saw a tour. Closes #14622 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
📄 Knowledge reviewDosu skipped reviewing this PR because your organization has used its |
🎨 Storybook: ✅ Built — View Storybook🎭 Playwright: ✅ 1784 passed, 0 failed · 3 flaky📊 Browser Reports
📦 Bundle: 8.24 MB gzip 🔴 +483 BDetailsSummary
Category Glance App Entry Points — 3.64 kB (baseline 3.64 kB) • ⚪ 0 BMain entry bundles and manifests
Status: 1 added / 1 removed Graph Workspace — 1.35 MB (baseline 1.35 MB) • 🔴 +1.41 kBGraph editor runtime, canvas, workflow orchestration
Status: 1 added / 1 removed / 1 unchanged Views & Navigation — 111 kB (baseline 111 kB) • ⚪ 0 BTop-level views, pages, and routed surfaces Status: 16 unchanged Panels & Settings — 551 kB (baseline 551 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens Status: 26 unchanged User & Accounts — 28.7 kB (baseline 28.7 kB) • ⚪ 0 BAuthentication, profile, and account management bundles Status: 10 unchanged Editors & Dialogs — 124 kB (baseline 124 kB) • ⚪ 0 BModals, dialogs, drawers, and in-app editors Status: 7 unchanged UI Components — 70 kB (baseline 70 kB) • ⚪ 0 BReusable component library chunks Status: 15 unchanged Data & Services — 3.47 MB (baseline 3.47 MB) • ⚪ 0 BStores, services, APIs, and repositories Status: 17 unchanged Utilities & Hooks — 386 kB (baseline 386 kB) • ⚪ 0 BHelpers, composables, and utility bundles Status: 36 unchanged Vendor & Third-Party — 15.7 MB (baseline 15.7 MB) • ⚪ 0 BExternal libraries and shared vendor chunks Status: 16 unchanged Other — 12.8 MB (baseline 12.8 MB) • ⚪ 0 BBundles that do not match a named category
Status: 1 added / 1 removed / 280 unchanged ⚡ Performance
|
📝 WalkthroughWalkthroughThe onboarding store now records the last tour ending. The first-run controller exposes ChangesOnboarding tour outcome tracking
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant onboardingTourStore
participant useFirstRunTourController
participant FirstRunTourNudge
participant Telemetry
onboardingTourStore->>useFirstRunTourController: provide lastEnding
useFirstRunTourController->>useFirstRunTourController: derive tourOutcome
useFirstRunTourController->>FirstRunTourNudge: provide tourOutcome
FirstRunTourNudge->>Telemetry: record tour_outcome
Possibly related PRs
Suggested labels: Suggested reviewers: Important Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 inconclusive)
✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/platform/onboarding/onboardingTourStore.ts`:
- Around line 36-40: Update TourEnding from a single interface into a
discriminated union keyed by outcome: require skipReason for skipped endings and
disallow it for completed endings. Adjust finish() to construct the appropriate
explicit union branch so completed results omit skipReason while skipped results
include it.
In `@src/renderer/extensions/firstRunTour/nudge/FirstRunTourNudge.test.ts`:
- Line 18: Update the mock tourOutcome declaration in FirstRunTourNudge.test.ts
to import OnboardingTourNudgeOutcome from `@/platform/telemetry/types` and use it
as the ref generic, replacing the duplicated string union while preserving the
existing initial value.
- Around line 184-211: Expand the FirstRunTourNudge telemetry tests around the
existing “says which population it was shown to” coverage by adding
parameterized cases for completed, skipped, and not_started outcomes. Verify
each outcome is reported correctly for both nudge_shown and
explore_templates_clicked, reusing the existing test setup and tracking symbols
such as mocks.tourOutcome, mocks.trackOnboardingTour, and renderNudge; preserve
the current assertions for the nudge copy and include the required
interruption/timeout coverage if those outcomes are supported by the test
fixtures.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 0ecfbea3-93e2-4732-b179-d0bcd8790218
📒 Files selected for processing (7)
src/platform/onboarding/onboardingTourStore.test.tssrc/platform/onboarding/onboardingTourStore.tssrc/platform/telemetry/types.tssrc/renderer/extensions/firstRunTour/nudge/FirstRunTourNudge.test.tssrc/renderer/extensions/firstRunTour/nudge/FirstRunTourNudge.vuesrc/renderer/extensions/firstRunTour/tour/useFirstRunTourController.test.tssrc/renderer/extensions/firstRunTour/tour/useFirstRunTourController.ts
Addresses CodeRabbit review on #14672: - `skipReason` is now required on `skipped` and absent on `completed`, so a writer cannot record a skip without saying why. `finish()` builds the two branches explicitly. - the nudge test's mock ref uses `OnboardingTourNudgeOutcome` rather than a duplicated inline union. - both `nudge_shown` and `explore_templates_clicked` are now covered for all three outcomes (`it.for`, per the repo's consistent-each-for rule) so a hard-coded or stale outcome is caught. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #14672 +/- ##
=======================================
Coverage 78.46% 78.46%
=======================================
Files 1790 1790
Lines 105382 105395 +13
Branches 33052 33059 +7
=======================================
+ Hits 82683 82696 +13
+ Misses 22264 22257 -7
- Partials 435 442 +7
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 3 files with indirect coverage changes 🚀 New features to boost your workflow:
|
The cloud e2e caught this: `gettingStartedTour.spec.ts` > "leaves the nudge until the upgrade dialog closes" asserts the nudge appears after a paywall-parked tour, and it is right to. That tour is stuck on a button that will never run, so the templates are the only place left to send the user. Suppressing on `markSeen: false` swept up `postponed` and `trigger_lost` along with the case the issue actually reported. Narrowed to the one ending that raises an error toast of its own — `target_timeout` — which is the "error toast and a congratulation for the same event" in #14622. `postponed` and `trigger_lost` keep their nudge and now get the `noTour` copy instead of the congratulation they used to get. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@MaanilVerma you are right and I am withdrawing the visibility half of this. Your decision that the nudge is outcome-blind is deliberate and documented — your #14144 comment says it directly: "a user who saw no tour is the one who most needs somewhere to go next, so suppressing it there removes the way forward. What was wrong was the copy claiming they'd made something." And you had already shipped exactly that: So this PR suppressing the nudge on Reduced to the one piece that is genuinely additive, if you want it: If that is useful, say so and I will strip this branch to just that. If not, close it — no argument from me on the visibility behaviour. |
|
Follow-up — the branch narrowed since my last comment, and the repo's own e2e is what forced it. Worth recording because it argues partly against me. My first version suppressed the nudge for every ending The mistake was conflating two different questions: So the branch is now down to one ending —
@MaanilVerma that reduces the disagreement to a single question, and it is yours to settle: when the tour dies with a "something went wrong" toast on screen, should the nudge still appear? Everything else now matches your outcome-blind rule. If your answer is still "always visible", say so and I will strip this to the 373 unit tests green; typecheck, lint, format, knip clean; mutation check still turns 4 controller tests red. |
|
Closing — @MaanilVerma's call that the nudge stays outcome-blind, and he'd already documented why on #14144: "a user who saw no tour is the one who most needs somewhere to go next." He'd also already fixed the real bug, the copy — Recording two things that came out of this so they aren't lost with the branch: The repo's own e2e caught my agent overreaching. The first rule suppressed the nudge for every ending marked The one genuinely additive piece, if anyone wants it later: |
|
Correcting myself on the reason I gave for closing this. I wrote that the copy bug was already fixed on
So the sharp case from the original issue — an error toast and a congratulation for the same event — is still live. @MaanilVerma's fix covered the never-started case, which is the one his probe reproduced. Closing this PR was still right: it changed visibility, and outcome-blind is his call. But the copy half is orthogonal to that call and is the thing he himself named as the actual defect — "What was wrong was the copy claiming they'd made something." Opening a narrow follow-up that keeps the nudge always visible and only chooses |
…ded (Comfy-Org#14677) ## Summary The first-run nudge picked its copy from whether a tour _started_, so a user who skipped on step 1 — or one whose tour was torn down by a target that never mounted — was congratulated for a first result they never made. This makes the copy follow how the tour actually ended. **The nudge's visibility is unchanged: every ending still shows it.** ## The defect `FirstRunTourNudge.vue` chose its copy with `` `…nudge.${tourWasShown ? 'ran' : 'noTour'}` ``, and `tourWasShown` was assigned from `started` — the return of `engine.startTour()` in `useFirstRunTourController.ts` — which means _a tour began_, not _the user completed one_: | ending | copy before | copy now | | -------------------------------- | ------------------------------------------------ | -------- | | completed | `ran` | `ran` | | never appeared | `noTour` ✓ | `noTour` | | **skipped on step 1** | **`ran`** — "You just made your first" | `noTour` | | **abandoned (`target_timeout`)** | **`ran`**, shown alongside the `loadError` toast | `noTour` | | postponed (paywall) | `ran` | `noTour` | | `trigger_lost` | `ran` | `noTour` | The abandoned case is the sharp one: the same event raises an error toast and a congratulation. ## Changes - **What**: `finish()` in `onboardingTourStore.ts` already computes `outcome` and `skipReason`; the store now hands that on as `lastEnding` (a discriminated union, cleared when a new run is requested so the previous ending cannot speak for it). `useFirstRunTourController` derives `tourWasCompleted` from it in place of `tourWasShown`, and the nudge reserves the `ran` copy for `completed`. Every other ending gets `noTour`, which reads as "here's where to go next" for a skipper or an abandoner. - **Breaking**: none. `lastEnding` is additive; `tourWasShown` had exactly one consumer (the nudge). ## Visibility is deliberately unchanged Every ending still arms the nudge, including the ones that now read as `noTour`. That is @MaanilVerma's product decision in Comfy-Org#14144: _"a user who saw no tour is the one who most needs somewhere to go next, so suppressing it there removes the way forward."_ Comfy-Org#14672 tried to suppress the nudge on the abandoned ending and was closed for exactly this reason — this PR treats the copy, not the visibility, as the defect. The regression guard for that is `still offers the nudge after a tour $named` in `useFirstRunTourController.test.ts`, which asserts `nudgeArmed` for all five endings, plus a sixth test for the never-started case and a component-level check that the nudge renders whether or not the tour finished. ## Tests New coverage, 19 tests: - `onboardingTourStore.test.ts` (6) — the ending recorded for a user skip, a `target_timeout` teardown, a `trigger_lost` teardown, a walked-to-the-end tour; that a running tour has no ending; that a new run drops the last one. - `useFirstRunTourController.test.ts` (10, plus an existing never-started test extended to assert visibility) — `completed` → congratulation; `user` / `target_timeout` / `postponed` / `trigger_lost` → none; **all five endings still arm the nudge**; never-started arms it too, with no congratulation. - `FirstRunTourNudge.test.ts` (3) — completed → `ran`, unfinished → `noTour`, and the nudge appears either way. ### Mutation evidence - Revert the rule to the old semantics (`tourWasCompleted = ending?.tour === 'firstRun'`, i.e. "a tour ran"): **4 failed | 45 passed**. The four are the skipped/abandoned/postponed/lost-context copy cases. The five visibility tests stayed green, confirming they are not coupled to the copy rule. - Pin the component's copy key to `ran`: **1 failed | 10 passed**. Pin it to `noTour`: **1 failed | 10 passed**. - Restored: **374 passed (23 files)** across `src/renderer/extensions/firstRunTour` and `src/platform/onboarding`. `pnpm install --frozen-lockfile`, `format:check`, `lint`, `typecheck` and `knip` all clean. ## Review Focus - `lastEnding` carries `skipReason` although no production code reads it today — the tests name each ending, so the reason has to be observable, and it is the shape `finish()` already computes. Say the word if you would rather it be trimmed to `{ tour, outcome }`. - The controller latches `tourWasCompleted` when the nudge arms rather than deriving it live, so a later tour ending cannot rewrite the copy of a nudge already on screen. - Not verified: no e2e run. `gettingStartedTour.spec.ts` → "leaves the nudge until the upgrade dialog closes" asserts visibility only (postponed still arms), and no e2e or unit test elsewhere asserts the nudge's copy strings — checked by grep, not by running Playwright. - Telemetry is untouched: `nudge_shown` and `explore_templates_clicked` still report only `{ tour: 'firstRun' }`, so the funnel still cannot tell a toured user from a tour-less one. That is a real gap, and deliberately out of scope here. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes #14622
The failure
The post-tour nudge decided its copy from whether a tour started, not from how it ended.
useFirstRunTourControllersettourWasShown = await engine.startTour('firstRun'), andFirstRunTourNudge.vuerendered the congratulatoryrancopy — "That was one of hundreds — You just made your first." — for any truthy value.So every ending that followed a started tour was congratulated:
abandonStep→finish('skipped', { skipReason: 'target_timeout' })) also raises theonboardingCoachmarks.loadErrortoast. The user got an error toast and a congratulation for the same event.Separately,
nudge_shownfired with only{ tour: 'firstRun' }, so the funnel could not tell the users the tour never reached from the users who saw one.The fix
finish()already computesoutcomeandskipReasonbut kept them to itself. The store now records them:exposed as
lastEnding, cleared when a new run is requested so a stale ending cannot speak for the run in progress. The controller arms the nudge off that instead of offstarted.What each ending shows, and why
completedran— there is a first result to congratulateskipped,skip_reason: usernoTourskipped,postponed(paywall)noTourskipped,trigger_lostnoTourskipped,target_timeoutnoTour(unchanged)Every ending still leaves the user somewhere to go next, so the nudge keeps appearing. What changes is that
ran— "You just made your first" — is now reserved for the one ending that made a first result.The single suppression is the case the issue reported: a tour abandoned because a target never mounted already raises the
loadErrortoast, and a panel arriving beside an error to celebrate it is the sharp case.I got this wrong first and the cloud e2e caught it. My initial rule suppressed the nudge for every ending
finish()calls withmarkSeen: false, which swept inpostponedandtrigger_losttoo.gettingStartedTour.spec.ts> "leaves the nudge until the upgrade dialog closes" failed, and it was right to: a paywalled tour is parked on a button that will never run, so the templates are the only place left to send that user. The rule is now narrowed to the one ending that raises an error toast of its own. Those two endings keep their nudge and lose only the congratulation.The judgement call — a deliberate skip. Nudge kept, congratulation dropped. Waving the tour away rejects guidance, not the product; a pointer at the templates is still the right next offer, and it is the one thing the nudge exists to do. What was wrong was claiming the user made something.
noTour's copy — "Hundreds more where that came from / Browse the templates and pick something to build" — reads correctly for a skipper, so no new locale key was added (onlyencarries these keys today; a new one would sit untranslated in the other 11 locales).Second half —
nudge_shownfor a tour that never startedHandled.
OnboardingTourNudgeMetadatagains a requiredtour_outcome: 'completed' | 'skipped' | 'not_started', carried by bothnudge_shownandexplore_templates_clicked. The two populations are now separable downstream, and because both ends of the funnel carry it, conversion is readable per population rather than only in aggregate. I chose the outcome over aran/noTourcopy flag because the copy is derivable from it and the outcome is not derivable from the copy.Verified
pnpm test:unit src/renderer/extensions/firstRunTour/ src/platform/onboarding/— 23 files, 368 tests, green.useFirstRunTourController.test.ts(completed / user-skip / target_timeout / postponed / trigger_lost / never-resolved), three inFirstRunTourNudge.test.tsfor the copy each outcome renders, one for the telemetry field, and five inonboardingTourStore.test.tsforlastEndingitself (including that a new run drops the previous ending).armNudge()to the pre-fix behaviour (arm unconditionally; outcome fromstarted) turns 4 of the new controller tests red —congratulates nobody for a tour the user skipped,stays away from a tour a missing target tore down, and bothstill offers somewhere to go ...cases. Restored, green again.playwright-tests (cloud): red on the first attempt (leaves the nudge until the upgrade dialog closes), which is what drove the narrowing above. Re-running on the fix.pnpm format:check,pnpm lint,pnpm typecheck,pnpm knipall clean.Not verified
target_timeoutsuppression has no browser coverage — it needs an 8 s deferred-target timeout to reproduce, and the only e2e that touches the nudge is the paywall one. Unit-level only.TourEnding, typed mock ref, per-outcome telemetry matrix) is applied in follow-up commits.tour_outcomeis a new property on two existing Mixpanel events. Nothing downstream is updated here; existing dashboards keep working on the untouchedtourproperty, but a dashboard that wants the split needs the new field added.