Skip to content

test(webview): use the real VS Code toolkit in Playwright visual baselines - #2163

Draft
zoomote2[bot] wants to merge 2 commits into
mainfrom
test/webview-gallery-real-toolkit
Draft

test(webview): use the real VS Code toolkit in Playwright visual baselines#2163
zoomote2[bot] wants to merge 2 commits into
mainfrom
test/webview-gallery-real-toolkit

Conversation

@zoomote2

@zoomote2 zoomote2 Bot commented Oct 24, 2026

Copy link
Copy Markdown

​Created by Roomote. View the session.

Related GitHub Issue

Closes: #2105

Description

The Playwright Story Gallery mocked the VS Code toolkit. Visual baselines rendered the mock, not the real components. Baselines did not match the extension (labels below inputs, no padding or borders).

This PR removes the toolkit mock from the gallery. Every story that uses toolkit components now renders the real components. This extends the approach from PR #1299 (Ollama story) to all stories. It replaces the per-story allowlist, which would need about 60 entries.

Key changes:

  • Remove the gallery toolkit mock. The Vitest unit-test mock stays unchanged.
  • Regenerate all affected baselines in the pinned Playwright container on CI.
  • 26 baseline diffs across 9 stories. All diffs are expected pixel changes: labels now sit above inputs, and text fields, checkboxes, and links show real toolkit borders and padding.

Points for careful review:

  • Confirm the new baselines match the rendered extension UI.
  • The flaky-test count is unchanged. Unchanged main and this branch produce the same 68/5 failure split on a host run. All failures are pre-existing host font-metric pixel noise. No functional failures. No net-new failures from this change.

Test Procedure

  1. Run the full webview visual suite on unchanged main on the host. Record the failure set (68 failed, 5 flaky, all pixel diffs from host font metrics).
  2. Run the same suite on this branch on the host. Confirm the failure set is identical. This proves no net-new failures.
  3. Push the branch and run the pinned-container visual-regression.yml workflow. Take the container-rendered actuals as the new baselines.
  4. Re-run the workflow to verify. Result: all 73 webview visual tests pass in the container. This includes the contrast audits and the layout contracts. The theme-fixture and extension-host visual suites also pass.
  5. Run pnpm typecheck. Result: pass.

Note: the repository forbids host-generated baselines. All committed baselines come from the pinned CI container.

Pre-Submission Checklist

  • Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
  • Scope: My changes are focused on the linked issue (one major feature/fix per PR).
  • Self-Review: I reviewed my own code with care.
  • Writing Style: The PR title and description use Simplified Technical English (ASD-STE100): short sentences, plain words, active voice, one idea per sentence, no contractions.
  • Testing: New and/or updated tests have been added to cover my changes (if applicable).
  • Visual Snapshot (UI changes only): This PR updates the committed *.visual.tsx baselines in webview-ui/.
  • Documentation Impact: I checked if my changes need documentation updates (see "Documentation Updates" section below).
  • Contribution Guidelines: I read and agree to the Contributor Guidelines.

Visual Snapshots

The primary artifacts are the 26 regenerated committed baselines across 9 stories. Before/after pairs were reviewed during validation: labels move above inputs, and inputs show real toolkit borders and padding.

Videos (interaction / animation only)

Not applicable. No interaction or animation changes.

Documentation Updates

  • No documentation updates are required.
  • Yes, documentation updates are required. (Describe what to update, or link to a PR in the docs repository).

This change is test infrastructure only. It changes no user-facing functionality.

Additional Notes

Baselines were regenerated by the pinned Playwright container through the visual-regression.yml workflow, per the documented fallback for environments without Docker.

Get in Touch

Discord: 0x7777777_

@roomote
test(webview): render the real VS Code toolkit in every gallery story
52cac98 
The gallery vite config resolved @vscode/webview-ui-toolkit/react to the
Vitest mock for every story except realToolkitImporters (only the Ollama
story). The mock draws a plain input with the label after the field, so
baselines showing toolkit components did not look like the extension,
which renders the real custom elements with the label above the field,
padding, and borders.

Drop the playwright-gallery-toolkit-mock plugin so every story imports
the real toolkit, and keep pre-bundling it via optimizeDeps.include so a
cold Vite cache does not re-optimize and reload mid-test. The Vitest mock
stays in place for unit tests.
@roomote
test(webview): regenerate toolkit baselines from the pinned container
4e740da 
Baselines for the nine stories that render toolkit components, captured
by the pinned mcr.microsoft.com/playwright:v1.62.1-noble container via
the Visual Regression workflow: auto-approve settings, Bedrock custom
ARN, empty history, IO Intelligence, OpenAI-compatible Azure guidance
and extra body, telemetry banner, terminal settings, and UI settings.

The real toolkit draws text-field labels above their inputs and gives
checkboxes, textareas, links, and buttons the extension's borders and
padding, so these baselines now match what users see.
@coderabbitai

coderabbitai Bot commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one): 1
  • coderabbit-review-active

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@zoomote2

zoomote2 Bot commented Oct 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

No code issues found. The gallery vite config change is clean, and the 26 regenerated baselines are consistent with the pinned-container webview-visual run that passed on this exact commit. See session

Reviewed 4e740da

@github-actions

github-actions Bot commented Oct 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review status

This PR was opened by an automated account. A human maintainer must verify the change intent, provenance, and validation before merging.

Current step: Mark the PR ready. Required CI must pass before CodeRabbit starts.

Review-state labels are managed by this workflow; do not edit them manually. community-approved is managed the same way — do not add or remove it manually. It signals a fresh community code approval for the current head as an advisory priority only; maintainer review is still required.

@codecov

codecov Bot commented Oct 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@taltas taltas Awaiting requested review from taltas taltas will be requested when the pull request is marked ready for review taltas is a code owner
@JamesRobert20 JamesRobert20 Awaiting requested review from JamesRobert20 JamesRobert20 will be requested when the pull request is marked ready for review JamesRobert20 is a code owner
@edelauna edelauna Awaiting requested review from edelauna edelauna will be requested when the pull request is marked ready for review edelauna is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

test(webview): use the real VS Code toolkit in Playwright visual baselines

1 participant

@roomote