Skip to content

Update VS Code LM model metadata and regression tests - #2255

Merged
taltas merged 1 commit into
Zoo-Code-Org:mainfrom
simurg79:update/vscode-lm-model-metadata
Oct 30, 2026
Merged

taltas merged 1 commit into
Zoo-Code-Org:mainfrom
simurg79:update/vscode-lm-model-metadata

Conversation

@simurg79

@simurg79 simurg79 commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add Astra, Grok 4.5/4.6, Gemini 3.7/3.8 Flash, and 13 additional measured models.
  • Set the Opus 5.5 accepted token budget to 677,108 and Astra to 266,789; refresh Opus 5 and Gemini 3.6 budgets.
  • Use borrowed rather than measured values for Fable 5.5; omit Astra pricing because it has not been measured.
  • Update metadata, provider, and selected-model regression tests.

Validation

  • Previously completed validation: 704 tests, type checks, and lint passed.
  • Commit hooks: ESLint and formatting passed.
  • Push hooks: all 11 type-check tasks passed.
  • Only the four intended files are included; no changesets or changelog edits.

@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 159c42e and 845002d.

Files selected for processing: 4
  • packages/types/src/__tests__/vscode-llm.spec.ts
  • packages/types/src/providers/vscode-llm.ts
  • src/api/providers/__tests__/vscode-lm.spec.ts
  • webview-ui/src/components/ui/hooks/__tests__/useSelectedModel.spec.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Recent review details
Context from checks skipped due to timeout: 6
  • GitHub Check: CodeRabbit
  • GitHub Check: webview-visual
  • GitHub Check: extension-host-visual
  • GitHub Check: platform-unit-test (ubuntu-latest)
  • GitHub Check: platform-unit-test (windows-latest)
  • GitHub Check: e2e-mock
Additional context used
Path-based instructions: 7
Treat model, provider, MCP, path, command, and tool data as untrusted.

CodeRabbit configuration file

Files:

  • src/api/providers/__tests__/vscode-lm.spec.ts
For persisted settings, verify the complete schema/storage/runtime/webview round trip, shared default semantics, and focused true plus false/unset tests.

CodeRabbit configuration file

Files:

  • packages/types/src/__tests__/vscode-llm.spec.ts
  • packages/types/src/providers/vscode-llm.ts
Require regression coverage at the lowest valid harness with behavior-focused assertions, including relevant negative, error, false/unset, and boundary cases.

CodeRabbit configuration file

Files:

  • webview-ui/src/components/ui/hooks/__tests__/useSelectedModel.spec.ts
  • src/api/providers/__tests__/vscode-lm.spec.ts
  • packages/types/src/__tests__/vscode-llm.spec.ts
Check strict typing and exhaustive behavior across normal, boundary, error, cancellation, retry, and compatibility paths.

CodeRabbit configuration file

Files:

  • webview-ui/src/components/ui/hooks/__tests__/useSelectedModel.spec.ts
  • src/api/providers/__tests__/vscode-lm.spec.ts
  • packages/types/src/__tests__/vscode-llm.spec.ts
  • packages/types/src/providers/vscode-llm.ts
Check React state and effect dependencies, cleanup, accessibility, i18n, and light/dark theme behavior.

CodeRabbit configuration file

Files:

  • webview-ui/src/components/ui/hooks/__tests__/useSelectedModel.spec.ts
Verify extension/webview contracts, cancellation and error propagation, VS Code lifecycle correctness, and behavior under retries and partial failure.

CodeRabbit configuration file

Files:

  • src/api/providers/__tests__/vscode-lm.spec.ts
Act as an adversarial second-opinion reviewer.

CodeRabbit configuration file

Files:

  • webview-ui/src/components/ui/hooks/__tests__/useSelectedModel.spec.ts
  • src/api/providers/__tests__/vscode-lm.spec.ts
  • packages/types/src/__tests__/vscode-llm.spec.ts
  • packages/types/src/providers/vscode-llm.ts
📝 Summary

Summary by CodeRabbit

  • New Features
    • Added support for additional Claude, Grok, Gemini, MAI, and GPT model variants.
    • Model listings now reflect image support and input-token limits for supported models, including updated limits for select existing models.

The VS Code language-model catalog adds model entries and updates assigned input-token budgets. Tests cover model metadata, image-capability reporting, advertised context windows, and model-selector budget resolution.

Changes: VS Code LM model catalog

Layer / File(s) Summary
Model catalog and metadata assertions
packages/types/src/providers/vscode-llm.ts, packages/types/src/__tests__/vscode-llm.spec.ts
The catalog adds Claude, Grok, Gemini, MAI, and GPT model entries and updates selected input-token budgets. Tests assert model values, capabilities, omitted prices, and ordering.
Provider capability and budget tests
src/api/providers/__tests__/vscode-lm.spec.ts
Tests check image-capability reporting and assigned budgets alongside advertised context windows. Astra budget resolution is tested across vendors.
Model selector budget tests
webview-ui/src/components/ui/hooks/__tests__/useSelectedModel.spec.ts
Tests check assigned input-token budgets for 24 model families and verify Astra’s budget across vendors.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Possibly related PRs

Suggested labels: coderabbit-review-active

Merge Risk: Minimal · up to 84500

The catalog and regression-test updates have no established merge-blocking defect. The borrowed Fable budget is disclosed, and no incompatible Astra vendor limit was demonstrated. Mergeable subject to normal checks.

Pre-merge checks: Passed 7 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Description check Warning The description summarizes the implementation and validation, but it omits the required linked issue, test procedure details, pre-submission checklist, documentation impact, and contributor contact se… Add an approved issue reference using “Closes: #”. Include the required template sections, complete the pre-submission checklist, describe repeatable test steps and relevant environments, state the documentation impact, and provide …
Passed Passed checks: 7 passed
Check name Status Explanation
Title check Passed The title clearly summarizes the main changes to VS Code LM metadata and regression tests.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Regression Evidence Passed PASS. The changed model rows and revised budgets have focused coverage. packages/types/src/__tests__/vscode-llm.spec.ts checks the added rows, changed Opus 5 and Gemini 3.6 values, omitted prices, i…
Security Boundaries Passed PASS. The pull request changes only the static VS Code LM metadata table and regression tests in four files. The new entries contain model names, capability flags, budgets, and public documentation li…
Persistence Integrity Passed The pull request changes only static VS Code LM metadata and regression tests. packages/types/src/providers/vscode-llm.ts defines an in-memory constant; it adds or updates model fields and keeps the…
Lifecycle Resource Cleanup Passed No changed lifecycle path can leak a resource or duplicate work. The only non-test production change is static model metadata in packages/types/src/providers/vscode-llm.ts; it adds no listeners, wat…
Full details: Regression Evidence

Explanation

PASS. The changed model rows and revised budgets have focused coverage. packages/types/src/__tests__/vscode-llm.spec.ts checks the added rows, changed Opus 5 and Gemini 3.6 values, omitted prices, image-disabled Grok rows, Fable 5.5 copying, and ordering. Provider tests exercise the assigned budgets while live limits differ, Astra selector resolution for two vendors, image capability true/false/unset cases, and unknown-family fallbacks. useSelectedModel.spec.ts checks the budgets for all added or changed families and the Astra vendor case. The PR changes metadata and tests only. It does not change a UI component, layout, or theme, so a Playwright component snapshot is not required.

Full details: Security Boundaries

Explanation

PASS. The pull request changes only the static VS Code LM metadata table and regression tests in four files. The new entries contain model names, capability flags, budgets, and public documentation links; they do not contain secrets or PII. No changed path adds input execution, network or environment access, or a bypass of approval or allowlist controls. Runtime consumers still perform exact family-key lookups and use the existing selector validation.

Full details: Persistence Integrity

Explanation

The pull request changes only static VS Code LM metadata and regression tests. packages/types/src/providers/vscode-llm.ts defines an in-memory constant; it adds or updates model fields and keeps the default ID unchanged at line 50. The consumers read this table synchronously for condensing and UI context limits (src/api/providers/vscode-lm.ts:1634-1644, webview-ui/src/components/ui/hooks/useSelectedModel.ts:453-499). No changed file performs persistence, asynchronous writes, rollback handling, or default-state propagation.

Full details: Lifecycle Resource Cleanup

Explanation

No changed lifecycle path can leak a resource or duplicate work. The only non-test production change is static model metadata in packages/types/src/providers/vscode-llm.ts; it adds no listeners, watchers, timers, tasks, providers, or cancellation sources. The new handler test creates selectedHandler and disposes it in finally; the shared test handler is disposed by the existing afterEach cleanup.

Full details: Description check

Explanation

The description summarizes the implementation and validation, but it omits the required linked issue, test procedure details, pre-submission checklist, documentation impact, and contributor contact sections.

Resolution

Add an approved issue reference using “Closes: #<number>”. Include the required template sections, complete the pre-submission checklist, describe repeatable test steps and relevant environments, state the documentation impact, and provide the contributor contact.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

packages/types/src/__tests__/vscode-llm.spec.ts

ESLint skipped: missing config or dependency (missing-dependency). The ESLint configuration references a package that is not available in the sandbox.

packages/types/src/providers/vscode-llm.ts

ESLint skipped: the matched ESLint configuration already failed (missing-dependency).

src/api/providers/__tests__/vscode-lm.spec.ts

ESLint skipped: missing config or dependency (missing-dependency). The ESLint configuration references a package that is not available in the sandbox.

  • 1 others

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Review status

Thanks for contributing. This comment tracks the review sequence and the next action.

Current step: The required review sequence passed. Remaining merge requirements apply.

Review-state labels are managed by this workflow; do not edit them manually. community-approved is managed the same way — do not add or remove it manually. It signals a fresh community code approval for the current head as an advisory priority only; maintainer review is still required.

@simurg79

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@codecov

codecov Bot commented Oct 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions github-actions Bot added the awaiting-maintainer CodeRabbit approved; waiting for a human maintainer label Oct 30, 2026

@taltas taltas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, thanks for the PR!

@taltas
taltas added this pull request to the merge queue Oct 30, 2026
@github-actions github-actions Bot removed the awaiting-maintainer CodeRabbit approved; waiting for a human maintainer label Oct 30, 2026
Merged via the queue into Zoo-Code-Org:main with commit 87b57cb Oct 30, 2026
25 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants