Skip to content

Merging staging into release - #978

Merged
jordanhunt22 merged 1 commit into
releasefrom
staging
Mar 12, 2026
Merged

Merging staging into release#978
jordanhunt22 merged 1 commit into
releasefrom
staging

Conversation

@jordanhunt22

@jordanhunt22 jordanhunt22 commented Mar 12, 2026

Copy link
Copy Markdown
Collaborator

Adds support for sonnet 4.6 and make it the default.

Removes support for sonnet 4, since it is now a pretty old model.

This will now send all traffic to bedrock.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@jordanhunt22
Add bedrock support for sonnet 4.5 and 4.6 (#976)
1326966 
Adds support for sonnet 4.6 and make it the default.

Removes support for sonnet 4, since it is now a pretty old model.

This will now send all traffic to bedrock.
@vercel

vercel Bot commented Mar 12, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
chef (staging) Ready Ready Preview Mar 12, 2026 0:40am

Request Review

@github-actions

github-actions Bot commented Mar 12, 2026 •
edited
Loading

Copy link
Copy Markdown

Braintrust eval report

chef (chef-claude-4.6-sonnet-41d1f5ce)

Score Average Improvements Regressions
1/Deploys 50% (+0pp) - 1 🔴
IsSuccess 100% (+0pp) - -
Llm_calls 9.75 (+5.75) 2 🟢 1 🔴
Tool_calls 0 (+0) - -
Errors 1.5 (-1) 2 🟢 1 🔴
Llm_errors 0 (-0.75) 3 🟢 -
Tool_errors 0 (+0) - -
Prompt_tokens 286357.5tok (+210886.75tok) - 4 🔴
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Completion_tokens 14245tok (+12006tok) - 4 🔴
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 300602.5tok (+222892.75tok) - 4 🔴
Estimated_cost 1.07$ (+0.81$) - 4 🔴
Duration 183.08s (+111.19s) - 4 🔴
Llm_duration 151.44s (+120.15s) - 4 🔴

‼️ chef (chef-claude-4.5-sonnet-0a20813b)

Score Average Improvements Regressions
Llm_calls 4 (-7.75) - 4 🔴
Tool_calls 0 (+0) - -
Errors 2.5 (+0.25) 1 🟢 2 🔴
Llm_errors 0.75 (+0.75) - 3 🔴
Tool_errors 0 (+0) - -
Prompt_tokens 68375tok (-224261tok) 4 🟢 -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Completion_tokens 2193tok (-3793.5tok) 3 🟢 1 🔴
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 70568tok (-228054.5tok) 4 🟢 -
Estimated_cost 0.24$ (-0.19$) 4 🟢 -
Duration 50.49s (-101.1s) 4 🟢 -
Llm_duration 29.65s (-55.02s) 3 🟢 1 🔴
Expand to see errors 
AI_APICallError: prompt is too long: 204541 tokens > 200000 maximum
    at eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14228:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async postToApi (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14133:28)
    at async AnthropicMessagesLanguageModel.doGenerate (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:18942:9)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29803:30)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async _retryWithExponentialBackoff (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:27940:12)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29763:32)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async type (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:35083:24)
AI_APICallError: prompt is too long: 201138 tokens > 200000 maximum
    at eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14228:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async postToApi (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14133:28)
    at async AnthropicMessagesLanguageModel.doGenerate (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:18942:9)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29803:30)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async _retryWithExponentialBackoff (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:27940:12)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29763:32)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async type (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:35083:24)
AI_APICallError: prompt is too long: 202374 tokens > 200000 maximum
    at eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14228:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async postToApi (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:14133:28)
    at async AnthropicMessagesLanguageModel.doGenerate (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:18942:9)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29803:30)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async _retryWithExponentialBackoff (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:27940:12)
    at async fn (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:29763:32)
    at async eval (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:28123:22)
    at async type (eval at <anonymous> (/home/runner/work/chef/chef/node_modules/.pnpm/braintrust@0.0.199_@aws-sdk+credential-provider-web-identity@3.782.0_openai@4.93.0_ws@8.18.0__wgerilt4ezktegc3ulzicjnwli/node_modules/braintrust/dist/cli.js:6941:5), <anonymous>:35083:24)

@jordanhunt22
jordanhunt22 merged commit 1326966 into release Mar 12, 2026
4 of 6 checks passed
@jordanhunt22
jordanhunt22 deleted the staging branch March 12, 2026 01:20

This branch was successfully deployed

3 active deployments
Production — 13269667 Deployed Mar 12, 2026 by vercel[bot]
View deployment
staging — 13269667 Deployed Mar 12, 2026 by vercel[bot]
View deployment
Preview — 13269667 Deployed Mar 11, 2026 by vercel[bot]
View deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jordanhunt22