Skip to content

ci(safeway): gate MCP evals in Modal - #71

Merged
walkerhughes merged 1 commit into
codex/safeway-marketplacefrom
codex/safeway-evals
Sep 8, 2026
Merged

walkerhughes merged 1 commit into
codex/safeway-marketplacefrom
codex/safeway-evals

Conversation

@walkerhughes

Copy link
Copy Markdown
Owner

Summary

  • add four deterministic Harbor tasks for nearest-store selection, milk pricing, product refresh, and basket totals
  • grade both exact task outcomes and required Safeway MCP usage
  • reject direct mock-endpoint bypasses, including delegated tool calls
  • run unit checks and the Harbor gate in Modal on Safeway changes
  • bump the plugin version to 0.1.1 so Claude Code refreshes its cache

Verification

  • make check
  • make coverage (89%)
  • 28 local process-check cases and 16 local outcome-check cases
  • Harbor 0.18 Modal job configuration resolution
  • official plugin and skill validators

Stack

Depends on #69, which depends on #68.

@walkerhughes
walkerhughes merged commit 10f5c5b into main Sep 8, 2026
2 of 3 checks passed
@walkerhughes
walkerhughes deleted the codex/safeway-evals branch September 8, 2026 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant