Skip to content

ci(target): gate MCP evals in Modal - #65

Merged
walkerhughes merged 1 commit into
codex/target-marketplacefrom
codex/target-evals
Sep 8, 2026
Merged

walkerhughes merged 1 commit into
codex/target-marketplacefrom
codex/target-evals

Conversation

@walkerhughes

Copy link
Copy Markdown
Owner

Summary

  • add four deterministic Harbor tasks for Target search, TCIN detail, pickup availability, and shopping-list pricing
  • score answer correctness independently from expected MCP use and endpoint bypasses
  • run the full Claude Code agent loop in Modal
  • pin Harbor 0.18.0 and install the exact PR SHA in each eval image

Validation

  • make check
  • make coverage (89.22%)
  • 28 process scenarios and 16 outcome scenarios across four evals
  • Harbor Modal configuration parse
  • official plugin and skill validators
  • workflow YAML parse

Stack

Depends on #64, which depends on #63.

stack merge was automatically disabled September 7, 2026 23:44

Pull Request is not mergeable

stack merge was automatically disabled September 7, 2026 23:58

Pull Request is not mergeable

stack merge was automatically disabled September 7, 2026 23:59

Pull Request is not mergeable

@walkerhughes
walkerhughes merged commit 1e6e54c into main Sep 8, 2026
5 of 6 checks passed
@walkerhughes
walkerhughes deleted the codex/target-evals branch September 8, 2026 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant