Skip to content

Assignment: AI Engineering Routing Policy — Avinash Anad - #11

Open
AvinashAnad wants to merge 1 commit into
theschoolofai:mainfrom
AvinashAnad:assignment/avi-routing-policy
Open

AvinashAnad wants to merge 1 commit into
theschoolofai:mainfrom
AvinashAnad:assignment/avi-routing-policy

Conversation

@AvinashAnad

Copy link
Copy Markdown

Summary

  • Custom 16-task AI engineering workload (tokenization, cost calculations, budget projections, caching economics, break-even analysis)
  • Three-rung cross-model ladder: economy (gpt-oss-120b/Groq) < standard (flash-lite/Gemini) < frontier (gpt-4.1/GitHub)
  • All five proofs pass (p1, p2, p3, p4, p7)
  • Signature failure mode observed: economy rung 97.2% cheaper per call, 26.5% more expensive per resolved task
  • Break-even resolution rate: 7.8% (economy resolved 6.25%, below threshold)
  • Budget enforcement holds under adversarial attack (200 rounds, 2 admitted, 198 refused)

Evidence

See EVIDENCE.md for full results, trace hierarchy, honest limitation, and reproduction commands.

Test plan

  • uv run pytest -q — 277 passed
  • p1_cost_per_task — signature failure mode observed
  • p2_budget_holds — ceiling held, downgrade and refusal verified
  • p3_denial_of_wallet — adversarial loop stopped by controller
  • p4_trace_export — span costs match ledger exactly
  • p7_cross_model_ladder — each rung is a different model, downgrades work

🤖 Generated with Claude Code

Custom task set spanning tokenization, cost calculations, budget projections,
caching economics, trace hierarchies, and break-even analysis. All five proofs
pass offline. Signature failure mode observed: economy rung 97.2% cheaper per
call but 26.5% more expensive per resolved task.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 7, 2026 19:36

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants