BYOK: users bring their own model and Tavily keys; free allowance on the platform key; Settings page

[coderconnoisseur]

BYOK (SPEC-BYOK, plan docs/superpowers/plans/2026-10-16-byok.md). A user can paste their own key for OpenAI, Anthropic, Gemini, Groq or OpenRouter, plus Tavily for web search. From then on Ask Lumen, invoice reading and email parsing use that key, with a model Lumen picks from a curated list. Users without a key get a free allowance on the platform key, then a prompt to add their own.

What changed

• llm/registry.yaml byok: catalogue (models per provider and role); OpenAI + Anthropic adapters (drop temperature, max_completion_tokens, thinking off). Env overrides are LUMEN_OPENAI_BASE_URL / LUMEN_ANTHROPIC_BASE_URL.
• user_keys table (created by create_all). Keys are encrypted with a dedicated KEYS_ENCRYPTION_KEY that has no SECRET_KEY fallback: without it, saving returns 503.
• llm/keys.py: a contextvar holding the request's user, set by require_auth, FastAPI agent/documents ask and the poller. With a user key for a role the chain is only the user's providers, never the platform's. A role no user key covers (e.g. vision for Groq-only) uses the platform chain. With no user in context, behaviour is byte-identical, so eval cassettes still replay.
• A rejected key (401, credits, or a 403 config error) gives 400 key_rejected ("Your OpenAI key was rejected. Update it in Settings."), sets user_keys.last_error, and is never retried on the platform key. A stored key that can't be decrypted fails without sending anything. Keys are masked in every log, error and response, including the shared 422 handler, which no longer logs Pydantic input.
• /api/keys: GET (last 4 only), PUT (validated with one cheap call, 5/min), POST test, DELETE.
• Email poller: a dead user key stops the poll and later polls skip that user until the key is fixed. An invoice over the free read allowance is skipped (retried later) while other emails keep parsing.
• Free allowance (config): PLATFORM_QUESTIONS_PER_DAY=20 for every keyless user, counted on /chat, /api/agent/ask and /api/documents/ask. PLATFORM_INVOICE_READS=20 lifetime, counted from review_items. Over either limit: 402 needs_key.
• Tavily: the user's own key runs web search without the daily caps; a bad Tavily key never stops the poller.
• /settings page (bare-bones): one row per provider with Add/Replace, Test, Remove and Advanced model picks; chat and inbox prompts link to Settings.
• Docs: CODEBASE-GUIDE (Model keys section), DEPLOY, .env.example, render.yaml, PROGRESS, STORY Record agent_sql + agent_safety; interim AGT-07 report #24, TODO.

Rulings made during the build

1. One PR instead of one per task (owner asked for one PR this session).
2. The allowance check in /api/agent/ask runs inside the user context.
3. Manual smoke with real provider keys not done (no owner keys).
4. Added the Tavily key entry (SPEC-AGENT-TOOLS §4 said it lands with BYOK).
5. Env overrides renamed to LUMEN_*_BASE_URL: Claude Code and the OpenAI SDK export ANTHROPIC_BASE_URL/OPENAI_BASE_URL, which would redirect user keys.
6. An undecryptable stored key fails as a key error instead of falling back to the platform key.
7. Only AUTH, CREDITS, or a 403 CONFIG count as key errors, so a 400 "context too long" doesn't freeze the poller.
8. Demo (anonymous) users can't save keys (403, like profile).
9. No SECRET_KEY fallback for KEYS_ENCRYPTION_KEY.
10. The poller skips only the over-allowance invoice email instead of stopping the whole poll (overrides plan line 519).
11. Documents questions count against the allowance.

Tests

• Backend: 901 passed, 15 skipped (was 857). New: test_byok_catalogue, test_byok_client, test_api_keys, test_byok_allowance, plus cases in poller/tools/documents. Includes zero-leak caplog+response tests and an eval selftest that is still green.
• Frontend: tsc --noEmit clean, lint clean, next build OK (/settings built).

Not tested

• Real provider calls (OpenAI/Anthropic/Gemini/Groq/OpenRouter/Tavily) with real keys, so the provider quirks (e.g. empty replies from thinking models under small max_tokens, Anthropic thinking: disabled) are unverified.
• The Settings page was not clicked through in a browser.

Before releasing to main (owner)

• Set KEYS_ENCRYPTION_KEY (random 32+ chars) on Render lumen-api-sg. render.yaml's generateValue only applies to new blueprint services. Without it, saving a key returns 503, and nothing else breaks.
• Signed-up keyless users are now capped at 20 questions/day (the demo cap now applies to everyone without a key).

Deferred minors (25, all triaged "can wait" by the whole-branch review): e.g. the Advanced model pick only saves with Replace, failed non-invoice /extract reads aren't counted, and a rejected Tavily key keeps the tool offered.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
lumen	Ready	Preview	Oct 19, 2026 7:33pm UTC