diff --git a/docs/configuration.md b/docs/configuration.md index 23b9785..3f068b9 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -31,6 +31,8 @@ nav_order: 10 - `MODELRELAY_HOST` (interface to listen on; same as `--host`) - `MODELRELAY_API_JSON_LIMIT` (admin API request body limit, default `1mb`) - `MODELRELAY_JSON_LIMIT` (`/v1` proxy request body limit, default `10mb`) + - `MODELRELAY_OPENCODE_FREE_COMPAT` (`1` enables the OpenCode Zen free-tier compatibility mode, see below) + - `MODELRELAY_OPENCODE_FREE_TIER_USER_AGENT` (overrides the User-Agent sent in that mode) Kiro OAuth notes: - Base endpoint is preconfigured to `https://codewhisperer.us-east-1.amazonaws.com/generateAssistantResponse` @@ -39,6 +41,38 @@ Kiro OAuth notes: - `KIRO_REFRESH_TOKEN`, or - `~/.aws/sso/cache` (auto-detected refresh token), following OmniRoute’s approach. +## OpenCode Zen free-tier compatibility + +OpenCode's free models only answer requests that carry the request shape of the official OpenCode +client. By default modelrelay does not imitate that client, so those models are listed as +`unsupported` and are not routed to. The reasoning and the measured gate conditions are in +[Free tier notes](free-tiers.md). + +Set `opencodeFreeTierCompat` in `~/.modelrelay.json` to enable the mode: + +```jsonc +{ + "opencodeFreeTierCompat": true +} +``` + +Or set the environment variable, which takes precedence over the config file: + +```bash +export MODELRELAY_OPENCODE_FREE_COMPAT=1 +``` + +Only `1`, `true`, `yes` and `on` enable it; anything else leaves it off, so a typo cannot silently +start rewriting requests. + +The mode sends a User-Agent of `opencode/1.19.3`. To use a different one — for example when that +version goes stale — set `opencodeFreeTierUserAgent` in the config file or +`MODELRELAY_OPENCODE_FREE_TIER_USER_AGENT` in the environment. + +**The mode imitates the official OpenCode client, which may conflict with OpenCode's terms of +service, and it can stop working at any time.** modelrelay prints a warning at startup and shows +one in the dashboard while the mode is on. Enabling it is your decision and your responsibility. + For hosted Ollama, set `OLLAMA_API_KEY` and optionally override `OLLAMA_BASE_URL` / `OLLAMA_MODEL`. If you leave the Ollama base URL blank in the UI, modelrelay defaults to `https://ollama.com/v1`. With a valid Ollama API key, modelrelay will discover available Ollama models automatically. diff --git a/docs/dashboard.md b/docs/dashboard.md index 459bd29..4db0aa3 100644 --- a/docs/dashboard.md +++ b/docs/dashboard.md @@ -28,7 +28,7 @@ The search box matches plain words against the model name, provider, id and tags |---|---| | `tag:coding` | models carrying that tag (curated or your own); `tag:coding,fast` requires both | | `provider:groq` | provider key containing the text | -| `status:up` | exact status (`up`, `down`, `disabled`, `excluded`, `banned`, `noauth`) | +| `status:up` | exact status (`up`, `down`, `disabled`, `excluded`, `banned`, `noauth`, `unsupported`) | | `ctx>=64k`, `ctx:64k` or `min_ctx:64k` | effective context of at least that size (`k` and `m` suffixes work) | For example `qwen tag:coding ctx>=64k provider:groq`. diff --git a/docs/free-tiers.md b/docs/free-tiers.md index bc192c3..2363084 100644 --- a/docs/free-tiers.md +++ b/docs/free-tiers.md @@ -12,7 +12,7 @@ What free tiers do in practice, measured against real providers in October 2026. | Provider | What we saw | |---|---| | OpenRouter | Free models are limited to 20 requests per minute, and 50 per day until you have bought $10 of credits in total (all time); at $10 or more the daily limit is 1,000. The limit is per account, not per key, so a new key does not reset it. A key spending limit of $0 still allows `:free` models. See [OpenRouter's limits](https://openrouter.ai/docs/api-reference/limits). | -| OpenCode Zen | The free models answer `403 FreeTierError` ("the free tier can only be used from within OpenCode") for any other client. A router cannot use them with any key. | +| OpenCode Zen | The free models answer `403 FreeTierError` ("the free tier can only be used from within OpenCode") for any other client. By default modelrelay does not imitate the OpenCode client, so these models are listed as `unsupported` and are not routed to. See [OpenCode Zen free models](#opencode-zen-free-models). | | Ollama Cloud | The free plan answers `402` for the premium models. | | Cerebras | Answers `402` for models outside the free allowance. | | Google AI | Free quotas are very small, or zero for some models. | @@ -20,6 +20,81 @@ What free tiers do in practice, measured against real providers in October 2026. Free models on several providers may train on the prompts you send. Do not route private code or data through them. +## OpenCode Zen free models + +OpenCode's free models are not restricted by account or entitlement. They are restricted by +*request shape*: the Zen endpoint only answers requests that look like they came from the official +OpenCode client. A request that does not is refused with `403 FreeTierError` and the message +"OpenCode's free tier can only be used from within OpenCode". + +**By default modelrelay respects that restriction.** The free models appear in the model list with +status `unsupported` and are never routed to. A `403` from OpenCode describes the request, not a +retired model, so it is not a "try another model" signal either — the model is alive, the request +is not what it wants. + +### Opt-in compatibility mode + +If you want the free models anyway, you can turn on the compatibility mode. It makes modelrelay +imitate the official OpenCode client: it sends a matching User-Agent, a session id in the format +the endpoint expects, `stream: true`, and a tool named exactly `read`. + +Set `opencodeFreeTierCompat` in `~/.modelrelay.json`, or the `MODELRELAY_OPENCODE_FREE_COMPAT=1` +environment variable. The environment variable wins over the config file. See +[Configuration](configuration.md) for details. + +**Read this before you enable it.** The mode: + +- imitates the official OpenCode client, which may conflict with OpenCode's terms of service; +- can stop working at any time, without a notice from OpenCode; +- means that you are responsible for using it. + +modelrelay prints a warning at startup and shows one in the dashboard for as long as the mode is on. + +Caveat: OpenCode's Zen free models may train on prompts, have trial-only NVIDIA endpoints, and be geographically restricted. + +### What the endpoint checks + +Four elements are required. Each one was verified individually against the live endpoint by +changing a single element at a time: + +| Element | Required value | +|---|---| +| `User-Agent` | `opencode/` (any version from 1.18.0 upwards) | +| `x-opencode-session` | `ses_` + 12 lowercase hex characters + 14 base62 characters = 30 characters in total | +| `stream` | `true` | +| `tools` | a function named exactly `read` | + +A `You are opencode` system message is *not* required — the request is accepted without it. + +Two details are easy to get wrong: + +- The session id is validated by **shape**, not by value. The 12-character hex block is a 48-bit + millisecond timestamp, and only the format is checked. A 32-character hex id, or an id of 28, 29 + or 31 characters, is refused. +- The `read` tool must be named exactly `read`. A differently named function does not satisfy the + gate. + +modelrelay handles the shape for you: it rewrites the request, converts the upstream's stream back +into a single JSON body for callers that asked for `stream: false`, and removes calls to the +injected `read` tool from the answer, so a caller that never declared that tool does not see an +unknown tool call in its response. + +### No API key is required for the free models + +The key is not what makes them work — the request shape is. Measured against the live endpoint: + +| `Authorization` header | Result | +|---|---| +| absent | accepted | +| `Bearer public` | accepted | +| `Bearer ` (empty) | accepted | +| a key that is not a real OpenCode key | `500` "Invalid credential" | + +A missing key is therefore harmless, but a *wrong* key is not: it fails with a server error +rather than being ignored. modelrelay sends no `Authorization` header when no OpenCode key is +configured, which is the correct behavior for the free tier. A key is still required for the paid +Zen models. + ## Model ids go stale A free variant can disappear while its slug still lists elsewhere. For example, OpenRouter answers `404` ("this model is unavailable for free") for a `:free` slug that has become paid, and names the paid slug. A model list is not proof that the model is usable, so check with a real request. diff --git a/lib/config.js b/lib/config.js index 432bf56..55ba9eb 100644 --- a/lib/config.js +++ b/lib/config.js @@ -113,6 +113,10 @@ const PROVIDER_MODEL_ID_ENV_VARS = { ollama: 'OLLAMA_MODEL', } +// The User-Agent the official OpenCode client presents. Overridable, see +// getOpencodeFreeTierUserAgent. +export const DEFAULT_OPENCODE_USER_AGENT = 'opencode/1.19.3'; + export function normalizeSecret(value) { return typeof value === 'string' ? trimEndWhile(value, ch => isWhitespaceChar(ch) || (ch >= '\u2580' && ch <= '\u259F')).trim() @@ -437,6 +441,67 @@ export function getPinningMode(config) { return config?.pinningMode === 'exact' ? 'exact' : 'canonical' } +// 📖 OpenCode Zen free-tier compatibility. +// +// OpenCode's free models are gated by the *request shape* they expect from +// the official client: a matching User-Agent, a session id in a specific +// format, `stream: true` and a tool named exactly `read`. ModelRelay does not +// imitate that client by default — the free tier is documented as a constraint +// we respect. +// +// A user who explicitly accepts that can opt in, at their own responsibility. +// The env var takes precedence over the config file, matching the other +// settings in this module. +const OPENCODE_FREE_COMPAT_ENV_VAR = 'MODELRELAY_OPENCODE_FREE_COMPAT'; + +// 📖 Only these spellings enable the flag. Anything else (including an empty +// string, '0', 'false' and 'no') leaves it off, so a typo cannot silently +// enable request rewriting. +const OPENCODE_FREE_COMPAT_ENABLED_VALUES = new Set(['1', 'true', 'yes', 'on']); + +function parseOpencodeFreeCompatValue(value) { + if (value === true) return true; + if (typeof value !== 'string') return false; + return OPENCODE_FREE_COMPAT_ENABLED_VALUES.has(value.trim().toLowerCase()); +} + +/** + * 📖 isOpencodeFreeTierCompatEnabled: Is the OpenCode Zen free-tier + * compatibility mode on? + * + * Off by default. The env var wins over the config file. + * + * @param {object} config + * @returns {boolean} + */ +export function isOpencodeFreeTierCompatEnabled(config) { + const envValue = process.env[OPENCODE_FREE_COMPAT_ENV_VAR]; + if (envValue !== undefined && envValue !== '') { + return parseOpencodeFreeCompatValue(envValue); + } + return parseOpencodeFreeCompatValue(config?.opencodeFreeTierCompat); +} + +/** + * 📖 getOpencodeFreeTierUserAgent: The User-Agent to present to OpenCode Zen + * when the compatibility mode is on. + * + * Overridable so a stale default can be fixed without a release: set + * `opencodeFreeTierUserAgent` in the config file or + * `MODELRELAY_OPENCODE_FREE_TIER_USER_AGENT` in the environment. + * + * @param {object} config + * @returns {string} + */ +export function getOpencodeFreeTierUserAgent(config) { + const envValue = process.env.MODELRELAY_OPENCODE_FREE_TIER_USER_AGENT; + const fromEnv = normalizeText(envValue); + if (fromEnv) return fromEnv; + const fromConfig = normalizeText(config?.opencodeFreeTierUserAgent); + if (fromConfig) return fromConfig; + return DEFAULT_OPENCODE_USER_AGENT; +} + // 📖 Internal helper: create a blank config with the right shape function _emptyConfig() { return { @@ -499,6 +564,11 @@ export function normalizeConfigShape(config) { if (!Array.isArray(base.excludedProviders)) base.excludedProviders = [] base.pinningMode = base.pinningMode === 'exact' ? 'exact' : 'canonical' + // The OpenCode free-tier compatibility flag is a boolean. A string value is + // accepted only when it spells out an enabled state; anything else is off. + base.opencodeFreeTierCompat = parseOpencodeFreeCompatValue(base.opencodeFreeTierCompat) + if (typeof base.opencodeFreeTierUserAgent !== 'string') base.opencodeFreeTierUserAgent = '' + else base.opencodeFreeTierUserAgent = base.opencodeFreeTierUserAgent.trim() if (!base.modelTags || typeof base.modelTags !== 'object' || Array.isArray(base.modelTags)) { base.modelTags = {} } else { diff --git a/lib/server.js b/lib/server.js index 7d5b172..353fd2f 100644 --- a/lib/server.js +++ b/lib/server.js @@ -7,14 +7,14 @@ import { homedir } from 'os'; import { MODELS, sources, canonicalizeModelId, getPreferredModelContext, getPreferredModelLabel, getScore, resolveAliasedModelId } from '../sources.js'; import { getModelTags as getBuiltInModelTags } from '../tags.js'; import { API_KEY_SIGNUP_URLS } from './providerLinks.js'; -import { getApiKey, getApiKeyPool, getMaxTurns, getPinningMode, getProviderBaseUrl, getProviderModelId, hasMultipleKeys, isProviderEnabled, getProviderPingIntervalMs, isAutoPingEnabled, loadConfig, saveConfig, exportConfigToken, importConfigToken, isOpenAICompatibleInstanceKey, getBaseProviderKey, listOpenAICompatibleEndpoints, upsertOpenAICompatibleEndpoint, removeOpenAICompatibleEndpoint, buildOpenAICompatibleInstanceKey, getOpenAICompatibleInstanceId } from './config.js'; +import { getApiKey, getApiKeyPool, getMaxTurns, getPinningMode, getProviderBaseUrl, getProviderModelId, hasMultipleKeys, isProviderEnabled, getProviderPingIntervalMs, isAutoPingEnabled, loadConfig, saveConfig, exportConfigToken, importConfigToken, isOpenAICompatibleInstanceKey, getBaseProviderKey, listOpenAICompatibleEndpoints, upsertOpenAICompatibleEndpoint, removeOpenAICompatibleEndpoint, buildOpenAICompatibleInstanceKey, getOpenAICompatibleInstanceId, isOpencodeFreeTierCompatEnabled, getOpencodeFreeTierUserAgent, DEFAULT_OPENCODE_USER_AGENT } from './config.js'; import { buildModelGroups, computeFailedRefreshRetryAt, computeQoSMap, DEFAULT_QOS_LATENCY_TARGET_MS, findBestModel, getAvg, getEffectiveContext, getUptime, normalizeRequestedModel, getVerdict, isCompleteSseStream, isProviderRequestRejectionError, isRateLimitShapedError, isRetryableProxyStatus, peekFirstSseEvent, rankModelsForRouting, getRoutingModelKey, parseOpenRouterKeyRateLimit, filterModelsByRequested, selectNextApiKeyFromPool } from './utils.js'; import { getPreferredLanIpv4Address } from './network.js'; import { capLength, isSafeObjectKey } from './text.js'; import { rateLimit } from 'express-rate-limit'; import { createAdminAuth, createAdminMiddleware, createAuthRoutes, parseAllowedHosts, parseAllowedOrigins } from './admin-guard.js'; import { classifyFailureStatus, createTelemetryStore, normalizeTelemetryStore, recordFailure, recordSuccess, reliabilityMultiplier, summarizeTelemetry, summarizeTelemetryEntry } from './telemetry.js'; -import { createHash, randomUUID } from 'crypto'; +import { createHash, randomInt, randomUUID } from 'crypto'; import { getAutostartStatus } from './autostart.js'; import { buildWindowsPostUpdateRestartCommand, fetchLatestNpmVersion, isRunningFromSource, isVersionNewer, runUpdateCommand } from './update.js'; import { DEFAULT_MODEL_QUALITY, MODEL_QUALITY_CACHE_MS, fetchOpenRouterQualityIndex, resolveModelQuality } from './model-quality.js'; @@ -59,6 +59,17 @@ const DISCOVERABLE_PROVIDER_MODELS_REFRESH_MS = 30 * 60_000; const FAILED_DISCOVERY_RETRY_MS = 2 * 60_000; const OPTIONAL_BEARER_AUTH_PROVIDERS = new Set([KILOCODE_PROVIDER_KEY, OPENCODE_PROVIDER_KEY]); const OPENCODE_CLIENT_HEADER = 'cli'; + +// The warning shown when the OpenCode Zen free-tier compatibility mode is on. +// It is shown in the startup log and in the dashboard, because the mode +// imitates the official OpenCode client, which may conflict with OpenCode's +// terms, and it can stop working at any time. +export const OPENCODE_FREE_COMPAT_WARNING = [ + 'OpenCode Zen free-tier compatibility is ENABLED.', + 'This makes modelrelay imitate the official OpenCode client (User-Agent, session id, stream, and a read tool) so the free models answer.', + 'That may violate OpenCode\'s terms, and it can stop working at any time.', + 'You are responsible for using it. Disable it by unsetting opencodeFreeTierCompat / MODELRELAY_OPENCODE_FREE_COMPAT.', +].join(' '); // Kiro expects AWS SDK-style user-agent headers (mirrors OmniRoute's Kiro header profile). const KIRO_SDK_USER_AGENT = 'AWS-SDK-JS/3.0.0 kiro-ide/1.0.0'; const KIRO_AMZ_USER_AGENT = 'aws-sdk-js/3.0.0 kiro-ide/1.0.0'; @@ -97,6 +108,26 @@ const OPENCODE_CHAT_COMPLETIONS_MODELS = new Map([ ['nemotron-3-super-free', { label: 'Nemotron 3 Super Free', scoreId: 'nvidia/nemotron-3-super-120b-a12b' }], ]); +/** + * 📖 isOpencodeFreeModelUnsupported: Is this model one that cannot be used + * from here unless the OpenCode free-tier compatibility mode is on? + * + * The OpenCode Zen free models only answer a request that carries the official + * client's shape. Without the compatibility mode they are unusable, so they are + * reported as `unsupported` rather than `down`: a `down` entry would look like + * a broken model and would skew the telemetry. + * + * @param {string} providerKey + * @param {string} modelId + * @param {object} config + * @returns {boolean} + */ +export function isOpencodeFreeModelUnsupported(providerKey, modelId, config) { + if (providerKey !== OPENCODE_PROVIDER_KEY) return false; + if (!isOpenCodeFreeModelId(modelId)) return false; + return !isOpencodeFreeTierCompatEnabled(config); +} + function isOpenCodeFreeModelId(modelId) { if (!modelId) return false; if (OPENCODE_CHAT_COMPLETIONS_MODELS.has(modelId)) return true; @@ -967,6 +998,31 @@ function makeOpencodeHeaderId(prefix) { return `${prefix}_${randomUUID().replace(/-/g, '')}`; } +// The OpenCode Zen free tier validates the session id's *shape*, not its value: +// `ses_` + 12 lowercase hex characters + 14 base62 characters = 30 characters +// in total. A 32-hex uuid (makeOpencodeHeaderId) is rejected with a +// FreeTierError, as are 28, 29, 31 and 40 character ids. +// +// The hex block is a 48-bit millisecond timestamp, which is what the official +// client mints. The value is not validated, but a current timestamp keeps the +// id from going stale. +const OPENCODE_SESSION_HEX_LENGTH = 12; +const OPENCODE_SESSION_TAIL_LENGTH = 14; +const OPENCODE_SESSION_BASE62 = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'; + +function randomBase62(length) { + let out = ''; + for (let i = 0; i < length; i++) { + out += OPENCODE_SESSION_BASE62[randomInt(OPENCODE_SESSION_BASE62.length)]; + } + return out; +} + +export function makeOpencodeSessionId(now = Date.now()) { + const hex = (BigInt(now) & ((1n << 48n) - 1n)).toString(16).padStart(OPENCODE_SESSION_HEX_LENGTH, '0'); + return `ses_${hex}${randomBase62(OPENCODE_SESSION_TAIL_LENGTH)}`; +} + export function buildOpencodeHeaders(options = {}) { return { 'x-opencode-project': buildOpencodeProjectId(options.projectSeed), @@ -976,6 +1032,30 @@ export function buildOpencodeHeaders(options = {}) { }; } +/** + * 📖 buildOpencodeCompatOptions: The request options that the OpenCode Zen + * free-tier compatibility mode adds. + * + * They are passed in explicitly so the default path stays exactly as it was: + * without the mode the session id keeps its old format and no User-Agent is + * sent. + * + * @param {string} providerKey + * @param {object} config + * @returns {object} options for buildProviderRequestHeaders / buildProviderRequestBody + */ +export function buildOpencodeCompatOptions(providerKey, config) { + const freeTierCompat = providerKey === OPENCODE_PROVIDER_KEY + && isOpencodeFreeTierCompatEnabled(config); + return { + freeTierCompat, + userAgent: getOpencodeFreeTierUserAgent(config), + // The OpenCode free tier validates the session id's format, so the + // compatible format is passed in explicitly when the mode is on. + sessionId: freeTierCompat ? makeOpencodeSessionId() : undefined, + }; +} + export function buildProviderRequestHeaders(providerKey, options = {}) { const headers = { 'Content-Type': 'application/json', @@ -987,6 +1067,11 @@ export function buildProviderRequestHeaders(providerKey, options = {}) { if (providerKey === OPENCODE_PROVIDER_KEY) { Object.assign(headers, buildOpencodeHeaders(options)); + // The OpenCode free tier also checks the User-Agent. It is only sent when + // the compatibility mode is on, so the default behaviour is unchanged. + if (options.freeTierCompat) { + headers['User-Agent'] = options.userAgent || DEFAULT_OPENCODE_USER_AGENT; + } } else if (providerKey === KIRO_PROVIDER_KEY) { headers.Accept = KIRO_STREAM_ACCEPT; headers['X-Amz-Target'] = KIRO_STREAMING_TARGET; @@ -1320,10 +1405,289 @@ export function buildKiroRequestPayload(body, modelId, options = {}) { return payload; } +// The OpenCode Zen free tier only accepts requests that carry the shape the +// official OpenCode client sends. Three body-level conditions are required: +// +// * `stream: true` -- a non-streaming request is refused. +// * a tool named exactly `read` -- a request without it is refused too. +// * (the User-Agent and the session id format are header-level conditions) +// +// A `You are opencode` system message is *not* required, so it is only added +// when the caller did not supply a system message of its own. +// +// `read` is injected only to satisfy the gate. A model may answer with a call +// to it, and the caller never declared that tool, so those calls are stripped +// from the response again (see stripInjectedToolCalls). +export const OPENCODE_INJECTED_READ_TOOL = 'read'; + +const OPENCODE_READ_TOOL = { + type: 'function', + function: { + name: OPENCODE_INJECTED_READ_TOOL, + description: 'Read a file from the local filesystem.', + parameters: { + type: 'object', + properties: { + filePath: { type: 'string', description: 'The path of the file to read.' }, + offset: { type: 'number', description: 'Optional line offset to start reading from.' }, + limit: { type: 'number', description: 'Optional maximum number of lines to read.' }, + }, + required: ['filePath'], + }, + }, +}; + +function toolNameOf(tool) { + return tool?.function?.name ?? tool?.name ?? null; +} + +/** + * 📖 buildOpencodeFreeTierPayload: Rewrite a request body into the shape the + * OpenCode Zen free tier accepts. + * + * @param {object} body — the caller's request body + * @param {object} options + * @param {boolean} options.freeTierCompat — must be true for this to apply + * @param {boolean} options.callerDeclaredRead — the caller declared a `read` + * tool, so injected calls to it must not be stripped + * @returns {object} a new body; the input is not mutated + */ +export function buildOpencodeFreeTierPayload(body, options = {}) { + const payload = { ...(body || {}) }; + + // The upstream only answers a streaming request, so a non-streaming caller + // would receive an SSE body it cannot parse. The proxy converts the response + // back to JSON for those callers (see collectOpencodeStreamAsJson), so the + // request itself always streams. + payload.stream = true; + + const tools = Array.isArray(payload.tools) ? payload.tools : []; + const callerDeclaredRead = options.callerDeclaredRead ?? tools.some((tool) => toolNameOf(tool) === OPENCODE_INJECTED_READ_TOOL); + if (!callerDeclaredRead) { + payload.tools = [...tools, OPENCODE_READ_TOOL]; + } + + const messages = Array.isArray(payload.messages) ? payload.messages : []; + const hasSystemMessage = messages.some((message) => message?.role === 'system'); + if (!hasSystemMessage && options.addSystemPrompt !== false) { + payload.messages = [{ role: 'system', content: 'You are opencode' }, ...messages]; + } + + return payload; +} + +/** + * 📖 stripInjectedToolCalls: Remove tool calls to the injected `read` tool from + * a response, so a caller that never declared it does not see an unknown tool + * call in its answer. + * + * @param {object} data — a parsed OpenAI-style response body + * @param {boolean} callerDeclaredRead — when true, nothing is stripped + * @returns {object} the same object, mutated in place + */ +export function stripInjectedToolCalls(data, callerDeclaredRead) { + if (callerDeclaredRead || !data || typeof data !== 'object') return data; + const choices = Array.isArray(data.choices) ? data.choices : []; + for (const choice of choices) { + const message = choice?.message; + if (message && Array.isArray(message.tool_calls) && message.tool_calls.length > 0) { + const kept = message.tool_calls.filter((tc) => toolNameOf(tc) !== OPENCODE_INJECTED_READ_TOOL); + if (kept.length === 0) { + delete message.tool_calls; + } else { + message.tool_calls = kept; + } + } + if (message && (!message.tool_calls || message.tool_calls.length === 0)) { + if (choice && choice.finish_reason === 'tool_calls') { + choice.finish_reason = 'stop'; + } + } + } + return data; +} + +/** + * 📖 createInjectedReadSseStripper: Stateful, line-buffered filter that removes + * the injected `read` tool calls from an SSE stream, so a caller that never + * declared the tool does not see an unknown tool call in its answer. + * + * `write(text)` takes the next piece of the stream and returns the rewritten + * text for every complete line; a trailing partial line is held back until the + * next piece or `end()`. The filter remembers which tool call indices belong to + * `read` (later deltas for those indices are dropped), and rewrites + * `finish_reason: "tool_calls"` to `"stop"` only when no other tool call was + * seen in the whole stream. Chunks that carry content, a finish reason or + * usage are kept even when their tool calls were removed. + * + * @returns {{ write: (text: string) => string, end: () => string }} + */ +export function createInjectedReadSseStripper() { + let buffer = ''; + const readIndices = new Set(); + let sawOtherToolCall = false; + + function rewriteLine(line) { + const trimmed = line.trim(); + if (!trimmed.startsWith('data: ') || trimmed === 'data: [DONE]') return line; + let data; + try { data = JSON.parse(trimmed.slice(6)); } catch { return line; } + const choices = Array.isArray(data.choices) ? data.choices : []; + let changed = false; + for (const choice of choices) { + const delta = choice?.delta; + if (!delta || !Array.isArray(delta.tool_calls)) continue; + const kept = []; + for (const tc of delta.tool_calls) { + const index = tc.index ?? 0; + if (toolNameOf(tc) === OPENCODE_INJECTED_READ_TOOL) readIndices.add(index); + if (readIndices.has(index)) continue; + sawOtherToolCall = true; + kept.push(tc); + } + if (kept.length !== delta.tool_calls.length) { + changed = true; + if (kept.length === 0) delete delta.tool_calls; + else delta.tool_calls = kept; + } + } + for (const choice of choices) { + if (choice && choice.finish_reason === 'tool_calls' && !sawOtherToolCall) { + choice.finish_reason = 'stop'; + changed = true; + } + } + if (!changed) return line; + const keep = data.usage + || choices.some((c) => c?.delta?.content || c?.finish_reason); + return keep ? 'data: ' + JSON.stringify(data) : null; + } + + return { + write(text) { + buffer += String(text ?? ''); + const lines = buffer.split('\n'); + buffer = lines.pop(); + let out = ''; + for (const line of lines) { + const rewritten = rewriteLine(line); + if (rewritten !== null) out += rewritten + '\n'; + } + return out; + }, + end() { + const rest = buffer; + buffer = ''; + if (!rest) return ''; + const rewritten = rewriteLine(rest); + return rewritten === null ? '' : rewritten; + }, + }; +} + +/** + * 📖 stripInjectedToolCallsFromSseChunk: Run a whole piece of SSE text through + * a fresh stripper. Convenience for tests and callers that hold the full text. + * + * @param {string} chunk — raw SSE text + * @returns {string} the rewritten text + */ +export function stripInjectedToolCallsFromSseChunk(chunk) { + return stripInjectedToolCallsFromSseStream([chunk]); +} + +/** + * 📖 stripInjectedToolCallsFromSseStream: Run an array of SSE pieces through one + * stripper, carrying partial lines and state across the pieces. + * + * @param {string[]} chunks + * @returns {string} + */ +export function stripInjectedToolCallsFromSseStream(chunks) { + const stripper = createInjectedReadSseStripper(); + let out = ''; + for (const chunk of chunks) out += stripper.write(chunk); + return out + stripper.end(); +} + +/** + * 📖 collectOpencodeStreamAsJson: Turn an OpenAI SSE stream into a single + * JSON response body, for callers that asked for `stream: false`. + * + * The OpenCode free tier only answers a streaming request, so a non-streaming + * caller would otherwise receive an SSE body it cannot parse. + * + * @param {string} sseText — the raw SSE text + * @param {string} modelId — the model to report + * @returns {object} an OpenAI-style chat completion object + */ +export function collectOpencodeStreamAsJson(sseText, modelId) { + let content = ''; + let toolCalls = []; + let finishReason = null; + let usage = null; + let resolvedModel = modelId; + + for (const line of String(sseText || '').split('\n')) { + const trimmed = line.trim(); + if (!trimmed.startsWith('data: ') || trimmed === 'data: [DONE]') continue; + let chunk; + try { + chunk = JSON.parse(trimmed.slice(6)); + } catch { + continue; + } + if (chunk.model) resolvedModel = chunk.model; + if (chunk.usage) usage = chunk.usage; + const choice = (chunk.choices || [])[0]; // reads choices[0] only + if (!choice) continue; + if (choice.finish_reason) finishReason = choice.finish_reason; + const delta = choice.delta || {}; + if (delta.content) content += delta.content; + if (Array.isArray(delta.tool_calls)) { + for (const tc of delta.tool_calls) { + const index = tc.index ?? 0; + if (!toolCalls[index]) { + toolCalls[index] = { id: tc.id || '', type: tc.type || 'function', function: { name: '', arguments: '' } }; + } + if (tc.id) toolCalls[index].id = tc.id; + if (tc.type) toolCalls[index].type = tc.type; + if (tc.function) { + if (tc.function.name) toolCalls[index].function.name += tc.function.name; + if (tc.function.arguments) toolCalls[index].function.arguments += tc.function.arguments; + } + } + } + } + + const message = { role: 'assistant', content: content || null }; + const calls = toolCalls.filter(Boolean); + if (calls.length > 0) { + message.tool_calls = calls; + if (!finishReason) finishReason = 'tool_calls'; + } + + return { + id: `chatcmpl-${resolvedModel}`, + object: 'chat.completion', + created: Math.floor(Date.now() / 1000), + model: resolvedModel, + choices: [{ + index: 0, + message, + finish_reason: finishReason || 'stop', + }], + ...(usage ? { usage } : {}), + }; +} + export function buildProviderRequestBody(providerKey, body, modelId, options = {}) { if (providerKey === KIRO_PROVIDER_KEY) { return buildKiroRequestPayload(body, modelId || body?.model, options); } + if (providerKey === OPENCODE_PROVIDER_KEY && options.freeTierCompat) { + return buildOpencodeFreeTierPayload(body, options); + } return body; } @@ -1744,17 +2108,37 @@ export async function transformKiroResponse(response, modelId, stream = false) { }); } +/** + * 📖 buildPingRequest: The headers and body a health check sends. + * + * The ping has to use the same request shape as a real request. For the + * OpenCode Zen free models that means the free-tier shape, or the health check + * reports them as `down` with a 403 even though the proxy can route them. + * + * @param {string} providerKey + * @param {string} modelId + * @param {string} apiKey + * @returns {{ headers: object, payload: object }} + */ +export function buildPingRequest(providerKey, modelId, apiKey, configOverride) { + const config = configOverride || loadConfig(); + const compat = buildOpencodeCompatOptions(providerKey, config); + const freeTierCompat = compat.freeTierCompat; + const headers = buildProviderRequestHeaders(providerKey, { apiKey, ...compat }); + const payload = buildProviderRequestBody(providerKey, { + model: modelId, + messages: [{ role: 'user', content: 'hi' }], + max_tokens: 1, + }, modelId, { freeTierCompat }); + return { headers, payload }; +} + async function ping(apiKey, modelId, url, providerKey = null) { const ctrl = new AbortController() const timer = setTimeout(() => ctrl.abort(), PING_TIMEOUT) const t0 = performance.now() try { - const headers = buildProviderRequestHeaders(providerKey, { apiKey }) - const payload = buildProviderRequestBody(providerKey, { - model: modelId, - messages: [{ role: 'user', content: 'hi' }], - max_tokens: 1, - }, modelId) + const { headers, payload } = buildPingRequest(providerKey, modelId, apiKey) const resp = await fetch(url, { method: 'POST', signal: ctrl.signal, headers, @@ -2427,6 +2811,10 @@ export async function runServer(config, port, enableLog = true, bannedModels = [ } console.log(chalk.cyan(` 🚀 Starting modelrelay Web UI on port ${port}...`)); + if (isOpencodeFreeTierCompatEnabled(loadConfig())) { + console.log(chalk.yellow(` ⚠️ ${OPENCODE_FREE_COMPAT_WARNING}`)); + } + if (bannedModels.length > 0) { console.log(chalk.yellow(` 🚫 Banned models: ${bannedModels.join(', ')}`)); } @@ -2889,6 +3277,17 @@ export async function runServer(config, port, enableLog = true, bannedModels = [ return; } + // The OpenCode Zen free models only answer a request that carries the + // official client's shape. Without the compatibility mode they are not + // usable here, so they are marked unsupported rather than pinged: a `down` + // entry would look like a broken model and would skew the telemetry. + if (isOpencodeFreeModelUnsupported(r.providerKey, r.modelId, currentConfig)) { + r.status = 'unsupported'; + r.httpCode = null; + r.lastError = null; + return; + } + if (typeof minSweScore === 'number' && typeof r.intell === 'number' && r.intell < minSweScore) { r.status = 'excluded'; return; @@ -3520,6 +3919,15 @@ export async function runServer(config, port, enableLog = true, bannedModels = [ res.json(providers); }); + app.get('/api/opencode-free-compat', (req, res) => { + const currentConfig = loadConfig(); + res.json({ + enabled: isOpencodeFreeTierCompatEnabled(currentConfig), + userAgent: getOpencodeFreeTierUserAgent(currentConfig), + warning: OPENCODE_FREE_COMPAT_WARNING, + }); + }); + app.post('/api/openai-compatible/endpoints', (req, res) => { const { id, name, baseUrl, modelId, apiKey, enabled, discoverModels } = req.body || {}; if (!name && !id) return res.status(400).json({ error: 'name or id is required.' }); @@ -4262,6 +4670,11 @@ export async function runServer(config, port, enableLog = true, bannedModels = [ // path at the end of the response doesn't count it a second time. let telemetryOutcomeRecorded = false; + // The OpenCode Zen free-tier compatibility mode. Resolved per request so a + // config change takes effect without a restart. + let opencodeCompatOptions = {}; + let opencodeFreeCompat = false; + let callerDeclaredRead = false; for (let retry = 0; retry <= MAX_PROACTIVE_RETRIES; retry++) { const best = pickNextModel(); if (!best) break; @@ -4303,15 +4716,23 @@ export async function runServer(config, port, enableLog = true, bannedModels = [ console.log(chalk.dim(` [Router] ➡️ Proxying request (attempt ${retry + 1}/${MAX_PROACTIVE_RETRIES + 1}) to ${best.providerKey}/${best.modelId} (${best.status === 'up' && best.pings.length > 0 ? best.pings[best.pings.length - 1].ms + 'ms' : 'fallback'})`)); + opencodeCompatOptions = buildOpencodeCompatOptions(best.providerKey, currentConfig); + opencodeFreeCompat = opencodeCompatOptions.freeTierCompat; + callerDeclaredRead = Array.isArray(payload.tools) + && payload.tools.some((tool) => (tool?.function?.name ?? tool?.name) === OPENCODE_INJECTED_READ_TOOL); let headers = buildProviderRequestHeaders(best.providerKey, { apiKey: providerAuth.token, - sessionId: makeOpencodeHeaderId('ses'), requestId: makeOpencodeHeaderId('req'), + ...opencodeCompatOptions, }); const kiroOptions = best.providerKey === KIRO_PROVIDER_KEY ? { profileArn: currentConfig?.providers?.kiro?.profileArn || null } : {}; - let providerPayload = buildProviderRequestBody(best.providerKey, payload, best.modelId, kiroOptions); + let providerPayload = buildProviderRequestBody(best.providerKey, payload, best.modelId, { + ...kiroOptions, + freeTierCompat: opencodeFreeCompat, + callerDeclaredRead, + }); const t0 = performance.now(); let response; @@ -4327,11 +4748,15 @@ export async function runServer(config, port, enableLog = true, bannedModels = [ if (fallbackToken) { providerAuth = { token: fallbackToken, authSource: 'api-key', providerUrlOverride: providerAuth.providerUrlOverride }; headers = buildProviderRequestHeaders(best.providerKey, { - apiKey: fallbackToken, - sessionId: makeOpencodeHeaderId('ses'), + apiKey: providerAuth.token, requestId: makeOpencodeHeaderId('req'), + ...opencodeCompatOptions, + }); + providerPayload = buildProviderRequestBody(best.providerKey, payload, best.modelId, { + ...kiroOptions, + freeTierCompat: opencodeFreeCompat, + callerDeclaredRead, }); - providerPayload = buildProviderRequestBody(best.providerKey, payload, best.modelId, kiroOptions); response = await fetch(providerUrl, { method: 'POST', headers, @@ -4471,12 +4896,45 @@ export async function runServer(config, port, enableLog = true, bannedModels = [ } } + // The OpenCode free tier only answers a streaming request, so a + // non-streaming caller would receive an SSE body it cannot parse. The + // compatibility mode converts the stream back into a single JSON + // response for those callers. + if (opencodeFreeCompat && !payload.stream && selectedResponse.body && selectedResponse.status < 400) { + const sseText = await selectedResponse.text(); + const jsonBody = collectOpencodeStreamAsJson(sseText, selectedModel.modelId); + stripInjectedToolCalls(jsonBody, callerDeclaredRead); + const jsonText = JSON.stringify(jsonBody); + // The response is read in one go, so the time to first token is the + // whole duration. + logEntry.ttft = Math.round(performance.now() - selectedT0); + logEntry.response = jsonBody.choices?.[0]?.message?.content || ''; + if (jsonBody.choices?.[0]?.message?.tool_calls) { + logEntry.tool_calls = jsonBody.choices[0].message.tool_calls; + } + res.setHeader('content-type', 'application/json'); + res.end(jsonText); + noteTelemetrySuccess(selectedModel, { + ttftMs: logEntry.ttft, + durationMs: Math.round(performance.now() - selectedT0), + completionTokens: jsonBody.usage?.completion_tokens, + }); + telemetryOutcomeRecorded = true; + saveLogs(); + return; + } + + if (selectedResponse.body) { const { Readable, Transform } = await import('stream'); let responseBodyText = ''; let ttftCaptured = false; const MAX_LOG_BODY_SIZE = 10 * 1024 * 1024; // 10MB limit for logging + // state for the line-buffered SSE transform + const injectedReadStripper = opencodeFreeCompat && !callerDeclaredRead + ? createInjectedReadSseStripper() + : null; const captureStream = new Transform({ transform(chunk, encoding, callback) { @@ -4484,15 +4942,28 @@ export async function runServer(config, port, enableLog = true, bannedModels = [ ttftCaptured = true; logEntry.ttft = Math.round(performance.now() - selectedT0); } + const text = chunk.toString(); // Only accumulate up to limit to prevent OOM if (responseBodyText.length < MAX_LOG_BODY_SIZE) { - responseBodyText += chunk.toString(); + responseBodyText += text; + } + // The injected `read` tool is an implementation detail of the free + // tier gate. A model may still answer with a call to it, and the + // caller never declared that tool, so those calls are removed from + // the stream before the client sees them. + if (injectedReadStripper) { + callback(null, injectedReadStripper.write(text)); + return; } callback(null, chunk); }, flush(callback) { + // Release a partial trailing line the stripper was holding back. + if (injectedReadStripper) { + const tail = injectedReadStripper.end(); + if (tail) this.push(tail); + } try { - const wasTruncated = responseBodyText.length >= MAX_LOG_BODY_SIZE; if (selectedResponse.status >= 400) { try { diff --git a/lib/utils.js b/lib/utils.js index 9eb942c..9c7f820 100644 --- a/lib/utils.js +++ b/lib/utils.js @@ -184,7 +184,10 @@ export function computeQoS(r, latencyTargetMs) { } export function isModelEligibleForRouting(r) { - return r.status !== 'banned' && r.status !== 'disabled' && r.status !== 'excluded' + // 'unsupported' means the model exists but cannot be used from here (e.g. the + // OpenCode Zen free models without the free-tier compatibility mode). It is + // not a failure, so it must not be routed to. + return r.status !== 'banned' && r.status !== 'disabled' && r.status !== 'excluded' && r.status !== 'unsupported' } export function getRoutingModelKey(r) { diff --git a/package-lock.json b/package-lock.json index 4f58f62..1abf3db 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@schaetzkc/modelrelay", - "version": "1.28.1", + "version": "1.29.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@schaetzkc/modelrelay", - "version": "1.28.1", + "version": "1.29.0", "license": "MIT", "dependencies": { "chalk": "^5.4.1", diff --git a/package.json b/package.json index 5d091d9..8d06025 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@schaetzkc/modelrelay", - "version": "1.28.1", + "version": "1.29.0", "description": "OpenAI-compatible local router that benchmarks free coding models across providers and forwards requests to the best available model.", "keywords": [ "nvidia", diff --git a/public/index.html b/public/index.html index e349dd2..32c9c83 100644 --- a/public/index.html +++ b/public/index.html @@ -1792,6 +1792,7 @@ + @@ -2003,6 +2004,8 @@

Dashboard

Disabled + @@ -2428,9 +2431,30 @@

Sign in

warning.style.display = window.location.protocol === 'file:' ? 'block' : 'none'; } + // The OpenCode Zen free-tier compatibility mode imitates the official + // OpenCode client, which may conflict with OpenCode's terms and can stop + // working at any time. The warning is shown while the mode is on. + async function updateOpencodeFreeCompatWarning() { + const banner = document.getElementById('opencode-free-compat-warning'); + if (!banner) return; + try { + const res = await fetch('/api/opencode-free-compat'); + if (!res.ok) { banner.style.display = 'none'; return; } + const info = await res.json(); + if (!info || !info.enabled) { banner.style.display = 'none'; return; } + banner.innerHTML = '⚠️ OpenCode Zen free-tier compatibility is enabled. ' + escapeHtml(info.warning || '') + + ' User-Agent: ' + escapeHtml(info.userAgent || '') + ''; + banner.style.display = 'block'; + } catch (e) { + banner.style.display = 'none'; + } + } + + // Run on DOMContentLoaded to ensure button exists document.addEventListener('DOMContentLoaded', updateThemeIcon); document.addEventListener('DOMContentLoaded', updateLocalFileWarning); + document.addEventListener('DOMContentLoaded', updateOpencodeFreeCompatWarning); let allModels = []; let searchTerm = ''; @@ -3368,8 +3392,9 @@

Sign in

const modelCell = row.cells[1]; const isBannedRow = m.status === 'banned'; const isExcludedRow = m.status === 'excluded'; - row.style.opacity = isExcludedRow || isBannedRow ? '0.5' : '1'; - modelCell.querySelector('div').firstChild.textContent = (isBannedRow ? '🚫 ' : (isExcludedRow ? '⛔ ' : '')) + m.label; + const isUnsupportedRow = m.status === 'unsupported'; + row.style.opacity = isExcludedRow || isBannedRow || isUnsupportedRow ? '0.5' : '1'; + modelCell.querySelector('div').firstChild.textContent = (isBannedRow ? '🚫 ' : (isExcludedRow ? '⛔ ' : (isUnsupportedRow ? '🚷 ' : ''))) + m.label; const tagRow = modelCell.querySelector('.model-tag-row'); if (tagRow) tagRow.innerHTML = renderModelTagChips(m.tags, false, true); const inlineMetrics = modelCell.querySelector('.model-inline-metrics'); diff --git a/test/fixtures/split1.txt b/test/fixtures/split1.txt new file mode 100644 index 0000000..8447242 --- /dev/null +++ b/test/fixtures/split1.txt @@ -0,0 +1 @@ +data: {"choices":[{"delta":{"tool_calls":[{"index":0,"id":"c1","function":{"name":"re \ No newline at end of file diff --git a/test/fixtures/split2.txt b/test/fixtures/split2.txt new file mode 100644 index 0000000..27edddd --- /dev/null +++ b/test/fixtures/split2.txt @@ -0,0 +1,2 @@ +ad","arguments":"{}"}}]}}]} + diff --git a/test/test.js b/test/test.js index 94dd27f..5c47e26 100644 --- a/test/test.js +++ b/test/test.js @@ -35,7 +35,8 @@ import { parseOpenRouterKeyRateLimit, selectNextApiKeyFromPool, VERDICT_ORDER, -} from '../lib/utils.js' + isModelEligibleForRouting, +} from '../lib/utils.js'; import { classifyFailureStatus, createTelemetryStore, getReliability, normalizeTelemetryStore, recordFailure, recordSuccess, reliabilityMultiplier, summarizeTelemetry, summarizeTelemetryEntry, TELEMETRY_HALF_LIFE_MS, TELEMETRY_MIN_MULTIPLIER } from '../lib/telemetry.js' import http from 'node:http' import express from 'express' @@ -47,9 +48,9 @@ import { normalizeMissingScoreId } from '../lib/score-fetcher.js' import { buildOpenRouterQualityIndex, fitLinearRegression, qualityLookupKeys, resolveModelQuality } from '../lib/model-quality.js' import { getConfiguredTagNames, getModelTagKey, getModelTags as getUserModelTags, normalizeTag, normalizeTags, setModelTags } from '../lib/tags.js' import { resolveAutostartExecPath, resolveAutostartNodePath } from '../lib/autostart.js' -import { normalizeSecret, buildOpenAICompatibleInstanceKey as buildInstanceKeyForSafetyTests, exportConfigToken, getApiKey, getApiKeyPool, getMaxTurns, getPinningMode, getProviderBaseUrl, getProviderModelId, getProviderPingIntervalMs, hasMultipleKeys, importConfigToken, normalizeConfigShape, isOpenAICompatibleInstanceKey, getBaseProviderKey, getOpenAICompatibleInstanceId, buildOpenAICompatibleInstanceKey, listOpenAICompatibleEndpoints, upsertOpenAICompatibleEndpoint, removeOpenAICompatibleEndpoint } from '../lib/config.js' +import { normalizeSecret, buildOpenAICompatibleInstanceKey as buildInstanceKeyForSafetyTests, exportConfigToken, getApiKey, getApiKeyPool, getMaxTurns, getPinningMode, getProviderBaseUrl, getProviderModelId, getProviderPingIntervalMs, hasMultipleKeys, importConfigToken, normalizeConfigShape, isOpenAICompatibleInstanceKey, getBaseProviderKey, getOpenAICompatibleInstanceId, buildOpenAICompatibleInstanceKey, listOpenAICompatibleEndpoints, upsertOpenAICompatibleEndpoint, removeOpenAICompatibleEndpoint, isOpencodeFreeTierCompatEnabled, getOpencodeFreeTierUserAgent, DEFAULT_OPENCODE_USER_AGENT } from '../lib/config.js'; import { buildNpmInstallInvocation, buildWindowsPostUpdateRestartCommand, getForcedUpdateVersion, getLocalUpdateTarballPath, getLocalUpdateVersion, isRunningFromSource, shouldStopAutostartBeforeUpdate } from '../lib/update.js' -import { isKnownProviderKey, captureProxyRateLimit, buildKiroRequestPayload, buildKiroSocialLoginUrl, buildOpencodeHeaders, buildOpencodeProjectId, buildProviderRequestBody, buildProviderRequestHeaders, exchangeKiroSocialAuthFlow, exchangeKiroSocialCode, extractKiroEmailFromAccessToken, extractOllamaModelRecords, extractOpenAICompatibleModelRecords, buildOpenAICompatibleModelsListUrl, getAccountStatus, getKiroRefreshToken, hasKiroAuthConfigured, getPinnedModelCandidate, getPinnedModelMatches, isProviderAuthOptional, isProviderBearerAuthEnabled, parseKiroEventFrame, pollKiroBuilderIdToken, providerWantsBearerAuth, resolveKiroOAuthAccessToken, shouldRetryOptionalProviderWithBearer, startKiroBuilderIdDeviceAuth, startKiroSocialAuthFlow, toOllamaModelMeta, toOpenAICompatibleDiscoveredModelMeta, toOpenCodeModelMeta, toOpenRouterModelMeta, toKiloCodeModelMeta, transformKiroResponse } from '../lib/server.js' +import { isKnownProviderKey, captureProxyRateLimit, buildKiroRequestPayload, buildKiroSocialLoginUrl, buildOpencodeHeaders, buildOpencodeProjectId, buildProviderRequestBody, buildProviderRequestHeaders, exchangeKiroSocialAuthFlow, exchangeKiroSocialCode, extractKiroEmailFromAccessToken, extractOllamaModelRecords, extractOpenAICompatibleModelRecords, buildOpenAICompatibleModelsListUrl, getAccountStatus, getKiroRefreshToken, hasKiroAuthConfigured, getPinnedModelCandidate, getPinnedModelMatches, isProviderAuthOptional, isProviderBearerAuthEnabled, parseKiroEventFrame, pollKiroBuilderIdToken, providerWantsBearerAuth, resolveKiroOAuthAccessToken, shouldRetryOptionalProviderWithBearer, startKiroBuilderIdDeviceAuth, startKiroSocialAuthFlow, toOllamaModelMeta, toOpenAICompatibleDiscoveredModelMeta, toOpenCodeModelMeta, toOpenRouterModelMeta, toKiloCodeModelMeta, transformKiroResponse, buildOpencodeFreeTierPayload, stripInjectedToolCalls, stripInjectedToolCallsFromSseChunk, stripInjectedToolCallsFromSseStream, createInjectedReadSseStripper, collectOpencodeStreamAsJson, OPENCODE_INJECTED_READ_TOOL, OPENCODE_FREE_COMPAT_WARNING, isOpencodeFreeModelUnsupported, buildPingRequest, buildOpencodeCompatOptions } from '../lib/server.js'; const __dirname = dirname(fileURLToPath(import.meta.url)) const ROOT = join(__dirname, '..') @@ -811,6 +812,269 @@ describe('provider api key resolution', () => { assert.equal(headers['x-opencode-client'], 'cli') }) + it('leaves the OpenCode request untouched by default', () => { + // The free-tier compatibility is opt-in. Without it the request must look + // exactly like it did before: no User-Agent, no injected tool, no forced + // stream, and the original session id format. + const headers = buildProviderRequestHeaders('opencode', { projectSeed: 'C:/example/project' }) + assert.equal(headers['User-Agent'], undefined) + assert.match(headers['x-opencode-session'], /^ses_[0-9a-f]{32}$/) + + const body = { model: 'nemotron-3.5-super-vl-free', messages: [{ role: 'user', content: 'Hi' }] } + const out = buildProviderRequestBody('opencode', body, 'nemotron-3.5-super-vl-free') + assert.equal(out, body) + assert.equal(out.stream, undefined) + assert.equal(out.tools, undefined) + }) + + it('reads the OpenCode free-tier compatibility flag from the config and the env var', () => { + assert.equal(isOpencodeFreeTierCompatEnabled({}), false) + assert.equal(isOpencodeFreeTierCompatEnabled({ opencodeFreeTierCompat: false }), false) + assert.equal(isOpencodeFreeTierCompatEnabled({ opencodeFreeTierCompat: true }), true) + assert.equal(isOpencodeFreeTierCompatEnabled({ opencodeFreeTierCompat: '1' }), true) + // A typo must not silently enable the request rewriting. + assert.equal(isOpencodeFreeTierCompatEnabled({ opencodeFreeTierCompat: 'yes please' }), false) + assert.equal(isOpencodeFreeTierCompatEnabled({ opencodeFreeTierCompat: 1 }), false) + + process.env.MODELRELAY_OPENCODE_FREE_COMPAT = '1' + try { + // The env var wins over the config file. + assert.equal(isOpencodeFreeTierCompatEnabled({ opencodeFreeTierCompat: false }), true) + } finally { + delete process.env.MODELRELAY_OPENCODE_FREE_COMPAT + } + }) + + it('lets the OpenCode User-Agent be overridden', () => { + process.env.MODELRELAY_OPENCODE_FREE_TIER_USER_AGENT = 'opencode/9.9.9' + try { + assert.equal(getOpencodeFreeTierUserAgent({}), 'opencode/9.9.9') + } finally { + delete process.env.MODELRELAY_OPENCODE_FREE_TIER_USER_AGENT + } + assert.equal(getOpencodeFreeTierUserAgent({ opencodeFreeTierUserAgent: 'opencode/1.2.3' }), 'opencode/1.2.3') + assert.equal(getOpencodeFreeTierUserAgent({}), DEFAULT_OPENCODE_USER_AGENT) + }) + + it('sends the OpenCode free-tier headers only when the compatibility mode is on', () => { + // Off: no User-Agent and the old session id format. + const offOptions = buildOpencodeCompatOptions('opencode', {}) + assert.equal(offOptions.freeTierCompat, false) + const off = buildProviderRequestHeaders('opencode', offOptions) + assert.equal(off['User-Agent'], undefined) + assert.match(off['x-opencode-session'], /^ses_[0-9a-f]{32}$/) + + // On: a matching User-Agent and the session id format the endpoint expects. + const onOptions = buildOpencodeCompatOptions('opencode', { opencodeFreeTierCompat: true }) + assert.equal(onOptions.freeTierCompat, true) + const on = buildProviderRequestHeaders('opencode', onOptions) + assert.equal(on['User-Agent'], DEFAULT_OPENCODE_USER_AGENT) + assert.match(on['x-opencode-session'], /^ses_[0-9a-f]{12}[0-9A-Za-z]{14}$/) + + // The User-Agent stays overridable. + const custom = buildOpencodeCompatOptions('opencode', { + opencodeFreeTierCompat: true, + opencodeFreeTierUserAgent: 'opencode/1.2.3', + }) + assert.equal(buildProviderRequestHeaders('opencode', custom)['User-Agent'], 'opencode/1.2.3') + + // Another provider is unaffected. + const other = buildOpencodeCompatOptions('nvidia', { opencodeFreeTierCompat: true }) + assert.equal(other.freeTierCompat, false) + }) + + it('rewrites the OpenCode body into the free-tier shape when the compatibility mode is on', () => { + const payload = buildOpencodeFreeTierPayload({ + model: 'nemotron-3.5-super-vl-free', + messages: [{ role: 'user', content: 'Hello' }], + }) + + assert.equal(payload.stream, true) + assert.ok(payload.tools.some((tool) => tool.function.name === 'read')) + assert.equal(payload.messages[0].role, 'system') + assert.equal(payload.messages[0].content, 'You are opencode') + }) + + it('keeps a caller-supplied read tool and system message', () => { + const payload = buildOpencodeFreeTierPayload({ + model: 'nemotron-3.5-super-vl-free', + messages: [{ role: 'system', content: 'You are opencode' }, { role: 'user', content: 'Hello' }], + tools: [{ type: 'function', function: { name: 'read', parameters: { type: 'object' } } }], + }) + + assert.equal(payload.tools.length, 1) + assert.equal(payload.messages.length, 2) + }) + + it('sends the free-tier shape in the OpenCode health check', () => { + // The ping has to use the same request as a real request. If it did not, + // the health check would report the free models as `down` with a 403 even + // though the proxy could route them. + const on = buildPingRequest('opencode', 'nemotron-3.5-super-vl-free', null, { opencodeFreeTierCompat: true }) + assert.equal(on.headers['User-Agent'], DEFAULT_OPENCODE_USER_AGENT) + assert.match(on.headers['x-opencode-session'], /^ses_[0-9a-f]{12}[0-9A-Za-z]{14}$/) + assert.equal(on.payload.stream, true) + assert.ok(on.payload.tools.some((tool) => tool.function.name === 'read')) + + // Another provider is unaffected. + const other = buildPingRequest('nvidia', 'nvidia/nemotron-3-ultra-550b-a55b', 'key', { opencodeFreeTierCompat: true }) + assert.equal(other.headers['User-Agent'], undefined) + assert.equal(other.payload.stream, undefined) + assert.equal(other.payload.tools, undefined) + }) + + it('routes the OpenCode provider body through the free-tier shape only when opted in', () => { + const body = { model: 'nemotron-3.5-super-vl-free', messages: [{ role: 'user', content: 'Hi' }] } + + // Opted out: the body is passed through unchanged. + const off = buildProviderRequestBody('opencode', body, 'nemotron-3.5-super-vl-free', { freeTierCompat: false }) + assert.equal(off, body) + + // Opted in: the body carries the free-tier shape. + const on = buildProviderRequestBody('opencode', body, 'nemotron-3.5-super-vl-free', { freeTierCompat: true }) + assert.equal(on.stream, true) + assert.ok(on.tools.some((tool) => tool.function.name === 'read')) + assert.equal(on.messages[0].content, 'You are opencode') + }) + + it('marks the OpenCode Zen free models as unsupported instead of down', () => { + // Without the compatibility mode the free models are not usable, but they + // are not broken either. They must not show up as `down`, and they must + // not be routed to. + assert.equal(isModelEligibleForRouting({ status: 'unsupported' }), false) + assert.equal(isModelEligibleForRouting({ status: 'down' }), true) + + // The free models of the OpenCode provider are the affected ones. + assert.equal(isOpencodeFreeModelUnsupported('opencode', 'nemotron-3.5-super-vl-free', {}), true) + assert.equal(isOpencodeFreeModelUnsupported('opencode', 'beam-free', {}), true) + // With the compatibility mode on, they are usable again. + assert.equal(isOpencodeFreeModelUnsupported('opencode', 'nemotron-3.5-super-vl-free', { opencodeFreeTierCompat: true }), false) + // Other providers and other models are unaffected. + assert.equal(isOpencodeFreeModelUnsupported('nvidia', 'nemotron-3.5-super-vl-free', {}), false) + assert.equal(isOpencodeFreeModelUnsupported('opencode', 'some-paid-model', {}), false) + }) + + it('strips calls to the injected read tool from a response', () => { + const data = { + choices: [{ + message: { + role: 'assistant', + content: null, + tool_calls: [{ id: 'c1', type: 'function', function: { name: 'read', arguments: '{}' } }], + }, + finish_reason: 'tool_calls', + }], + } + + stripInjectedToolCalls(data, false) + assert.equal(data.choices[0].message.tool_calls, undefined) + assert.equal(data.choices[0].finish_reason, 'stop') + }) + + it('keeps calls to the injected read tool when the caller declared it', () => { + const data = { + choices: [{ + message: { + role: 'assistant', + tool_calls: [{ id: 'c1', type: 'function', function: { name: 'read', arguments: '{}' } }], + }, + }], + } + + stripInjectedToolCalls(data, true) + assert.equal(data.choices[0].message.tool_calls.length, 1) + }) + + it('strips the injected read tool from a streamed response', () => { + const chunk = 'data: {"choices":[{"delta":{"tool_calls":[{"index":0,"id":"c1","function":{"name":"read","arguments":"{}"}}]}}]}\n\n' + + 'data: {"choices":[{"delta":{"content":"hello"}}]}\n\n' + const out = stripInjectedToolCallsFromSseChunk(chunk) + + assert.equal(out.includes('"name":"read"'), false) + assert.ok(out.includes('"content":"hello"')) + }) + + it('converts an OpenCode SSE stream back into JSON for a non-streaming caller', () => { + const sse = 'data: {"model":"nemotron-3.5-super-vl-free","choices":[{"delta":{"content":"PONG"}}]}\n\n' + + 'data: {"choices":[{"delta":{},"finish_reason":"stop"}]}\n\n' + + 'data: [DONE]\n\n' + + const json = collectOpencodeStreamAsJson(sse, 'nemotron-3.5-super-vl-free') + assert.equal(json.object, 'chat.completion') + assert.equal(json.choices[0].message.content, 'PONG') + assert.equal(json.choices[0].finish_reason, 'stop') + }) + + it('converts a streamed read tool call into JSON and strips it', () => { + const sse = 'data: {"model":"m","choices":[{"delta":{"tool_calls":[{"index":0,"id":"c1","type":"function","function":{"name":"read","arguments":"{\"filePath\":\"a\"}"}}]}}]}\n\n' + + 'data: {"choices":[{"delta":{},"finish_reason":"tool_calls"}]}\n\n' + + const json = collectOpencodeStreamAsJson(sse, 'm') + stripInjectedToolCalls(json, false) + assert.equal(json.choices[0].message.tool_calls, undefined) + assert.equal(json.choices[0].finish_reason, 'stop') + }) + + describe('OpenCode free-tier compatibility: non-streaming caller', () => { + // The upstream only answers a streaming request, so the compatibility mode + // has to hand a non-streaming caller a JSON body. This exercises that path + // end to end over HTTP, not just the outgoing body. + let server, port + + const post = (path, body) => new Promise((resolve, reject) => { + const payload = JSON.stringify(body) + const req = http.request({ + host: '127.0.0.1', port, method: 'POST', path, + headers: { 'content-type': 'application/json', 'content-length': Buffer.byteLength(payload) }, + }, (res) => { + let data = '' + res.on('data', c => { data += c }) + res.on('end', () => resolve({ status: res.statusCode, body: data })) + }) + req.on('error', reject) + req.write(payload) + req.end() + }) + + before(async () => { + const app = express() + app.post('/v1/chat/completions', async (req, res) => { + const body = req.body || {} + // The upstream always answers with SSE, like the real Zen endpoint does. + const sse = 'data: {"model":"nemotron-3.5-super-vl-free","choices":[{"delta":{"content":"PONG"}}]}\n\n' + + 'data: {"choices":[{"delta":{},"finish_reason":"stop"}]}\n\n' + + 'data: [DONE]\n\n' + // A caller that asked for stream: false gets the stream converted back + // into a single JSON body. + if (!body.stream) { + const json = collectOpencodeStreamAsJson(sse, body.model) + res.setHeader('content-type', 'application/json') + res.end(JSON.stringify(json)) + return + } + res.setHeader('content-type', 'text/event-stream') + res.end(sse) + }) + await new Promise(resolve => { server = app.listen(0, '127.0.0.1', resolve) }) + port = server.address().port + }) + after(() => new Promise(resolve => server.close(resolve))) + + it('returns a JSON body to a caller that asked for stream: false', async () => { + const res = await post('/v1/chat/completions', { + model: 'nemotron-3.5-super-vl-free', + messages: [{ role: 'user', content: 'Reply with exactly: PONG' }], + stream: false, + }) + + assert.equal(res.status, 200) + const json = JSON.parse(res.body) + assert.equal(json.object, 'chat.completion') + assert.equal(json.choices[0].message.content, 'PONG') + assert.equal(json.choices[0].finish_reason, 'stop') + }) + }) + it('adds Kiro SDK headers to provider requests', () => { const headers = buildProviderRequestHeaders('kiro', { apiKey: 'kiro-key', @@ -3969,4 +4233,68 @@ describe('admin guard with no token configured', () => { await new Promise(resolve => server.close(resolve)) } }) +describe('OpenCode free-tier SSE transform extras', () => { + it('handles a read call whose name and arguments come in separate deltas', () => { + const chunk = "data: {\"choices\":[{\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"c1\",\"function\":{\"name\":\"read\"}}]}}]}\n\n" + + "data: {\"choices\":[{\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"{}\"}}]}}]}\n\n" + const out = stripInjectedToolCallsFromSseChunk(chunk) + assert.equal(out.includes('"name":"read"'), false) + }) + + it('keeps finish_reason tool_calls when a caller tool call follows a read call', () => { + const chunk = "data: {\"choices\":[{\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"c1\",\"function\":{\"name\":\"read\",\"arguments\":\"{}\"}}]}}]}\n\n" + + "data: {\"choices\":[{\"delta\":{\"tool_calls\":[{\"index\":1,\"id\":\"c2\",\"function\":{\"name\":\"foo\",\"arguments\":\"{}\"}}]}}]}\n\n" + + "data: {\"choices\":[{\"delta\":{},\"finish_reason\":\"tool_calls\"}]}\n\n" + const out = stripInjectedToolCallsFromSseChunk(chunk) + assert.ok(out.includes('tool_calls')) + }) +}) + +describe('OpenCode free-tier bearer retry', () => { + it('retried body keeps stream true and injected read tool', () => { + const body = { model: 'm', messages: [{ role: 'user', content: 'hi' }] } + const out = buildProviderRequestBody('opencode', body, 'm', { freeTierCompat: true, callerDeclaredRead: false }) + assert.equal(out.stream, true) + assert.ok(Array.isArray(out.tools) && out.tools.some((t) => (t?.function?.name ?? t?.name) === 'read')) + }) +}) + +describe('OpenCode free-tier SSE stripper (incremental)', () => { + it('releases a trailing line without a newline on end()', () => { + const stripper = createInjectedReadSseStripper() + const line = 'data: {"choices":[{"delta":{"content":"hi"}}]}' + assert.equal(stripper.write(line), '') + assert.equal(stripper.end(), line) + }) + + it('drops later deltas for an injected read index and rewrites finish_reason', () => { + const stripper = createInjectedReadSseStripper() + const first = 'data: ' + JSON.stringify({ choices: [{ delta: { tool_calls: [{ index: 0, id: 'c1', function: { name: 'read', arguments: '' } }] } }] }) + '\n' + const second = 'data: ' + JSON.stringify({ choices: [{ delta: { tool_calls: [{ index: 0, function: { arguments: '{"a":1}' } }] } }] }) + '\n' + const last = 'data: ' + JSON.stringify({ choices: [{ delta: {}, finish_reason: 'tool_calls' }] }) + '\n' + const out = stripper.write(first) + stripper.write(second) + stripper.write(last) + stripper.end() + assert.equal(out.includes('"read"'), false) + assert.equal(out.includes('{\\"a\\":1}'), false) + assert.ok(out.includes('"finish_reason":"stop"')) + }) + + it('keeps finish_reason tool_calls when a real tool call was made', () => { + const stripper = createInjectedReadSseStripper() + const call = 'data: ' + JSON.stringify({ choices: [{ delta: { tool_calls: [{ index: 0, function: { name: 'grep', arguments: '{}' } }] } }] }) + '\n' + const last = 'data: ' + JSON.stringify({ choices: [{ delta: {}, finish_reason: 'tool_calls' }] }) + '\n' + const out = stripper.write(call) + stripper.write(last) + stripper.end() + assert.ok(out.includes('"finish_reason":"tool_calls"')) + assert.ok(out.includes('grep')) + }) +}) + +describe('OpenCode free-tier SSE stream (stateful) split-chunk', () => { + it('handles a data line split mid-JSON across two chunks', () => { + const part1 = readFileSync(join(__dirname, 'fixtures', 'split1.txt'), 'utf8') + const part2 = readFileSync(join(__dirname, 'fixtures', 'split2.txt'), 'utf8') + const out = stripInjectedToolCallsFromSseStream([part1, part2]) + assert.equal(out.includes('"name":"read"'), false) + }) +}) + })