Skip to content

feat(provider): connect a local endpoint that needs no authentication - #56

Merged
tournierjc merged 2 commits into
mainfrom
feat/local-model-endpoint
Sep 23, 2026
Merged

tournierjc merged 2 commits into
mainfrom
feat/local-model-endpoint

Conversation

@tournierjc

Copy link
Copy Markdown
Owner

What

A model served on a plain OpenAI-compatible endpoint — Ollama, llama.cpp, vLLM, LM Studio — needs no credential, and kcode refused to connect one at three independent layers:

  • kcode provider add failed with Provider API key is missing. Set MCODE_PROVIDER_API_KEY or pass --api-key-env <name>.
  • kcode provider test answered Provider API key is not configured.
  • A turn stopped in the resolver: LocalModelResolver: api_key not configured for provider "custom_provider:…".

The key is now optional everywhere a provider is created, saved, tested or resolved. Nothing infers "no authentication" from a URL, and no new vocabulary was introduced: an entry that carries no key simply declares no credential scheme, and no request to it is sent with one.

How

packages/shared/src/credential-headers.ts (new) names the credential headers the three protocols use, clears them, and owns the placeholder key the vendored transport requires (openai-completions throws on a falsy key, so packages/agent-core hands it one that must never reach the wire).

Layer Change
Resolution The BYOK plan and the managed path resolve a keyless endpoint to the placeholder plus unauthenticatedEndpoint; composeStreamFn turns that into cleared credential headers on the request's own options — the last header source pi-ai merges, and the one its SDKs read a null from as "remove this default header". A credential the connection declares itself survives.
Auxiliary calls The remote token counter skips its probe for such an endpoint, because its requests are credential-shaped; it estimates locally instead. buildProviderHeaders and buildModelDiscoveryHeaders omit the credential header when no key is configured, and the connection test's redaction list tolerates an absent key.
Management A candidate or a created provider without a key is stored with no apiKey and no authMode; an empty key clears a saved one instead of being rejected as invalid. Such a connection is selectable — the connection test is the gate, as it already is for a connection whose test failed.
CLI provider add no longer requires a key or --api-key-env (naming a variable that is empty is still an error: the key was asked for and not found). provider list reports no key sent rather than no key.
TUI The catalogue gains a Local model entry that pre-fills an OpenAI-compatible endpoint (editable; Ollama's default port) and skips the protocol step. The API Key step is labelled optional, an empty value connects, and the /provider editor tests and saves a connection without a key and clears a saved one when its API Key field is submitted empty.

Evidence

test:byok runs the whole path against a loopback stand-in — provider add, provider list, the connection test and a real turn — and asserts that no request carries Authorization or x-api-key, including the token-counter probe the first run caught.

A PTY run of the built CLI against an isolated data directory and the same kind of stand-in drove the new catalogue entry end to end: the row renders as Local model OpenAI-compatible server · API key optional, the flow reaches API Key (optional) with Optional: a server that needs no authentication is connected with the field left empty, an empty key connects (Provider added: Local model), and the stand-in logged one request — /v1/chat/completions — with authorization: null and x-api-key: null.

Unit coverage: 7 cases in packages/agent-core/test/unit/pi-turn-runner/unauthenticated-endpoint.test.ts pin the header clearing and the declared-header exception; the resolver, plan, counter, management, onboarding and editor suites pin the keyless plan, the placeholder, the local estimate, the saved shape, the selectable connection, the new catalogue entry, the optional key step and the cleared key. pnpm verify is green — the full gate list and its counts are in the docs/verification.md entry this PR adds.

Limits

  • The legacy v1 host (packages/local-runtime/src/runtime/host.ts and its own model resolver) still requires a key; the product drives the v2 runtime, and only its token counter is shared with v1 (that one is handled here).
  • An endpoint that needs a credential but is saved without one now fails with the endpoint's own 401 at connection-test time rather than a local "api key not configured" error. That is the deliberate cost of "the key is optional everywhere": the runtime does not guess which endpoints require one.
  • The remote token counter estimates locally for such an endpoint instead of probing it, so auto-compaction thresholds rely on the shared BPE estimate there.

A model served on a plain OpenAI-compatible endpoint — Ollama, llama.cpp,
vLLM, LM Studio — needs no credential, and the runtime refused to connect one
at three independent layers: `provider add` demanded an API key, the
connection test reported `Provider API key is not configured`, and a turn
failed in `LocalModelResolver` with `api_key not configured`.

A key is now optional everywhere a provider is created, saved, tested or
resolved. Nothing infers "no authentication" from a URL and no new vocabulary
was introduced: an entry that carries no key simply declares no credential
scheme, and every request to it is sent without one.

- `packages/shared/src/credential-headers.ts` (new) names the credential
  headers the three protocols use, clears them, and owns the placeholder key
  the vendored transport requires (`openai-completions` throws on a falsy key).
- Resolution: the BYOK plan and the managed path resolve a keyless endpoint to
  that placeholder plus `unauthenticatedEndpoint`, which `composeStreamFn`
  turns into cleared credential headers on the request's own options — the
  last header source pi-ai merges, and the one its SDKs read a `null` from as
  "remove this default header". A credential the connection declares itself
  survives.
- Auxiliary calls stay credential-free too: the remote token counter skips its
  probe for such an endpoint (its requests are credential-shaped and it
  estimates locally instead), and the connectivity paths
  (`buildProviderHeaders`, `buildModelDiscoveryHeaders`, the connection test,
  model discovery) omit the credential header when no key is configured.
- Management: a candidate or a created provider without a key is stored with
  no `apiKey` and no `authMode`, and an empty key clears a saved one instead of
  being rejected as invalid. A keyless connection is selectable; the connection
  test is the gate, as it already is for a failed test.
- CLI: `provider add` no longer requires a key or `--api-key-env`, and
  `provider list` reports `no key sent` rather than `no key`.
- TUI: the catalogue gains a **Local model** entry that pre-fills an
  OpenAI-compatible endpoint (editable, Ollama's default port) and skips the
  protocol step; the API Key step is labelled optional and an empty value
  connects; the `/provider` editor tests and saves a connection with no key and
  clears a saved one when its API Key field is submitted empty.

Coverage: a new agent-core case set pins the header clearing, the resolver and
plan cases pin the keyless plan and the placeholder, the counter case pins the
local estimate, the management cases pin the saved shape and the selectable
connection, the TUI cases pin the entry, the optional key step and the cleared
key, and `test:byok` runs the whole path against a loopback stand-in —
`provider add`, `provider list`, the connection test and a turn — asserting
that no request carries `Authorization` or `x-api-key`.
The verification record carries the code revision, the full gate list with counts, the 16 tests the change adds, the end-to-end BYOK run and the PTY session, and its NOT RUN list. The provider documentation now describes the keyless case where it already described a local server: README and docs/examples.md state the lookup rule for the key variable instead of promising that no credential is ever sent when the flag is omitted.
@tournierjc
tournierjc merged commit dc2c32b into main Sep 23, 2026
17 checks passed
@tournierjc
tournierjc deleted the feat/local-model-endpoint branch September 23, 2026 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants