Skip to content

Extension API: host.registerModelProvider plugs a model provider into Pi - #120

Open
grigio wants to merge 2 commits into
TannerMidd:mainfrom
grigio:feat/host-register-model-provider
Open

grigio wants to merge 2 commits into
TannerMidd:mainfrom
grigio:feat/host-register-model-provider

Conversation

@grigio

@grigio grigio commented Oct 21, 2026

Copy link
Copy Markdown

What changed

  • Adds host.registerModelProvider(provider, config) to the extension host (server, types/pocket.d.ts, and the reference docs), backed by ModelRuntime.registerProvider / unregisterProvider.
  • It takes Pi's own ProviderConfig (name, base URL, API, key, headers, models or refreshModels), and returns a function that removes the provider.
  • The removal is tied to the version of the module that registered it: a reload starts the new version before the old cleanups run, so an edit does not take out the provider the new version just registered. An older version's cleanup leaves the newer registration alone.
  • Documented as experimental in 0.20.0 and added to the API versions table.
  • New test test/model-providers.test.ts: a drop-in registers a provider, a reload keeps it, turning the module off removes it. I verified it fails without the registration guard.

Why

Extensions can shape model requests and read models, but a module that brings its own service could not give its models to Pi. This is the extension point that lets a drop-in (for example a provider package beside it) make its models selectable in every session.

Tested

  • npm run check
  • npm run lint
  • npm test (385 passing)

Extensions could shape requests and read models, but not add a provider
of their own: a module that brings a new service had no way to give its
models to Pi. `host.registerModelProvider(provider, config)` registers
Pi's own `ProviderConfig` with the model runtime, and its return value
removes the provider again.

The cleanup leaves a newer registration alone: a reloaded module starts
before the old version's cleanups run, so without the guard an edit
would take out the provider the new version just put in. The provider's
models appear at Pi's next model refresh, as with Refresh models.

Docs and types say it is experimental in 0.20.0. The test loads a
drop-in that registers a provider, checks a reload keeps it, and checks
that turning the module off removes it (it fails without the guard).
@peyloride

Copy link
Copy Markdown

I tested this branch with a real provider that has a dynamic catalog: Charm Hyper.

Setup: a drop-in calls host.registerModelProvider("hyper", { name, baseUrl: "https://hyper.charm.land/v1", api: "openai-completions", refreshModels }). The function reads the credential from context.credential, calls Hyper's /v1/provider, and maps the response to ProviderModelConfig[]. The config has no apiKey and no oauth, so the stored hyper credential is used.

Result:

  • The stored API key was used.
  • app.models.refresh({ allowNetwork: true, force: true }) produced 23 models.
  • hasConfiguredAuth("hyper") was true, and all 23 were available.
  • The branch's test/model-providers.test.ts passes.

Two findings:

  1. No models until a refresh. After the module loads, the provider has 0 models. They appear only after a refresh. The docs say this, but a dynamic provider looks broken when the owner turns it on. A static models list can cover the gap. It may be better for registerModelProvider to refresh that one provider after registration, then send the models update. If network on module load is not wanted, the reference should tell authors to ship a fallback list.

  2. The local pass calls refreshModels. With allowNetwork: false, a provider that returns [] empties its catalog until the network pass. Returning the stored catalog keeps it:

refreshModels: async ({ allowNetwork, stored, ...rest }) => {
    if (!allowNetwork) return stored?.models ?? [];
    // fetch the live catalog
}

A sentence in the reference and a test for this path would help. I can add the test if you want it. The rest of the design works for a real provider.

Two findings from a test with a real dynamic provider (Charm
Hyper): a provider without a static models list shows nothing
until a network refresh, and a refreshModels that ignores
allowNetwork and returns [] empties the catalog on every local
pass, before the network pass can restore it. The reference now
tells authors about both: ship a fallback models list, and
return the stored catalog when allowNetwork is false.
@grigio

grigio commented Oct 21, 2026

Copy link
Copy Markdown
Author

Thanks — both findings check out, and I traced them to the runtime to be sure:

  1. ModelRuntime.registerProvider ends with refresh({ allowNetwork: false }), so a provider with only refreshModels sees a local pass and shows 0 models until a network refresh. Documented below rather than auto-refreshed: network access at module load is a policy question I'd rather not settle inside an experimental API. If the provider ships a fallback static models list, those models are in the runtime immediately and refreshModels replaces them on the network pass — good enough until we decide.

  2. Right — every refresh, even the local one, calls refreshModels, and since an empty list is a return value (not a no-op), an oblivious [] empties the catalog on the local pass before the network pass can restore it. The stored pattern you wrote is the correct one.

I have pushed 0c628ac, which adds both to the registerModelProvider reference in docs/extensions/reference/host.md: the fallback models list for looking right away, and the if (!allowNetwork) return stored?.models ?? [] rule with a note that an empty list still counts as the provider's answer.

A test for the local-pass path is a good addition — please do add it if you have the setup fresh; otherwise I'll write one along the original drop-in's lines (register with refreshModels that returns stored?.models on the local pass, assert the network pass restores the catalog).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants