Back ChatSnowflake() with the Cortex REST API: thin Anthropic/OpenAI provider subclasses, model-name dispatch, credentials-callable auth

[cpsievert]

Proposal: back ChatSnowflake() with the Cortex REST API, delegating to AnthropicProvider/OpenAICompletionsProvider

Context

claude-sonnet-5 on Snowflake Cortex sends extended thinking blocks that the current SnowflakeProvider (built on snowflake-ml-python's CompleteRequest) never had to handle. Supporting them requires Snowflake-specific parsing/merging logic for thinking + signature fragments that duplicates what AnthropicProvider already does. Every future Anthropic (or OpenAI) feature Snowflake adopts — redacted thinking, interleaved thinking, server tools, new delta types — will require another patch against Snowflake's proprietary, undocumented streaming format, which the provider's own comments already flag as unstable. (Short-term fix for the immediate crash: branch fix/snowflake-anthropic-thinking-delta.)

The opportunity

Snowflake's Cortex REST API exposes two industry-standard endpoints:

	Chat Completions API	Messages API
Compatibility	OpenAI Chat Completions	Anthropic Messages API
Endpoint	/api/v2/cortex/v1/chat/completions	/api/v2/cortex/v1/messages
Models	All (Claude, OpenAI, Llama, Mistral, ...)	Claude only (incl. claude-sonnet-5)
SDK	openai	anthropic

Both are explicitly documented to work with the stock SDKs by overriding base_url and setting a Bearer token. The Messages API supports adaptive/extended thinking with signatures that round-trip in multi-turn conversations, proper SSE streaming, Anthropic-format tool calling, prompt caching, structured output, and anthropic-beta headers (interleaved thinking, effort, etc.).

Proposal

Combine two patterns already in chatlas:

Subclassing mechanics from BedrockMessagesProvider (chatlas/_provider_bedrock.py): thin subclasses that customize client construction and inherit everything else.

@no_file_management  # no Files API on the gateway
class SnowflakeAnthropicProvider(AnthropicProvider):
    """Reaches Snowflake Cortex's Anthropic Messages API."""
    # ~60 lines: anthropic.Anthropic with
    # base_url="https://<account>.snowflakecomputing.com/api/v2/cortex"

class SnowflakeOpenAIProvider(OpenAICompletionsProvider):
    """Reaches Snowflake Cortex's OpenAI Chat Completions API."""
    # base_url="https://<account>.snowflakecomputing.com/api/v2/cortex/v1"

Dispatch and auth ergonomics from ChatPosit (chatlas/_provider_posit.py):

• Dispatch by model name, not an api= param. ChatPosit selects its Anthropic vs OpenAI provider via model.startswith("claude"). Snowflake's endpoint coverage is cleanly partitioned by model family (Claude → Messages, everything else → Chat Completions), so unlike Bedrock — where api= exists because the APIs overlap in model coverage — no explicit api= argument is needed. (An optional override could still force the legacy path during a transition period.)
• A credentials: Callable[[], str] parameter + an httpx auth flow that swaps the SDK's default x-api-key header for Authorization: Bearer <token> — exactly what Snowflake requires, per its own docs. Because the callable is invoked per request, expiring tokens (OAuth, key-pair JWT) are naturally supported. One callable serves both REST flavors.

Benefits:

• All Anthropic/OpenAI thinking, tool-use, and streaming logic is inherited, not duplicated; future Snowflake-adopted features arrive for free.
• Thinking signatures round-trip correctly in multi-turn tool loops (the CompleteRequest path currently drops ContentThinking when re-serializing history).
• End state mirrors ChatPosit: two thin provider subclasses plus auth, and the snowflake-ml-python dependency — along with its janky stream-merging code — can be deprecated rather than patched.

Open questions / costs

• Auth is the main work. The REST API needs a Bearer token: PAT (simplest for users), OAuth, or key-pair JWT (X-Snowflake-Authorization-Token-Type: KEYPAIR_JWT; the header is optional since Snowflake sniffs token type). To preserve the current connections.toml / private_key_file ergonomics, chatlas should ship a default credentials helper that mints/refreshes a JWT from the existing connection params. Session tokens should be discouraged: their expiration surfaces as HTTP 200 with error code 390112, which would need a response-body-inspecting error hook.
• Observability differs: REST API calls don't write to AI_OBSERVABILITY_EVENTS; usage moves to CORTEX_REST_API_USAGE_HISTORY. Worth documenting.
• Transition plan: keep the legacy provider as the default initially, gate the new path behind opt-in (or new-model dispatch), and deprecate once the auth story is proven.

Related

• Short-term thinking-delta fix on the legacy path: branch fix/snowflake-anthropic-thinking-delta

[cpsievert]

Auth investigation: how involved is it really?

Follow-up analysis on the authentication question above. Short version: rolling our own token handling is cheap, and snowflake-connector-python only needs to be a soft import for the session-sharing case.

Dependency math

The current chatlas[snowflake] extra installs snowflake-ml-python (55 direct deps, including torch — hence the <=1.9.0 pin and Python >3.11 incompatibility workaround in pyproject.toml). A self-maintained auth layer needs only cryptography + PyJWT (both small, both already transitive deps of the connector), with snowflake-connector-python (34 direct deps) demoted to an optional, soft-imported dependency.

Cost per auth method

Method	Implementation	Notes
PAT / static token (SNOWFLAKE_TOKEN)	~5 lines	Env var or param, sent as Bearer. Zero deps.
Key-pair JWT	~40 lines, self-maintained	Snowflake officially documents the recipe: RS256 JWT, iss = ACCOUNT.USER.SHA256:<b64 fingerprint>, ~10-min lifetime + cache. ellmer already maintains exactly this in R (incl. privatelink account normalization). The connector's AuthByKeyPair does the same internally via jwt.encode but doesn't expose a clean "give me a REST token" API, so borrowing it buys little.
connections.toml	~20 lines with stdlib tomllib	Parse account/user/private_key_file ourselves — no connector needed.
Workbench-managed	~15 lines	ellmer's trick: re-read $SNOWFLAKE_HOME/connections.toml per request for a fresh OAuth token.
Connect viewer credentials	Small, TBD	ellmer uses the R connectcreds package; need to confirm the Python equivalent before claiming parity.

Parity baseline

ellmer's chat_snowflake() is already REST-based (ProviderSnowflakeCortex extending its OpenAI-compatible provider, hitting /api/v2/cortex/inference:complete, credentials as a header-returning callable). Its ambient credential support — SNOWFLAKE_TOKEN, SNOWFLAKE_USER/SNOWFLAKE_PRIVATE_KEY, Workbench-managed, Connect viewer — is all achievable with no Snowflake SDK, so that sets the parity bar.

Session sharing survives

Session tokens are accepted by the Cortex REST API, so ChatSnowflake(session=...) keeps working: a credentials callable that duck-types the passed snowpark Session and reads session.connection.rest.token at request time. Caveats:

• rest.token is quasi-private connector API — stable for years, but isolate it behind one small function with a clear error message.
• Idle sessions produce expired tokens, returned as HTTP 200 with error code 390112. Extend the Posit-style error-hook pattern: detect it, and if we hold a live session, refresh-and-retry once.

Password auth deprecation note

The one casualty of a pure-REST design: you can't turn a password into a REST Bearer token without opening a connector session. Options: soft-import the connector only when password auth is requested, or drop password auth on the new path (defensible — Snowflake is phasing out password-only sign-in) and point affected users at key-pair/PAT, with the legacy provider available during the transition. Either way, this is a behavior change to document in the changelog.

Suggested API shape

credentials: Callable[[], str] as the escape hatch (PAT, OAuth, anything), ambient defaults resolved in order (env vars → connections.toml → Workbench), and session= for the shared-session case — consistent with the ChatPosit pattern referenced above.

[AbarnaaSree]

Hi @cpsievert, I'd be interested in working on this issue.

I went through the proposal and the main direction makes sense to me: thin Snowflake-specific subclasses over the existing Anthropic/OpenAI providers, model-name dispatch, and a callable credentials flow.

Before starting implementation, I'd like to inspect the existing Snowflake, Bedrock, ChatPosit, Anthropic, and OpenAI provider patterns and propose the concrete changes, particularly around authentication and session= compatibility.

If this approach still matches your intended direction, I'd be happy to take this up.