diff --git a/docs/api/orcarouter.md b/docs/api/orcarouter.md new file mode 100644 index 0000000..5741106 --- /dev/null +++ b/docs/api/orcarouter.md @@ -0,0 +1,79 @@ +# OrcaRouter Cyber Inference — BYOK provider + +OrcaRouter Cyber Inference is a named, OpenAI-compatible provider for security +teams. It is a *user-owned* credential path like every other AXGuard provider: +nothing is sent to OrcaRouter unless you select it explicitly and set a key. + +- Provider page: [https://www.orcarouter.ai/cyber](https://www.orcarouter.ai/cyber) +- Cyber model catalog: [https://www.orcarouter.ai/models?tier=cyber](https://www.orcarouter.ai/models?tier=cyber) +- API base: `https://api.orcarouter.ai/v1` + +It is aimed at teams doing **authorized, scoped** security work: red-team and +adversary-style analysis with cyber-tuned models on the red side, and the same +endpoint's gateway controls on the blue side (pre-ship review, triage, fix +verification). General-purpose gateways often reject or silently downgrade this +kind of request; this provider lets verified security teams route it to the +cyber tier. Requests outside that scope still work through the general layer. + +## Configure (local API / CLI) + +```bash +export AXGUARD_AI_MODE=user_key +export AXGUARD_AI_PROVIDER=orcarouter +export AXGUARD_AI_API_KEY=sk-orca-… # never logged; masked in /v1/health +# optional: +# export AXGUARD_AI_MODEL=orcarouter/auto # default +# export AXGUARD_AI_BASE_URL=https://api.orcarouter.ai/v1 # explicit override +``` + +Or in `.axguard.yml` for the GitHub bot (key still comes from the environment): + +```yaml +ai: + provider: orcarouter + mode: user_key + model: orcarouter/auto +``` + +## Models + +| Role | Model id | Notes | +|---|---|---| +| Default | `orcarouter/auto` | Auto-routed; works with any OrcaRouter key | +| Recommended for security teams | `orcacyber-ultra-1.0` (OrcaCyber Ultra) | Security code scanning, large-codebase multi-step reasoning; needs business verification | +| Recommended for security teams | `orcacyber-zero-1.5` / `orcacyber-zero-1.0` (OrcaCyber Zero) | OrcaCyber Zero harness models; needs business verification | +| General fallback | `orcarouter/fusion` | Non-cyber workloads | + +The provider's model list is live: `GET {base_url}/models` with your key, then +filtered per capability (see `engines/api/providers/catalog.py`). Ids keep their +`vendor/model` namespace. If the catalog fetch fails, `/v1/models` reports +`"degraded": true` with `"catalog_source": "seed"` and only the small verified +cold-start set (`engines/api/providers/orcarouter.py`); seed entries are never +mixed into a successful live result. + +```bash +axguard api models --capability chat # text dropdown +axguard api models --capability chat --modality image # multimodal understanding +axguard api models --check # non-zero if degraded +curl 'http://127.0.0.1:8787/v1/models?capability=chat' +``` + +Capability filters (metadata only, never model names): + +| `capability` | Rule | +|---|---| +| `chat` (default) | `supported_endpoint_types` includes `openai`, `anthropic`, `gemini` or `openai-response` | +| `chat&modality=image\|audio\|video` | chat model whose `architecture.input_modalities` declares the modality (undeclared = excluded) | +| `embedding` | `embeddings` endpoint | +| `image` | `image-generation` endpoint | +| `video` | `openai-video` endpoint | +| `rerank` | `jina-rerank` endpoint | + +The local API keeps the key server-side; `/v1/models` and `/v1/health` return +metadata only. + +## Errors + +401/403 from the catalog → `/v1/models` degrades to the verified seed and reports +the error text (key redacted). A rejected completion raises +`LLM provider HTTP 401/403: …` with any `sk-orca-…`/`Bearer …` token scrubbed. diff --git a/docs/api/overview.md b/docs/api/overview.md index 0e203d7..3d62108 100644 --- a/docs/api/overview.md +++ b/docs/api/overview.md @@ -32,4 +32,4 @@ axguard api start # → http://127.0.0.1:8787 ``` -See [quickstart](quickstart.md), [authentication](authentication.md), [providers](providers.md), [security](security.md), [mcp-contracts](mcp-contracts.md). +See [quickstart](quickstart.md), [authentication](authentication.md), [providers](providers.md), [OrcaRouter Cyber Inference](orcarouter.md), [security](security.md), [mcp-contracts](mcp-contracts.md). diff --git a/docs/api/providers.md b/docs/api/providers.md index 9dd1656..69f40f1 100644 --- a/docs/api/providers.md +++ b/docs/api/providers.md @@ -11,3 +11,14 @@ AXGuard has **no hosted model**. | `AXGUARD_AI_MODEL` | Model id | `enrichment=llm` returns **422** unless a real local/BYOK provider is configured. `NoneProvider` is valid for no-llm (`available=True`) but still rejected for enrichment. + +## Named provider: OrcaRouter Cyber Inference + +`AXGUARD_AI_PROVIDER=orcarouter` selects the named OrcaRouter Cyber Inference +provider ([OrcaRouter](https://www.orcarouter.ai), base +`https://api.orcarouter.ai/v1`, key prefix `sk-orca-`). Default model +`orcarouter/auto`; cyber-tier models (`orcacyber-ultra-1.0`, +`orcacyber-zero-1.5`) for verified security teams; `orcarouter/fusion` as the +general fallback. The model list is discovered live per capability +(`axguard api models`, `GET /v1/models`). Full guide: +[orcarouter.md](orcarouter.md). diff --git a/docs/github/ai-providers.md b/docs/github/ai-providers.md index 333b0a6..dccab24 100644 --- a/docs/github/ai-providers.md +++ b/docs/github/ai-providers.md @@ -16,7 +16,7 @@ Related: [privacy.md](privacy.md) · [config.md](config.md). | `user_key` | Customer-provided cloud key via `AXGUARD_AI_API_KEY` | `ai.provider` selects the adapter (`none`, `local`, `ollama`, `openai`, -`anthropic`, `openrouter`, …). Keys are never read from the analyzed repo. +`anthropic`, `openrouter`, `orcarouter`, …). Keys are never read from the analyzed repo. --- @@ -46,6 +46,28 @@ export AXGUARD_AI_BASE_URL=http://127.0.0.1:11434 --- +## Example — OrcaRouter Cyber Inference + +```yaml +ai: + provider: orcarouter + mode: user_key + # model: orcarouter/auto (default) +``` + +```bash +export AXGUARD_AI_API_KEY=sk-orca-… +``` + +Named provider for security teams ([OrcaRouter](https://www.orcarouter.ai/cyber), +base `https://api.orcarouter.ai/v1`). Default model `orcarouter/auto`; cyber-tier +models `orcacyber-ultra-1.0` / `orcacyber-zero-1.5` for verified security teams; +`orcarouter/fusion` as general fallback. Catalog: +[cyber tier](https://www.orcarouter.ai/models?tier=cyber) · +[local API guide](../api/orcarouter.md). + +--- + ## Example — user-owned cloud key ```yaml diff --git a/engines/api/cli.py b/engines/api/cli.py index 413f8d5..ec33341 100644 --- a/engines/api/cli.py +++ b/engines/api/cli.py @@ -23,6 +23,25 @@ def add_api_parser(sub: argparse._SubParsersAction) -> None: start.add_argument("--reload", action="store_true", help="Dev reload (uvicorn)") api_sub.add_parser("status", help="Show API settings and provider status") + models = api_sub.add_parser( + "models", help="List the capability-filtered model catalog for the configured provider" + ) + models.add_argument( + "--capability", + default="chat", + choices=["chat", "multimodal", "embedding", "image", "video", "rerank"], + ) + models.add_argument( + "--modality", + default=None, + choices=["image", "audio", "video"], + help="Narrow chat/multimodal models to declared input modalities", + ) + models.add_argument( + "--check", + action="store_true", + help="Exit non-zero unless a live (non-degraded) catalog was returned", + ) api_sub.add_parser("projects", help="List local projects in the API DB") keys = api_sub.add_parser("keys", help="Manage local API keys") @@ -95,6 +114,34 @@ def run_api_command(args: argparse.Namespace) -> int: ) return 0 + if cmd == "models": + provider = get_provider(settings=settings) + catalog_fn = getattr(provider, "catalog", None) + if not callable(catalog_fn): + print( + json.dumps( + { + "error": "catalog_unavailable", + "provider": getattr(provider, "provider_name", "none"), + "hint": "Set AXGUARD_AI_PROVIDER=orcarouter and AXGUARD_AI_API_KEY=sk-orca-…", + }, + indent=2, + ) + ) + return 1 + try: + result = catalog_fn( + capability=getattr(args, "capability", "chat"), + modality=getattr(args, "modality", None), + ) + except ValueError as exc: + print(json.dumps({"error": "validation_error", "message": str(exc)}), file=sys.stderr) + return 2 + print(json.dumps(result, indent=2, default=str)) + if getattr(args, "check", False) and result.get("degraded"): + return 1 + return 0 + if cmd == "projects": store = Store(settings.db_path) try: diff --git a/engines/api/providers/catalog.py b/engines/api/providers/catalog.py new file mode 100644 index 0000000..72dae7c --- /dev/null +++ b/engines/api/providers/catalog.py @@ -0,0 +1,186 @@ +"""Model catalog and capability filters for OpenAI-compatible providers. + +The catalog is the provider's own ``GET {base_url}/models`` listing. Model ids +are kept verbatim (``vendor/model`` namespaces included) and capabilities are +read from directory metadata only — never guessed from model names. + +Endpoints are filtered per capability: + +- ``chat``: models whose ``supported_endpoint_types`` include one of + ``openai`` / ``anthropic`` / ``gemini`` / ``openai-response`` + (or, when the listing carries no endpoint metadata, models without a + non-text generation endpoint — fail closed for image/video/rerank-only). +- ``embedding``: ``embeddings`` endpoint type. +- ``image``: ``image-generation`` endpoint type. +- ``video``: ``openai-video`` endpoint type. +- ``rerank``: ``jina-rerank`` endpoint type. + +Multimodal understanding is not an endpoint type: it is a *chat* model whose +``architecture.input_modalities`` explicitly lists the attachment modality. +Models that do not declare a modality are excluded (fail closed). +""" + +from __future__ import annotations + +import json +import urllib.error +import urllib.request +from typing import Any, Iterable + +CATALOG_TIMEOUT = 60.0 + +CHAT_ENDPOINT_TYPES: frozenset[str] = frozenset( + {"openai", "anthropic", "gemini", "openai-response"} +) +NON_TEXT_ONLY_ENDPOINT_TYPES: frozenset[str] = frozenset( + {"image-generation", "openai-video", "jina-rerank", "embeddings", "moderations", "audio"} +) +CAPABILITY_ENDPOINT_TYPES: dict[str, frozenset[str]] = { + "embedding": frozenset({"embeddings"}), + "image": frozenset({"image-generation"}), + "video": frozenset({"openai-video"}), + "rerank": frozenset({"jina-rerank"}), +} +KNOWN_CAPABILITIES: tuple[str, ...] = ("chat", "multimodal", "embedding", "image", "video", "rerank") +INPUT_MODALITIES: frozenset[str] = frozenset({"image", "audio", "video"}) + + +class CatalogError(RuntimeError): + """Catalog fetch failed (auth, network or malformed payload).""" + + def __init__(self, message: str, *, status: int | None = None) -> None: + super().__init__(message) + self.status = status + + +def _string_list(value: Any) -> list[str]: + if isinstance(value, str): + return [v.strip().lower() for v in value.split(",") if v.strip()] + if isinstance(value, (list, tuple, set)): + out: list[str] = [] + for item in value: + if isinstance(item, dict): + # Directory lists may carry structured entries; the endpoint + # itself is named by ``type``. + item = item.get("type") or item.get("name") or "" + text = str(item).strip().lower() + if text: + out.append(text) + return out + return [] + + +def normalize_model(entry: dict[str, Any]) -> dict[str, Any]: + """Reduce a catalog entry to the metadata the filters need.""" + model_id = str(entry.get("id") or "").strip() + endpoint_types = _string_list(entry.get("supported_endpoint_types")) + architecture = entry.get("architecture") + if not isinstance(architecture, dict): + architecture = {} + modalities = _string_list(architecture.get("input_modalities")) + return { + "id": model_id, + "name": str(entry.get("name") or model_id), + "supported_endpoint_types": endpoint_types, + "input_modalities": modalities, + "context_length": entry.get("context_length") or entry.get("context_window"), + "reasoning": isinstance(entry.get("supported_parameters"), list) + and "reasoning" in [str(p).lower() for p in entry["supported_parameters"]], + } + + +def is_chat_model(model: dict[str, Any]) -> bool: + endpoints = set(model.get("supported_endpoint_types") or []) + if endpoints & CHAT_ENDPOINT_TYPES: + return True + if endpoints & NON_TEXT_ONLY_ENDPOINT_TYPES: + return False + # No text endpoint declared: fail closed unless the entry has no endpoint + # metadata at all (older/proxied catalogs omit it). + return not endpoints + + +def _endpoint_matches(model: dict[str, Any], endpoint_types: frozenset[str], *, fallback: bool) -> bool: + endpoints = set(model.get("supported_endpoint_types") or []) + if endpoints: + return bool(endpoints & endpoint_types) + # No endpoint metadata: only fall back for chat, and only when nothing + # marks the model as a non-text generator. + return fallback and not (endpoints & NON_TEXT_ONLY_ENDPOINT_TYPES) + + +def supports_modality(model: dict[str, Any], modality: str) -> bool: + modality = modality.strip().lower() + if modality not in INPUT_MODALITIES: + raise ValueError(f"unsupported modality: {modality}") + return modality in (model.get("input_modalities") or []) and is_chat_model(model) + + +def filter_models( + models: Iterable[dict[str, Any]], + *, + capability: str | None = None, + modality: str | None = None, +) -> list[dict[str, Any]]: + """Return models compatible with the requested capability. + + ``capability`` must be one of ``KNOWN_CAPABILITIES``. ``modality`` narrows a + ``chat``/``multimodal`` request to models that declare that input modality. + """ + cap = (capability or "chat").strip().lower() + if cap not in KNOWN_CAPABILITIES: + raise ValueError(f"unknown capability: {cap}") + if modality is not None and cap not in {"chat", "multimodal"}: + raise ValueError("modality only applies to chat/multimodal") + + out: list[dict[str, Any]] = [] + for model in models: + if not model.get("id"): + continue + if cap in {"chat", "multimodal"}: + if not is_chat_model(model): + continue + if modality is not None and not supports_modality(model, modality): + continue + elif not _endpoint_matches(model, CAPABILITY_ENDPOINT_TYPES[cap], fallback=False): + continue + out.append(model) + return out + + +def parse_catalog(payload: Any) -> list[dict[str, Any]]: + if isinstance(payload, dict): + data = payload.get("data") + if data is None: + data = payload.get("models") + else: + data = payload + if not isinstance(data, list): + raise CatalogError("model catalog payload is not a list") + return [normalize_model(e) for e in data if isinstance(e, dict)] + + +def fetch_catalog(base_url: str, api_key: str | None, *, timeout: float = CATALOG_TIMEOUT) -> list[dict[str, Any]]: + """Fetch ``GET {base_url}/models`` with a Bearer key. Key never leaves this process.""" + url = f"{base_url.rstrip('/')}/models" + headers = {"Accept": "application/json"} + if api_key: + headers["Authorization"] = f"Bearer {api_key}" + req = urllib.request.Request(url, headers=headers, method="GET") + try: + with urllib.request.urlopen(req, timeout=timeout) as resp: + raw = resp.read().decode("utf-8") + except urllib.error.HTTPError as exc: + if exc.code in {401, 403}: + raise CatalogError( + "model catalog rejected the credential (401/403) — check the OrcaRouter API key", + status=exc.code, + ) from None + raise CatalogError(f"model catalog HTTP {exc.code}", status=exc.code) from None + except urllib.error.URLError as exc: + raise CatalogError(f"model catalog unreachable: {exc.reason}") from None + try: + payload = json.loads(raw) + except json.JSONDecodeError: + raise CatalogError("model catalog returned non-JSON") from None + return parse_catalog(payload) diff --git a/engines/api/providers/factory.py b/engines/api/providers/factory.py index a339794..11154e7 100644 --- a/engines/api/providers/factory.py +++ b/engines/api/providers/factory.py @@ -8,6 +8,7 @@ from engines.api.providers.anthropic_compat import AnthropicCompatProvider from engines.api.providers.none import NoneProvider from engines.api.providers.openai_compat import OpenAICompatProvider +from engines.api.providers.orcarouter import BASE_URL as ORCAROUTER_BASE_URL, OrcaRouterProvider PROVIDER_BASE_URLS: dict[str, str] = { "openai": "https://api.openai.com/v1", @@ -17,6 +18,8 @@ "mistral": "https://api.mistral.ai/v1", "ollama": "http://127.0.0.1:11434/v1", "cerebras": "https://api.cerebras.ai/v1", + # Named OrcaRouter Cyber Inference provider (see engines/api/providers/orcarouter.py). + "orcarouter": ORCAROUTER_BASE_URL, } @@ -67,6 +70,14 @@ def get_provider( else ("configured" if mode_n == "configured" else "user_key") ) + if provider_n in {"orcarouter", "orca"}: + return OrcaRouterProvider( + api_key=api_key, + model=model, + base_url=base_url, + mode=runtime_mode, + ) + if provider_n in {"anthropic", "claude"}: if not api_key: return NoneProvider() @@ -103,7 +114,7 @@ def provider_status(provider: Any | None = None, *, settings: Any | None = None) p = provider if provider is not None else get_provider(settings=settings) mode = getattr(p, "mode", "none") label = "no-llm" if mode == "none" else str(mode) - return { + status = { "llm_mode": label, "provider": getattr(p, "provider_name", getattr(p, "name", type(p).__name__)), "available": bool(getattr(p, "available", False)), @@ -111,6 +122,15 @@ def provider_status(provider: Any | None = None, *, settings: Any | None = None) "cloud": False, "hosted_llm": False, } + display = getattr(p, "display_name", None) + if display: + # Named provider extras — masked hint only, never the raw key. + status["display_name"] = display + status["base_url"] = getattr(p, "base_url", None) + hint_only = getattr(p, "credential_hint", "") + status["credential"] = "configured" if hint_only else "missing" + status["credential_hint"] = hint_only + return status def require_provider_for_enrichment(enrichment: str | None, provider: Any) -> None: diff --git a/engines/api/providers/orcarouter.py b/engines/api/providers/orcarouter.py new file mode 100644 index 0000000..a47b350 --- /dev/null +++ b/engines/api/providers/orcarouter.py @@ -0,0 +1,236 @@ +"""OrcaRouter Cyber Inference — named OrcaRouter provider. + +OrcaRouter is an OpenAI-compatible AI gateway for security teams +(https://www.orcarouter.ai). This module is the single seam for everything +OrcaRouter-specific in AXGuard: base URL, default model, curated cyber tier +metadata, credential redaction and the model catalog. + +OrcaRouter is only reached when the user selects it explicitly +(``AXGUARD_AI_PROVIDER=orcarouter`` / ``ai.provider: orcarouter``); AXGuard has +no hosted model and never silently routes to a third party. +""" + +from __future__ import annotations + +import re + +from engines.api.providers.catalog import CatalogError, fetch_catalog, filter_models +from engines.api.providers.openai_compat import OpenAICompatProvider + +PROVIDER_NAME = "orcarouter" +DISPLAY_NAME = "OrcaRouter Cyber Inference" +BASE_URL = "https://api.orcarouter.ai/v1" +KEY_PREFIX = "sk-orca-" +PROVIDER_PAGE = "https://www.orcarouter.ai/cyber" +CYBER_CATALOG_PAGE = "https://www.orcarouter.ai/models?tier=cyber" + +# Default model: available to any OrcaRouter key, auto-routed. +DEFAULT_MODEL = "orcarouter/auto" +# General fallback for non-cyber workloads. +FALLBACK_MODEL = "orcarouter/fusion" +# Dedicated cyber models for verified security teams (unlocked after business +# verification). Kept as metadata only — never assume a key can call them. +CYBER_MODELS: tuple[dict[str, str], ...] = ( + { + "id": "orcacyber-ultra-1.0", + "name": "OrcaCyber Ultra", + "tier": "cyber", + "description": "Security code scanning and multi-step reasoning over large codebases.", + }, + { + "id": "orcacyber-zero-1.5", + "name": "OrcaCyber Zero 1.5", + "tier": "cyber", + "description": "OrcaCyber Zero harness model (1.5 generation).", + }, + { + "id": "orcacyber-zero-1.0", + "name": "OrcaCyber Zero 1.0", + "tier": "cyber", + "description": "OrcaCyber Zero harness model (1.0 generation).", + }, +) + +# Cold-start seeds used only when a live catalog fetch fails. Small, verified +# metadata only; live discovery always replaces them. +VERIFIED_SEED: tuple[dict[str, object], ...] = ( + { + "id": "orcarouter/auto", + "name": "OrcaRouter Auto", + "supported_endpoint_types": ["openai"], + "input_modalities": ["text"], + "context_length": None, + "reasoning": False, + "verified": True, + }, + { + "id": "orcarouter/fusion", + "name": "OrcaRouter Fusion", + "supported_endpoint_types": ["openai"], + "input_modalities": ["text"], + "context_length": None, + "reasoning": False, + "verified": True, + }, + { + "id": "openai/gpt-5.5", + "name": "GPT-5.5", + "supported_endpoint_types": ["openai", "openai-response"], + "input_modalities": ["text", "image"], + "context_length": None, + "reasoning": True, + "reasoning_effort": ["low", "medium", "high", "xhigh"], + "verified": True, + }, + { + "id": "anthropic/claude-opus-4.8", + "name": "Claude Opus 4.8", + "supported_endpoint_types": ["anthropic", "openai"], + "input_modalities": ["text", "image"], + "context_length": None, + "reasoning": True, + "verified": True, + }, + { + "id": "google/gemini-3.5-flash", + "name": "Gemini 3.5 Flash", + "supported_endpoint_types": ["gemini", "openai"], + "input_modalities": ["text", "image", "audio", "video"], + "context_length": None, + "reasoning": True, + "verified": True, + }, + { + "id": "deepseek/deepseek-v4-pro", + "name": "DeepSeek V4 Pro", + "supported_endpoint_types": ["openai"], + "input_modalities": ["text"], + "context_length": None, + "reasoning": True, + "verified": True, + }, +) + +_KEY_PATTERN = re.compile(r"sk-orca-[A-Za-z0-9_\-]+") +_BEARER_PATTERN = re.compile(r"Bearer\s+\S+", re.IGNORECASE) + + +def is_key_shaped(value: str | None) -> bool: + return bool(value) and value.strip().startswith(KEY_PREFIX) + + +def mask_secret(value: str | None) -> str: + """Return a display-safe hint; never the full credential.""" + if not value: + return "" + v = value.strip() + if len(v) <= 10: + return "…" + v[-2:] + return v[:9] + "…" + v[-2:] + + +def scrub(text: str, secret: str | None = None) -> str: + """Remove credentials from provider error text before it is surfaced or stored.""" + out = text or "" + if secret: + out = out.replace(secret, mask_secret(secret)) + out = _KEY_PATTERN.sub(lambda m: mask_secret(m.group(0)), out) + out = _BEARER_PATTERN.sub("Bearer sk-orca-…", out) + return out + + +def seed_catalog() -> list[dict[str, object]]: + return [dict(m) for m in VERIFIED_SEED] + + +class OrcaRouterProvider(OpenAICompatProvider): + """OpenAI-compatible OrcaRouter provider with a named catalog.""" + + def __init__( + self, + *, + api_key: str | None = None, + model: str | None = None, + base_url: str | None = None, + mode: str = "user_key", + timeout: float = 60.0, + ) -> None: + super().__init__( + base_url=base_url or BASE_URL, + api_key=api_key, + model=model or DEFAULT_MODEL, + mode=mode, + provider_name=PROVIDER_NAME, + timeout=timeout, + ) + + @property + def display_name(self) -> str: + return DISPLAY_NAME + + @property + def credential_hint(self) -> str: + """Masked key for status output (never the raw value).""" + return mask_secret(self.api_key) + + def list_models(self) -> list[dict[str, object]]: + """Live catalog for the configured base URL (requires a key).""" + return fetch_catalog(self.base_url, self.api_key) + + def catalog( + self, + *, + capability: str | None = "chat", + modality: str | None = None, + discovery: bool = True, + ) -> dict[str, object]: + """Capability-filtered catalog with explicit degraded state on failure.""" + source = "live" + error: str | None = None + models: list[dict[str, object]] + if discovery: + try: + models = self.list_models() + except CatalogError as exc: + source = "seed" + error = scrub(str(exc), self.api_key) + models = seed_catalog() + else: + models = seed_catalog() + filtered = filter_models(models, capability=capability, modality=modality) + return { + "provider": PROVIDER_NAME, + "display_name": DISPLAY_NAME, + "catalog_source": source, + "degraded": source != "live", + "error": error, + "capability": capability, + "modality": modality, + "model_count": len(filtered), + "models": filtered, + } + + def complete( + self, + messages: list[dict[str, str]], + *, + model: str | None = None, + temperature: float = 0.0, + max_tokens: int = 1024, + timeout: float | None = None, + ) -> dict[str, object]: + previous = self.timeout + if timeout is not None: + self.timeout = timeout + try: + return super().complete( + messages, model=model, temperature=temperature, max_tokens=max_tokens + ) + except RuntimeError as exc: + raise RuntimeError(scrub(str(exc), self.api_key)) from None + except OSError as exc: # truncated body / reset / timeout on the socket + raise RuntimeError( + scrub(f"OrcaRouter request failed: {type(exc).__name__}", self.api_key) + ) from None + finally: + self.timeout = previous diff --git a/engines/api/routes/__init__.py b/engines/api/routes/__init__.py index 143a124..df6e16b 100644 --- a/engines/api/routes/__init__.py +++ b/engines/api/routes/__init__.py @@ -81,6 +81,35 @@ async def health(): "provider": provider_status(provider, settings=settings), } + @app.get("/v1/models") + async def list_models( + capability: str = "chat", + modality: str | None = None, + ): + """Capability-filtered model catalog for the configured provider. + + Only first-class named providers expose a catalog; the provider key is + held server-side and never returned to the caller. Callers can select + ``capability`` chat|multimodal|embedding|image|video|rerank and, for + chat, an attachment ``modality`` (image|audio|video). On catalog fetch + failure the response is marked ``degraded`` and ``catalog_source`` is + ``seed`` (verified cold-start metadata only). + """ + provider = deps.provider or get_provider(settings=settings) + catalog_fn = getattr(provider, "catalog", None) + if not callable(catalog_fn): + raise ApiError( + "CATALOG_UNAVAILABLE", + "The configured provider does not expose a model catalog.", + status_code=404, + details={"provider": getattr(provider, "provider_name", "none")}, + ) + try: + result = catalog_fn(capability=capability, modality=modality) + except ValueError as exc: + raise ApiError("VALIDATION_ERROR", str(exc), status_code=422) from None + return result + @app.get("/v1/version") async def version(): return {"version": __version__, "api": "v1", "product": "axguard-api"} diff --git a/engines/github/config.py b/engines/github/config.py index 154d63b..d485149 100644 --- a/engines/github/config.py +++ b/engines/github/config.py @@ -33,7 +33,7 @@ class AIConfig: - ``user_key``: user-provided key via env; never from repo source """ - provider: str = "none" # none | local | openai | anthropic | openrouter | ollama | … + provider: str = "none" # none | local | openai | anthropic | openrouter | orcarouter | ollama | … mode: str = "no-llm" # no-llm | local | user_key model: str | None = None # Env var names only — never literal secrets @@ -305,12 +305,22 @@ def resolve_ai_credentials(config: GitHubBotConfig) -> dict[str, str | None]: return {"mode": "no-llm", "api_key": None, "base_url": None, "provider": "none"} api_key = os.environ.get(config.ai.api_key_env) base_url = os.environ.get(config.ai.base_url_env) + provider = (config.ai.provider or "none").lower() + model = config.ai.model + if provider in {"orcarouter", "orca"}: + # Named OrcaRouter Cyber Inference provider: default model and base URL + # are OrcaRouter's; explicit env overrides still win. + from engines.api.providers.orcarouter import BASE_URL, DEFAULT_MODEL, PROVIDER_NAME + + provider = PROVIDER_NAME + base_url = base_url or BASE_URL + model = model or DEFAULT_MODEL return { "mode": mode, "api_key": api_key, "base_url": base_url, - "provider": config.ai.provider, - "model": config.ai.model, + "provider": provider, + "model": model, } diff --git a/tests/test_orcarouter_api.py b/tests/test_orcarouter_api.py new file mode 100644 index 0000000..7736c09 --- /dev/null +++ b/tests/test_orcarouter_api.py @@ -0,0 +1,233 @@ +"""API + GitHub-config wiring tests for OrcaRouter Cyber Inference.""" + +from __future__ import annotations + +import json +import os +import threading +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path + +import pytest + +pytest.importorskip("fastapi") +pytest.importorskip("httpx") + +from fastapi.testclient import TestClient + +from engines.api.app import create_app +from engines.api.settings import load_settings +from engines.github.config import load_github_config, resolve_ai_credentials + +FAKE_KEY = "sk-orca-testfake0000000000000000000000" + +CHAT = {"id": "orcarouter/auto", "supported_endpoint_types": ["openai"], "architecture": {"input_modalities": ["text"]}} +IMAGE_CHAT = { + "id": "openai/gpt-5.5", + "supported_endpoint_types": ["openai", "openai-response"], + "architecture": {"input_modalities": ["text", "image"]}, +} +RERANK = {"id": "jina/jina-reranker-v3", "supported_endpoint_types": ["jina-rerank"]} +IMAGE_GEN = {"id": "openai/gpt-image-1", "supported_endpoint_types": ["image-generation"]} + + +class _Handler(BaseHTTPRequestHandler): + payload: dict = {"data": [CHAT, IMAGE_CHAT, RERANK, IMAGE_GEN]} + requests: list = [] + + def log_message(self, *args): + pass + + def do_GET(self): + type(self).requests.append((self.path, dict(self.headers))) + raw = json.dumps(self.payload).encode("utf-8") + self.send_response(200) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(raw))) + self.end_headers() + self.wfile.write(raw) + + +@pytest.fixture() +def fake_catalog(): + _Handler.requests = [] + httpd = ThreadingHTTPServer(("127.0.0.1", 0), _Handler) + threading.Thread(target=httpd.serve_forever, daemon=True).start() + try: + yield f"http://127.0.0.1:{httpd.server_address[1]}/v1" + finally: + httpd.shutdown() + httpd.server_close() + + +def _client(tmp_path, monkeypatch, base_url, provider="orcarouter", key=FAKE_KEY): + data_dir = tmp_path / "api-data" + monkeypatch.setenv("AXGUARD_API_DATA_DIR", str(data_dir)) + monkeypatch.delenv("AXGUARD_API_REQUIRE_AUTH", raising=False) + monkeypatch.setenv("AXGUARD_AI_MODE", "user_key") + monkeypatch.setenv("AXGUARD_AI_PROVIDER", provider) + monkeypatch.setenv("AXGUARD_AI_MODEL", "orcarouter/auto") + monkeypatch.setenv("AXGUARD_AI_BASE_URL", base_url or "") + if key: + monkeypatch.setenv("AXGUARD_AI_API_KEY", key) + else: + monkeypatch.delenv("AXGUARD_AI_API_KEY", raising=False) + return TestClient(create_app(load_settings(data_dir=data_dir))) + + +def test_models_route_returns_filtered_live_catalog(tmp_path, monkeypatch, fake_catalog): + with _client(tmp_path, monkeypatch, fake_catalog) as client: + resp = client.get("/v1/models", params={"capability": "chat"}) + assert resp.status_code == 200 + body = resp.json() + assert body["provider"] == "orcarouter" + assert body["catalog_source"] == "live" + assert body["degraded"] is False + ids = [m["id"] for m in body["models"]] + assert "orcarouter/auto" in ids + assert "jina/jina-reranker-v3" not in ids + assert "openai/gpt-image-1" not in ids + assert FAKE_KEY not in resp.text + + img = client.get("/v1/models", params={"capability": "chat", "modality": "image"}).json() + assert [m["id"] for m in img["models"]] == ["openai/gpt-5.5"] + + rerank = client.get("/v1/models", params={"capability": "rerank"}).json() + assert [m["id"] for m in rerank["models"]] == ["jina/jina-reranker-v3"] + + path, headers = _Handler.requests[-1] + assert path == "/v1/models" + assert headers.get("Authorization") == f"Bearer {FAKE_KEY}" + + +def test_models_route_validates_capability(tmp_path, monkeypatch, fake_catalog): + with _client(tmp_path, monkeypatch, fake_catalog) as client: + resp = client.get("/v1/models", params={"capability": "omniscient"}) + assert resp.status_code == 422 + assert resp.json()["error"]["code"] == "VALIDATION_ERROR" + + +def test_models_route_404_for_generic_provider(tmp_path, monkeypatch): + with _client(tmp_path, monkeypatch, None, provider="groq", key="gsk_fake") as client: + resp = client.get("/v1/models") + assert resp.status_code == 404 + assert resp.json()["error"]["code"] == "CATALOG_UNAVAILABLE" + assert "gsk_" not in resp.text + + +def test_health_reports_named_provider_without_key(tmp_path, monkeypatch, fake_catalog): + with _client(tmp_path, monkeypatch, fake_catalog) as client: + provider = client.get("/v1/health").json()["provider"] + assert provider["provider"] == "orcarouter" + assert provider["display_name"] == "OrcaRouter Cyber Inference" + assert provider["base_url"] == fake_catalog + assert FAKE_KEY not in json.dumps(provider) + + +def test_llm_enrichment_accepts_named_orcarouter_provider(tmp_path, monkeypatch, fake_catalog): + with _client(tmp_path, monkeypatch, fake_catalog) as client: + target = tmp_path / "proj" + target.mkdir() + (target / "x.py").write_text("x = 1\n", encoding="utf-8") + pid = client.post("/v1/projects", json={"name": "p", "path": str(target)}).json()["id"] + resp = client.post(f"/v1/projects/{pid}/scans", json={"mode": "lite", "enrichment": "llm"}) + # Accepted (202) — the provider is a real BYOK provider, not NoneProvider. + assert resp.status_code == 202, resp.text + + +def test_github_config_orcarouter_defaults(tmp_path, monkeypatch): + cfg_path = tmp_path / ".axguard.yml" + cfg_path.write_text("ai:\n provider: orcarouter\n mode: user_key\n", encoding="utf-8") + monkeypatch.setenv("AXGUARD_AI_API_KEY", FAKE_KEY) + monkeypatch.delenv("AXGUARD_AI_BASE_URL", raising=False) + ai = resolve_ai_credentials(load_github_config(cfg_path)) + assert ai["provider"] == "orcarouter" + assert ai["mode"] == "user_key" + assert ai["base_url"] == "https://api.orcarouter.ai/v1" + assert ai["model"] == "orcarouter/auto" + + +def test_github_config_orcarouter_respects_explicit_overrides(tmp_path, monkeypatch): + cfg_path = tmp_path / ".axguard.yml" + cfg_path.write_text( + "ai:\n provider: orcarouter\n mode: user_key\n model: orcacyber-ultra-1.0\n", + encoding="utf-8", + ) + monkeypatch.setenv("AXGUARD_AI_API_KEY", FAKE_KEY) + monkeypatch.setenv("AXGUARD_AI_BASE_URL", "https://proxy.internal/v1") + ai = resolve_ai_credentials(load_github_config(cfg_path)) + assert ai["model"] == "orcacyber-ultra-1.0" + assert ai["base_url"] == "https://proxy.internal/v1" + + +def test_github_config_never_reads_literal_key_from_repo(tmp_path, monkeypatch): + cfg_path = tmp_path / ".axguard.yml" + cfg_path.write_text( + "ai:\n provider: orcarouter\n mode: user_key\n api_key: sk-orca-should-not-be-read\n", + encoding="utf-8", + ) + monkeypatch.delenv("AXGUARD_AI_API_KEY", raising=False) + ai = resolve_ai_credentials(load_github_config(cfg_path)) + assert ai["api_key"] is None + + +def test_cli_models_command(tmp_path, monkeypatch, fake_catalog, capsys): + monkeypatch.setenv("AXGUARD_API_DATA_DIR", str(tmp_path / "api-data")) + monkeypatch.setenv("AXGUARD_AI_MODE", "user_key") + monkeypatch.setenv("AXGUARD_AI_PROVIDER", "orcarouter") + monkeypatch.setenv("AXGUARD_AI_BASE_URL", fake_catalog) + monkeypatch.setenv("AXGUARD_AI_API_KEY", FAKE_KEY) + rc = _run_api_cli( + ["api", "models", "--capability", "chat", "--modality", "image", "--check"] + ) + out = capsys.readouterr().out + assert rc == 0 + payload = json.loads(out) + assert payload["catalog_source"] == "live" + assert [m["id"] for m in payload["models"]] == ["openai/gpt-5.5"] + assert FAKE_KEY not in out + + +def test_cli_models_check_fails_on_degraded(tmp_path, monkeypatch, capsys): + monkeypatch.setenv("AXGUARD_API_DATA_DIR", str(tmp_path / "api-data")) + monkeypatch.setenv("AXGUARD_AI_MODE", "user_key") + monkeypatch.setenv("AXGUARD_AI_PROVIDER", "orcarouter") + monkeypatch.setenv("AXGUARD_AI_BASE_URL", "http://127.0.0.1:9/v1") + monkeypatch.setenv("AXGUARD_AI_API_KEY", FAKE_KEY) + rc = _run_api_cli(["api", "models", "--check"]) + out = capsys.readouterr().out + assert rc == 1 + payload = json.loads(out) + assert payload["degraded"] is True + assert payload["catalog_source"] == "seed" + assert FAKE_KEY not in out + + +def test_api_parser_exposes_models_subcommand(): + import argparse + + from engines.api.cli import add_api_parser + + parser = argparse.ArgumentParser() + add_api_parser(parser.add_subparsers(dest="command", required=True)) + args = parser.parse_args(["api", "models", "--capability", "image"]) + assert args.command == "api" + assert args.api_command == "models" + assert args.capability == "image" + + +def _run_api_cli(argv): + """Run ``axguard api ...`` through argparse + run_api_command. + + ``cli.main`` (whose parser this subcommand is mounted on) is imported + lazily so on interpreters where an unrelated module in the CLI import + chain only parses under newer Python versions the models path is still + exercised end-to-end. + """ + import argparse + + from engines.api.cli import add_api_parser, run_api_command + + parser = argparse.ArgumentParser(prog="axguard") + add_api_parser(parser.add_subparsers(dest="command", required=True)) + return run_api_command(parser.parse_args(argv)) diff --git a/tests/test_orcarouter_provider.py b/tests/test_orcarouter_provider.py new file mode 100644 index 0000000..1445838 --- /dev/null +++ b/tests/test_orcarouter_provider.py @@ -0,0 +1,443 @@ +"""OrcaRouter Cyber Inference provider: registry, catalog, redaction, live path. + +All network calls go to a local HTTP server except the opt-in live check, which +runs only when ``ORCAROUTER_API_KEY`` is set and uses the provider code path. +""" + +from __future__ import annotations + +import json +import os +import threading +import urllib.error +import urllib.request +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + +import pytest + +from engines.api.providers import catalog as catalog_mod +from engines.api.providers.catalog import ( + CatalogError, + fetch_catalog, + filter_models, + normalize_model, + parse_catalog, +) +from engines.api.providers.factory import get_provider, provider_status +from engines.api.providers.none import NoneProvider +from engines.api.providers.openai_compat import OpenAICompatProvider +from engines.api.providers.orcarouter import ( + BASE_URL, + DEFAULT_MODEL, + OrcaRouterProvider, + mask_secret, + scrub, +) + +FAKE_KEY = "sk-orca-fake000000000000000000000000" + + +def _model(model_id, endpoints=None, modalities=None, **extra): + entry = {"id": model_id, "name": model_id} + if endpoints is not None: + entry["supported_endpoint_types"] = endpoints + if modalities is not None: + entry["architecture"] = {"input_modalities": modalities} + entry.update(extra) + return entry + + +CATALOG_FIXTURE = [ + _model("openai/gpt-5.5", ["openai", "openai-response"], ["text", "image"], + supported_parameters=["reasoning", "temperature"]), + _model("anthropic/claude-opus-4.8", ["anthropic", "openai"], ["text", "image"]), + _model("google/gemini-3.5-flash", ["gemini", "openai"], ["text", "image", "audio", "video"]), + _model("deepseek/deepseek-v4-pro", ["openai"], ["text"]), + _model("orcarouter/auto", ["openai"], ["text"]), + _model("orcarouter/fusion", ["openai"], ["text"]), + _model("orcacyber-ultra-1.0", ["openai"], ["text"]), + _model("meta/llama-text-nodecl", ["openai"]), + _model("openai/gpt-image-1", ["image-generation"], ["text"]), + _model("kuaishou/kling-video", ["openai-video"], ["text"]), + _model("jina/jina-reranker-v3", ["jina-rerank"], ["text"]), + _model("openai/text-embedding-4", ["embeddings"], ["text"]), + _model("vendor/audio-tts", ["audio"], ["text"]), + _model("vendor/system-only", ["systemone"], ["text", "image"]), + _model("vendor/dict-endpoints", [{"type": "openai"}, {"type": "anthropic"}], ["text"]), +] + + +class _Handler(BaseHTTPRequestHandler): + """Records requests; serves the fixture catalog and a fake completion.""" + + payload: dict = {"data": CATALOG_FIXTURE} + status: int = 200 + requests: list = [] + chat_response: dict = { + "id": "chatcmpl-test", + "choices": [{"message": {"content": "summary ok"}}], + "usage": {"prompt_tokens": 1, "completion_tokens": 1}, + } + chat_status: int = 200 + + def log_message(self, *args): # silence + pass + + def _send(self, status, body): + raw = json.dumps(body).encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(raw))) + self.end_headers() + self.wfile.write(raw) + + def do_GET(self): + type(self).requests.append(("GET", self.path, dict(self.headers))) + self._send(self.status, self.payload) + + def do_POST(self): + length = int(self.headers.get("Content-Length") or 0) + body = self.rfile.read(length).decode("utf-8") + type(self).requests.append(("POST", self.path, dict(self.headers), body)) + self._send(self.chat_status, self.chat_response) + + +@pytest.fixture() +def server(): + _Handler.requests = [] + _Handler.payload = {"data": CATALOG_FIXTURE} + _Handler.status = 200 + _Handler.chat_status = 200 + httpd = ThreadingHTTPServer(("127.0.0.1", 0), _Handler) + thread = threading.Thread(target=httpd.serve_forever, daemon=True) + thread.start() + try: + yield f"http://127.0.0.1:{httpd.server_address[1]}/v1" + finally: + httpd.shutdown() + httpd.server_close() + + +# --- registry --------------------------------------------------------------- + + +def test_factory_returns_named_orcarouter_provider(): + p = get_provider(mode="user_key", provider="orcarouter", api_key=FAKE_KEY) + assert isinstance(p, OrcaRouterProvider) + assert isinstance(p, OpenAICompatProvider) + assert p.provider_name == "orcarouter" + assert p.base_url == BASE_URL == "https://api.orcarouter.ai/v1" + assert p.default_model == DEFAULT_MODEL == "orcarouter/auto" + assert p.display_name == "OrcaRouter Cyber Inference" + + +def test_factory_orcarouter_defaults_without_model(): + p = get_provider(mode="user_key", provider="orca", api_key=FAKE_KEY) + assert isinstance(p, OrcaRouterProvider) + assert p.provider_name == "orcarouter" + + +def test_factory_orcarouter_without_key_is_unavailable(): + p = get_provider( + mode="user_key", provider="orcarouter", api_key=None, base_url=None + ) + assert isinstance(p, OrcaRouterProvider) + assert p.available is False + + +def test_factory_explicit_base_url_override_wins(): + p = get_provider( + mode="user_key", + provider="orcarouter", + api_key=FAKE_KEY, + base_url="http://127.0.0.1:9/v1", + ) + assert p.base_url == "http://127.0.0.1:9/v1" + + +def test_status_masks_credential(monkeypatch): + p = get_provider(mode="user_key", provider="orcarouter", api_key=FAKE_KEY) + status = provider_status(p) + blob = json.dumps(status) + assert FAKE_KEY not in blob + assert status["provider"] == "orcarouter" + assert status["display_name"] == "OrcaRouter Cyber Inference" + assert status["credential"] == "configured" + assert status["credential_hint"].startswith("sk-orca-") + + +def test_none_provider_status_unchanged(): + status = provider_status(NoneProvider()) + assert status["provider"] == "none" + assert "display_name" not in status + + +def test_scrub_removes_keys_from_error_text(): + text = f"LLM provider HTTP 401: Invalid key {FAKE_KEY} (Bearer {FAKE_KEY})" + out = scrub(text, FAKE_KEY) + assert FAKE_KEY not in out + assert "sk-orca-" in out + assert FAKE_KEY not in scrub(f"leaked {FAKE_KEY}") # no explicit secret arg + + +def test_mask_secret_short_values(): + assert mask_secret(None) == "" + assert "sk-orca-x" not in mask_secret("sk-orca-x") + assert mask_secret(FAKE_KEY).endswith("00") + + +# --- catalog parsing and capability filters -------------------------------- + + +def test_parse_catalog_keeps_vendor_namespace_and_metadata(): + models = parse_catalog({"data": CATALOG_FIXTURE}) + ids = [m["id"] for m in models] + assert "openai/gpt-5.5" in ids + gpt = next(m for m in models if m["id"] == "openai/gpt-5.5") + assert gpt["input_modalities"] == ["text", "image"] + assert gpt["reasoning"] is True + assert "supported_endpoint_types" in models[0] + + +def test_parse_catalog_rejects_garbage(): + with pytest.raises(CatalogError): + parse_catalog({"data": "nope"}) + with pytest.raises(CatalogError): + parse_catalog(42) + + +def test_normalize_tolerates_missing_metadata(): + m = normalize_model({"id": "x/y"}) + assert m["input_modalities"] == [] + assert m["supported_endpoint_types"] == [] + assert m["reasoning"] is False + + +def _ids(models): + return {m["id"] for m in models} + + +def test_chat_filter_excludes_image_video_rerank_embedding_audio(): + models = parse_catalog({"data": CATALOG_FIXTURE}) + chat = _ids(filter_models(models, capability="chat")) + assert "openai/gpt-5.5" in chat + assert "anthropic/claude-opus-4.8" in chat + assert "google/gemini-3.5-flash" in chat + assert "deepseek/deepseek-v4-pro" in chat + assert "orcarouter/auto" in chat + assert "orcacyber-ultra-1.0" in chat + # Missing endpoint metadata: allowed only if it is not a non-text generator + assert "meta/llama-text-nodecl" in chat + assert "openai/gpt-image-1" not in chat + assert "kuaishou/kling-video" not in chat + assert "jina/jina-reranker-v3" not in chat + assert "openai/text-embedding-4" not in chat + assert "vendor/audio-tts" not in chat + # Unknown endpoint types are not chat; structured endpoint entries still are. + assert "vendor/system-only" not in chat + assert "vendor/dict-endpoints" in chat + + +def test_multimodal_filter_fails_closed_without_declared_modality(): + models = parse_catalog({"data": CATALOG_FIXTURE}) + image = _ids(filter_models(models, capability="chat", modality="image")) + assert "openai/gpt-5.5" in image + assert "anthropic/claude-opus-4.8" in image + assert "google/gemini-3.5-flash" in image + # text-only and undeclared-modality chat models are excluded + assert "deepseek/deepseek-v4-pro" not in image + assert "meta/llama-text-nodecl" not in image + assert "vendor/system-only" not in image + assert "orcarouter/auto" not in image + + audio = _ids(filter_models(models, capability="chat", modality="audio")) + assert audio == {"google/gemini-3.5-flash"} + video = _ids(filter_models(models, capability="chat", modality="video")) + assert video == {"google/gemini-3.5-flash"} + + +def test_capability_specific_filters(): + models = parse_catalog({"data": CATALOG_FIXTURE}) + assert _ids(filter_models(models, capability="embedding")) == {"openai/text-embedding-4"} + assert _ids(filter_models(models, capability="image")) == {"openai/gpt-image-1"} + assert _ids(filter_models(models, capability="video")) == {"kuaishou/kling-video"} + assert _ids(filter_models(models, capability="rerank")) == {"jina/jina-reranker-v3"} + + +def test_filter_rejects_unknown_capability_and_modality(): + models = parse_catalog({"data": CATALOG_FIXTURE}) + with pytest.raises(ValueError): + filter_models(models, capability="omniscient") + with pytest.raises(ValueError): + filter_models(models, capability="chat", modality="smell") + with pytest.raises(ValueError): + filter_models(models, capability="embedding", modality="image") + + +# --- live-ish catalog via local server ------------------------------------- + + +def test_list_models_uses_get_v1_models_with_bearer(server): + p = OrcaRouterProvider(api_key=FAKE_KEY, base_url=server) + models = p.list_models() + assert len(models) == len(CATALOG_FIXTURE) + method, path, headers = _Handler.requests[-1] + assert method == "GET" + assert path == "/v1/models" + assert headers.get("Authorization") == f"Bearer {FAKE_KEY}" + + +def test_catalog_filters_live_result_and_never_mixes_seed(server): + p = OrcaRouterProvider(api_key=FAKE_KEY, base_url=server) + result = p.catalog(capability="chat", modality="image") + assert result["catalog_source"] == "live" + assert result["degraded"] is False + ids = {m["id"] for m in result["models"]} + assert ids == {"openai/gpt-5.5", "anthropic/claude-opus-4.8", "google/gemini-3.5-flash"} + # seed-only models such as deepseek/... must never be merged into live output + assert "deepseek/deepseek-v4-pro" not in ids + + +def test_catalog_auth_error_degrades_to_verified_seed(server): + _Handler.status = 401 + p = OrcaRouterProvider(api_key=FAKE_KEY, base_url=server) + result = p.catalog(capability="chat") + assert result["degraded"] is True + assert result["catalog_source"] == "seed" + assert "401" in result["error"] + assert FAKE_KEY not in json.dumps(result) + ids = {m["id"] for m in result["models"]} + assert {"orcarouter/auto", "orcarouter/fusion", "openai/gpt-5.5"} <= ids + assert "openai/gpt-image-1" not in ids + + +def test_catalog_network_error_degrades(monkeypatch): + def boom(*a, **k): + raise urllib.error.URLError("connection refused") + + monkeypatch.setattr(catalog_mod.urllib.request, "urlopen", boom) + result = OrcaRouterProvider(api_key=FAKE_KEY).catalog(capability="chat", modality="image") + assert result["degraded"] is True + assert "unreachable" in result["error"] + ids = {m["id"] for m in result["models"]} + # verified seed: only models declaring image input survive + assert "openai/gpt-5.5" in ids + assert "deepseek/deepseek-v4-pro" not in ids + + +def test_catalog_empty_catalog_degraded_state(server): + _Handler.payload = {"data": []} + result = OrcaRouterProvider(api_key=FAKE_KEY, base_url=server).catalog(capability="chat") + assert result["catalog_source"] == "live" + assert result["model_count"] == 0 + assert result["models"] == [] + + +def test_fetch_catalog_non_json(server): + _Handler.payload = ["not-a-dict"] + models = fetch_catalog(server, FAKE_KEY) + assert models == [] + + +def test_seed_is_small_and_verified_only(): + result = OrcaRouterProvider(api_key=FAKE_KEY).catalog(discovery=False) + ids = [m["id"] for m in result["models"]] + assert len(ids) <= 6 + for seed in result["models"]: + assert seed["verified"] is True + assert seed["supported_endpoint_types"] + gpt = next(m for m in result["models"] if m["id"] == "openai/gpt-5.5") + assert gpt["reasoning_effort"] == ["low", "medium", "high", "xhigh"] + + +# --- completion path -------------------------------------------------------- + +def test_complete_wraps_socket_errors_without_leaking(monkeypatch): + def truncated(*a, **k): + raise TimeoutError("The read operation timed out") + + monkeypatch.setattr(catalog_mod.urllib.request, "urlopen", truncated) + p = OrcaRouterProvider(api_key=FAKE_KEY) + with pytest.raises(RuntimeError) as ei: + p.complete([{"role": "user", "content": "x"}]) + assert "TimeoutError" in str(ei.value) + assert FAKE_KEY not in str(ei.value) + assert p.timeout == 60.0 + + + +def test_complete_posts_chat_completions_with_default_model(server): + p = OrcaRouterProvider(api_key=FAKE_KEY, base_url=server) + out = p.complete([{"role": "user", "content": "hi"}]) + assert out["content"] == "summary ok" + assert out["model"] == "orcarouter/auto" + method, path, headers, body = _Handler.requests[-1] + assert method == "POST" + assert path == "/v1/chat/completions" + assert headers.get("Authorization") == f"Bearer {FAKE_KEY}" + assert json.loads(body)["model"] == "orcarouter/auto" + + +def test_complete_explicit_cyber_model(server): + p = OrcaRouterProvider(api_key=FAKE_KEY, base_url=server) + p.complete([{"role": "user", "content": "scan"}], model="orcacyber-ultra-1.0") + _, _, _, body = _Handler.requests[-1] + assert json.loads(body)["model"] == "orcacyber-ultra-1.0" + + +def test_complete_error_text_is_redacted(server): + _Handler.chat_status = 401 + _Handler.chat_response = {"error": f"invalid key {FAKE_KEY}"} + p = OrcaRouterProvider(api_key=FAKE_KEY, base_url=server) + with pytest.raises(RuntimeError) as ei: + p.complete([{"role": "user", "content": "x"}]) + assert "401" in str(ei.value) + assert FAKE_KEY not in str(ei.value) + + +def test_complete_without_key_raises(): + p = OrcaRouterProvider(api_key=None) + with pytest.raises(RuntimeError): + p.complete([{"role": "user", "content": "x"}]) + + +# --- real live check (opt-in) ---------------------------------------------- + + +@pytest.mark.skipif( + not (os.environ.get("ORCAROUTER_API_KEY") or "").startswith("sk-orca-"), + reason="ORCAROUTER_API_KEY not set", +) +def test_live_orcarouter_catalog_and_completion(): + """Real request through the implemented provider against api.orcarouter.ai.""" + key = os.environ["ORCAROUTER_API_KEY"] + p = get_provider(mode="user_key", provider="orcarouter", api_key=key) + assert isinstance(p, OrcaRouterProvider) + + cat = p.catalog(capability="chat") + assert cat["catalog_source"] == "live", cat.get("error") + assert cat["model_count"] > 0 + ids = {m["id"] for m in cat["models"]} + # Catalog entries carry directory metadata: chat-capable for the text + # dropdown, image input declared for the image dropdown. + for model in cat["models"]: + assert set(model["supported_endpoint_types"]) & catalog_mod.CHAT_ENDPOINT_TYPES + assert not (ids & {"jina/jina-reranker-v3", "openai/gpt-image-1", "kuaishou/kling-video"}) + image_cat = p.catalog(capability="chat", modality="image") + assert image_cat["catalog_source"] == "live", image_cat.get("error") + for model in image_cat["models"]: + assert "image" in model["input_modalities"] + assert set(model["supported_endpoint_types"]) & catalog_mod.CHAT_ENDPOINT_TYPES + + # Prefer cyber tier when the verification key is unlocked; otherwise use a + # public model this key can actually call to prove the wiring. + model = "deepseek/deepseek-v4-flash" if "deepseek/deepseek-v4-flash" in ids else next(iter(sorted(ids))) + out = p.complete( + [{"role": "user", "content": "Reply with the single word: ok"}], + model=model, + max_tokens=16, + timeout=120.0, + ) + assert out["provider"] == "orcarouter" + assert isinstance(out["content"], str) + assert key not in json.dumps(out)