LLM_OPENAI_COMPATIBLE.md

LLM configuration (OpenAI-compatible)

CreditAI talks to models only via OpenAI-compatible:

POST {Llm:BaseUrl}/v1/chat/completions

Compatible with any OpenAI-compatible inference server (vLLM, llama.cpp server, TensorRT-LLM HTTP, enterprise gateways) and optional ContextMemory.

Config

"Llm": {
  "Enabled": true,
  "BaseUrl": "http://localhost:8000",
  "Model": "qwen3.5:9b",
  "Temperature": 0.1,
  "ThinkEnabled": false,
  "MaxGroundingRetries": 1
}

BaseUrl is the host without /v1. Env: LLM_ENABLED, LLM_BASE_URL, LLM_MODEL.

Optional ContextMemory:

"ContextMemory": {
  "BaseUrl": "http://localhost:5100",
  "AppId": "creditai",
  "ApiKey": "cm_creditai_local_key"
}

docker compose -f docker-compose.yml -f docker-compose.contextmemory.yml up -d

Smoke tests

curl -s http://localhost:8000/v1/models
curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3.5:9b","stream":false,"messages":[{"role":"user","content":"ping"}]}'

Client

• OpenAiCompatibleNarrativeLlmClient — narrative + document parsing
• JSON mode: response_format: { "type": "json_object" }
• If Llm:Enabled=false and ContextMemory is unset → deterministic narrative fallback

See also: integrations.md · compliance/narrative-ai-policy.md