Vane Proxy

A FastAPI-based proxy for LLM API calls designed to clean and normalize outputs from models like LM Studio. It strips reasoning/thinking tags from model responses while preserving the actual content.

Version: 1.0.5
Status: Stable

Features

• Chat Completions Proxy — Full OpenAI-compatible /v1/chat/completions endpoint with streaming support
• Reasoning Tag Removal — Automatically removes <reasoning>, <thinking>, and <thought> tags (case-insensitive, multiline) from model outputs
• Markdown Code Block Cleanup — Strips markdown code fences that break JSON parsers in streaming responses
• Smart Streaming Buffering — Accumulates raw content during reasoning tag blocks, then yields only clean tokens once actual output appears
• Reasoning Content Promotion — If main content is empty but reasoning_content has output, promotes it to the primary response field
• Embeddings Passthrough — Raw /v1/embeddings endpoint with no transformation
• Models Listing — /v1/models endpoint reflecting upstream model catalog
• Health Check — /health endpoint for load balancers and monitoring (includes version and timestamp)

Installation

Prerequisites

• Python 3.10+
• Docker & Docker Compose (recommended)

Using Docker Compose

docker compose up --build -d

The proxy will be available at http://localhost:9090.

Configuration

Environment Variable	Default Value	Description
LM_STUDIO_URL	http://host.docker.internal:1234/v1	Upstream LLM API endpoint
SOURCE_DATE_EPOCH	Current timestamp	Fixed epoch for reproducible builds (optional)

API Endpoints

Chat Completions

curl -X POST http://localhost:9090/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "What is 2+2?"}
    ],
    "stream": false
  }'

Streaming Chat Completions

curl -X POST http://localhost:9090/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "Write a poem about cats."}
    ],
    "stream": true
  }'

Embeddings

curl -X POST http://localhost:9090/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "text-embedding-3-large",
    "input": "Hello world"
  }'

Models

curl http://localhost:9090/v1/models

Health Check

curl http://localhost:9090/health
# Returns:
# {"status": "ok", "version": "1.0.5", "timestamp": "..."}

Cleaning Logic Details

The proxy applies the following transformations in order:

1. Tag Removal — Strips all <reasoning>, <thinking>, and <thought> tags (case-insensitive, handles multiline content)
2. Markdown Fence Removal — Removes leading/trailing json or blocks that may appear in streaming responses
3. Whitespace Normalization — Trims leading/trailing whitespace from final output

Development

# Install dependencies
pip install fastapi httpx uvicorn[standard]

# Run locally
uvicorn proxy:app --host 0.0.0.0 --port 8080

Version History

v1.0.5 (2026-05)

--

v1.0.4 (2026-05)

• Added structured logging: Replaced print() statements with proper Python logging module for production-ready diagnostics
• Improved error handling: Added validation and graceful degradation on malformed JSON requests
• Enhanced observability: All errors, warnings, and info events now include timestamps and log levels

v1.0.3

• Fixed /v1/embeddings passthrough: Switched to raw request/response forwarding to prevent local JSON parsing errors (400 Bad Request) and ensure data integrity for large payloads.
• Updated documentation to reflect raw passthrough behavior.

v1.0.2

• Increased request timeouts from 60s to 300s (5 minutes) to prevent ReadTimeout errors with slower local models or large context windows
• Improved stream error logging to display actual exception types for easier debugging

v1.0.1

• Added passthrough support for /v1/embeddings endpoint (embedding models)

v1.0.0 (Initial Release)

• Core proxy functionality with tag removal
• Streaming and non-streaming support
• Smart buffering for streaming responses
• Content promotion from reasoning_content
• Docker containerization with compose setup

License

MIT