Skip to content

Llm proxy - #54

Open
iidsample wants to merge 1 commit into
mainfrom
llm_proxy
Open

Llm proxy#54
iidsample wants to merge 1 commit into
mainfrom
llm_proxy

Conversation

@iidsample

Copy link
Copy Markdown
Member

Added LLM Proxy for replacing OpenAI, Anthropic and Bedrock calls.

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 576400bc-c38d-4402-9269-2e0818cfdea5


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Comment thread llm_proxy/app.py
return proxy_request(prov, subpath, request)
except Exception as exc: # surface upstream/adapter errors as 502
log.exception("proxy error for %s/%s", provider, subpath)
return jsonify(error="proxy_error", detail=str(exc)), 502
Comment thread llm_proxy/README.md
- **OpenAI / Anthropic** — straight HTTP reverse-proxy: rewrite host, swap in the
real key, forward with `requests`, return the response.
- **Bedrock** — re-issued through the proxy's own `boto3` client (handles SigV4
signing + URL-encoding correctly). Only `invoke` is wired up.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No .converse functionality? Would we be changing all the current calls to use .invoke?

@nickhuo

nickhuo commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

i'm thinking about how to adapt the llm proxy to the agent skill, some questions:

  • skill adaptation happens before the Ventis build, but it seems the base URL can only be obtained after deployment, so the agent cannot fill in the base URL for the user, while the controller set it up. Is my understanding correct?
  • if it's right, the agent skill can identify if this project can use this version of llm proxy (bedrock, openai etc)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants