Skip to content

[Feature Request] Multi model support #21

Closed
Closed
[Feature Request] Multi model support#21
Assignees
bcurts

Description

@omsenjalia
omsenjalia
opened on Mar 2, 2026

🤖 Feature Request: Multi-Model Support (Ollama + Cloud Models)
Problem
agentchattr is locked to Claude Code, Codex, and Gemini CLI. No way to use local models or mix models by task.
Proposed solution
Add Ollama and OpenAI-compatible endpoint support as a new agent type in config.toml:

[agents.mistral]
type = "ollama"
model = "mistral-nemo"
ollama_url = "http://localhost:11434"
color = "#f97316"
label = "Mistral"
roles = ["review", "summarize"]

[agents.qwen]
type = "openai_compat"
base_url = "https://api.ollama.ai/v1"
api_key_env = "OLLAMA_CLOUD_KEY"
model = "qwen2.5-coder:32b"
roles = ["codegen", "refactor"]

These agents join the same chat room, respond to @mentions, and participate in agentic loops like any other agent — no CLI subprocess needed, just a polling wrapper calling the model API.
Why it's useful

Route cheap/fast tasks (triage, summaries) to a local 3B model instead of burning cloud tokens
Keep sensitive code fully local via Ollama
Assign models by role: one for codegen, one for review, one for routing
Works in air-gapped environments

Implementation surface
Mostly contained to agents.py (new OllamaAgent class) and a new wrapper_api.py for the trigger loop. config.toml schema would need type, model, base_url, api_key_env, and optionally roles. MCP tools are already model-agnostic so mcp_bridge.py wouldn't need changes.

Activity

  1. bcurts commented on Mar 2, 2026

    @bcurts
    bcurts
    on Mar 2, 2026
    Owner

    Thanks - I will look into this just as soon as multi-agent sessions (multiple claude's/codex/geminis etc) is shipped with a few more ux tweaks that are already working.

  2. ghostreindeer09 commented on Mar 2, 2026

    @ghostreindeer09
    ghostreindeer09
    on Mar 2, 2026

    Hey! Happy to help out here. Before we bring in new agents like Ollama or Qwen, I can put together a small PR to introduce a BaseAgent interface.
    It’d refactor the current agents (Claude, Codex, Gemini) so they all follow the same standard interface (send, receive, name) without changing how they work.
    This way, adding new agents later will be way easier and safer.
    Does that sound like a good first step?

  3. bcurts commented on Mar 3, 2026

    @bcurts
    bcurts
    on Mar 3, 2026
    Owner

    Thanks for the offer! We actually went ahead and built this a bit differently than a BaseAgent refactor.

    The existing CLI agents (Claude, Codex, Gemini) each have their own terminal wrappers with tmux/conpty integration, keystroke injection, process management, etc. They're fundamentally different from API-only models, so a shared interface wouldn't buy much — it'd be an abstraction over things that don't actually share behavior.

    Instead we built wrapper_api.py — a lightweight bridge that connects any OpenAI-compatible model endpoint to the chat room. It:

    Registers with the server (gets a status pill, activity tracking, multi-instance support — all the same infrastructure the CLI agents use)

    • Watches for @mentions via the existing queue system
    • Reads recent chat context, formats it into OpenAI messages, calls the model's /v1/chat/completions, and posts the response back

    To add a local model, you just create a config.local.toml (gitignored) with a few lines:

    [agents.qwen]
type = "api"
base_url = "[http://localhost:8189/v1"](http://localhost:8189/v1%22)
model = "qwen3-4b"
label = "Qwen"

    Then run python wrapper_api.py qwen (or double-click start_api_agent.bat). Works with Ollama, llama-server, LM Studio, vLLM — anything that exposes the OpenAI-compatible API.

    Already tested with Qwen3-4B locally and it's working, including multi-instance support (qwen, qwen-2, etc.).

    Shipping it now!

    Image
  4. bcurts
    self-assigned this
    on Mar 3, 2026

  5. omsenjalia commented on Mar 3, 2026

    @omsenjalia
    omsenjalia
    on Mar 3, 2026
    Author

    Thanks for the offer! We actually went ahead and built this a bit differently than a BaseAgent refactor.

    The existing CLI agents (Claude, Codex, Gemini) each have their own terminal wrappers with tmux/conpty integration, keystroke injection, process management, etc. They're fundamentally different from API-only models, so a shared interface wouldn't buy much — it'd be an abstraction over things that don't actually share behavior.

    Instead we built wrapper_api.py — a lightweight bridge that connects any OpenAI-compatible model endpoint to the chat room. It:

    Registers with the server (gets a status pill, activity tracking, multi-instance support — all the same infrastructure the CLI agents use)

    * Watches for @mentions via the existing queue system

* Reads recent chat context, formats it into OpenAI messages, calls the model's /v1/chat/completions, and posts the response back

    To add a local model, you just create a config.local.toml (gitignored) with a few lines:

    [agents.qwen]
type = "api"
base_url = "[http://localhost:8189/v1"](http://localhost:8189/v1%22)
model = "qwen3-4b"
label = "Qwen"

    Then run python wrapper_api.py qwen (or double-click start_api_agent.bat). Works with Ollama, llama-server, LM Studio, vLLM — anything that exposes the OpenAI-compatible API.

    Already tested with Qwen3-4B locally and it's working, including multi-instance support (qwen, qwen-2, etc.).

    Shipping it now!
    Image

    Another suggestion: if possible in the ui give us option to make an agent (people who are not good with editing configs can create agents from the website). And another thing. In the config file add an option "context window". As not all ollama models have the same context window. (Default is like 128K)

  6. omsenjalia commented on Mar 3, 2026

    @omsenjalia
    omsenjalia
    on Mar 3, 2026
    Author

    And if you can create cron loops (the AI agents work togather to create an app or an feature togather everyday at a specific time) it would be great...

  7. omsenjalia commented on Mar 3, 2026

    @omsenjalia
    omsenjalia
    on Mar 3, 2026
    Author

    A freind of mine has a project like the one I suggested above. Check it out! https://github.com/ikislay/auto-claude

  8. bcurts commented on Mar 3, 2026

    @bcurts
    bcurts
    on Mar 3, 2026
    Owner

    Both good ideas! For now:

    UI agent creation: on the radar for a future update. Right now it's a 5-line config file which keeps things simple and predictable. PRs welcome if someone wants to tackle it.

    Context windown: the wrapper sends a fixed window of recent chat messages (configurable in local toml), so context limits aren't really a factor. The OpenAI-compatible API handles context sizing on the model side.

  9. omsenjalia commented on Mar 3, 2026

    @omsenjalia
    omsenjalia
    on Mar 3, 2026
    Author

    Both good ideas! For now:

    UI agent creation: on the radar for a future update. Right now it's a 5-line config file which keeps things simple and predictable. PRs welcome if someone wants to tackle it.

    Context windown: the wrapper sends a fixed window of recent chat messages (configurable in local toml), so context limits aren't really a factor. The OpenAI-compatible API handles context sizing on the model side.

    Okay. 1 more thing... How to automatically generated those ai hat icons for the new added ai agents?

  10. bcurts commented on Mar 3, 2026

    @bcurts
    bcurts
    on Mar 3, 2026
    Owner

    Re: the scheduler good idea! I'll add it to the roadmap :)

  11. omsenjalia commented on Mar 3, 2026

    @omsenjalia
    omsenjalia
    on Mar 3, 2026
    Author

    Re: the scheduler good idea! I'll add it to the roadmap :)

    Another thing. Generate a to do board/kanban dashboard with this same themeing schemes so the AI models can keep up with tasks. And another thing, add a like "enabled = true" so we can enable to disable the ai model. For example I don't have access to claude or codex (free tier) so I don't enable then and only use ollama/open router free models.

    In the kanban dashboard you can let the ai model list it's task to it's section and move it to completed or to be done by itself.

  12. omsenjalia commented on Mar 3, 2026

    @omsenjalia
    omsenjalia
    on Mar 3, 2026
    Author

    We still need image support btw.

  13. omsenjalia commented on Mar 3, 2026

    @omsenjalia
    omsenjalia
    on Mar 3, 2026
    Author

    And a chat history button

  14. omsenjalia commented on Mar 3, 2026

    @omsenjalia
    omsenjalia
    on Mar 3, 2026
    Author

    And you know that claude code has that esc button shortcut that lets us move back (for example you wanna go the start of the convo without starting another chat). Maybe do something like this.

  15. bcurts commented on Mar 3, 2026

    @bcurts
    bcurts
    on Mar 3, 2026
    Owner

    Re: the scheduler good idea! I'll add it to the roadmap :)

    Another thing. Generate a to do board/kanban dashboard with this same themeing schemes so the AI models can keep up with tasks. And another thing, add a like "enabled = true" so we can enable to disable the ai model. For example I don't have access to claude or codex (free tier) so I don't enable then and only use ollama/open router free models.

    In the kanban dashboard you can let the ai model list it's task to it's section and move it to completed or to be done by itself.

    In the current version you don't see agents that aren't actively running.

  16. bcurts commented on Mar 3, 2026

    @bcurts
    bcurts
    on Mar 3, 2026
    Owner

    Both good ideas! For now:
    UI agent creation: on the radar for a future update. Right now it's a 5-line config file which keeps things simple and predictable. PRs welcome if someone wants to tackle it.
    Context windown: the wrapper sends a fixed window of recent chat messages (configurable in local toml), so context limits aren't really a factor. The OpenAI-compatible API handles context sizing on the model side.

    Okay. 1 more thing... How to automatically generated those ai hat icons for the new added ai agents?

    I just made it so agents can create hats for other agents, so just ask claude or codex or gemini to make one for your local agent :)

  17. bcurts commented on Mar 3, 2026

    @bcurts
    bcurts
    on Mar 3, 2026
    Owner

    We still need image support btw.

    We already have image sharing in the web UI (upload/paste/drag). I assume you mean making API agents see images? That's trickier:

    The wrapper would need to fetch image attachments from chat messages, then convert them to base64 and include in the model's messages array as image_url content blocks. This only works if the model supports vision.

    It's doable but not trivial, and most small local models don't have vision anyway. I'll consider that for the future though!

  18. bcurts commented on Mar 3, 2026

    @bcurts
    bcurts
    on Mar 3, 2026
    Owner

    Re: the scheduler good idea! I'll add it to the roadmap :)

    Another thing. Generate a to do board/kanban dashboard with this same themeing schemes so the AI models can keep up with tasks. And another thing, add a like "enabled = true" so we can enable to disable the ai model. For example I don't have access to claude or codex (free tier) so I don't enable then and only use ollama/open router free models.

    In the kanban dashboard you can let the ai model list it's task to it's section and move it to completed or to be done by itself.

    I really don't want this to become like JIRA for agents tbh, I have enough of that at work! So a kanban probably isn't the direction I will go in - but I do have plans to experiment with a 'swarm' mode that will let you do a focused coordinated effort on a bigger piece of work, if it works out well that will likely be the next major feature.

  19. omsenjalia commented on Mar 3, 2026

    @omsenjalia
    omsenjalia
    on Mar 3, 2026
    Author

    Okay thanks 👍

  20. Repository owner locked and limited conversation to collaborators on Mar 3, 2026
  21. bcurts
    converted this issue into a discussion #23 on Mar 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions