A modular LLM chatbot UI for OPACA platforms with multi-agent orchestration and tool calling.

❓ What is SAGE?

SAGE, also referred to as the 'OPACA LLM UI', is a powerful chatbot that can fulfill user requests by calling actions from a connected OPACA platform. It can be easily extended with additional agents, developed in any programming language. Multiple reference implementations can be found in the opaca-example-containers repository. Further, the repository for our python package opaca includes a detailed step-by-step guide on how to develop and deploy new agents very easily using Python.

🚀 Quickstart

Docker (recommended)

1. Create a new .env file by copying the example

cp .env.example .env

2. Configure (.env)

URL="http://<YOUR_IP>"                     # For example "http://192.168.1.100"
PUBLIC_URL="${URL}:8000"
VITE_PLATFORM_URL="${PUBLIC_URL}"
VITE_BACKEND_URL="${URL}:3001"
CORS_WHITELIST="${URL}:5173"

3. Start SAGE using Docker

docker compose --profile platform up --build

The --profile platform flag will automatically start and configure an OPACA platform SAGE can connect to.

Native

Backend

# Change into the backend directory
cd Backend

# Install dependencies
pip install -r requirements.txt

# Start the server
python -m src.server

Frontend

# Change into the frontend directory
cd Frontend

# Install dependencies
npm install

# Start the frontend
npm run host

OPACA Platform

Please refer to the OPACA documentation for more information on how to set up your own OPACA platform.

✨ Features

OPACA Agent Actions

SAGE can connect to a running OPACA platform and call actions from agent containers. Actions are internally transformed into tools and passed to an LLM, which then returns tool calls including their names and parameters. All generated tools/actions are then invoked via the OPACA platform.

Internal Tools

Alongside the OPACA agent actions, SAGE defines a few internal tools that can be used to perform core tasks. These include:

• ScheduleIntervalTask: Schedules a task to be executed with a given delay and a specified amount of repititions.
• ScheduleDailyTask: Schedules a daily task to be executed at a given time.
• GetScheduledTasks: Returns a list of all scheduled tasks, yet to be executed.
• CancelScheduledTask: Cancels a scheduled task.
• GatherUserInfo: Compiles a short summary of the user (session) based on all existing conversations.
• SearchChats: Searches all conversations for a given query.
• ReadFileFromUrl: Downloads a file from a given URL and uses it as input for the LLM.

Method Configuration

Multiple task-solving methods have been implemented in SAGE. Each method can be selected and configured using the UI. Especially, each LLM component can be configured to a different model, even a locally running model. This allows for a more flexible and customizable approach to task-solving.

Container Login

Some agent containers provide function which can only be called when authenticated, for example the Exchange Agent or Gitlab Agent available in the opaca-example-containers repository. When called, these functions will prompt a login window to enter credentials for the specific service. Credentials are then sent via the platform to the agent container, which will use them to authenticate the request user. Please note that entered credentials are never stored in SAGE or the OPACA platform, and that after the first successful login, tokens are generated to authenticate the user instead and sent in the header for each subsequent request to that container. However, please note that the credentials will also be sent to the agent container for authentication, so only add trusted containers to your platform environment.

Also be aware that tokens are bound to an OPACA user, not to a session. This means that if you are running your OPACA platform with authentication disabled, ALL sessions will share their container login tokens. To enable platform authentication, see the Authentication section below.

Speech I/O

The chatbot-UI supports speech-to-text (STT) and text-to-speech (TTS) using either the builtin WebSpeech functions of e.g. the Google Chrome browser, or OpenAI's Whisper model. Accordant proxy-routes are included in the backend. Also, in any case TTS and STT will only work if the frontend is using HTTPS or running on the same host (i.e. localhost).

Sample Prompts

When you start SAGE, you will notice a set of sample prompts that give an overview of the capabilities of SAGE. These prompts may only work if specific agents are available in the connected OPACA platform. Users are free to modify them to their liking and admins can change the default prompts.

File Handling

Currently, SAGE can support the following file types as input:

• PDFs: .pdf
• Images: .jpg, .jpeg, .png, .gif, .webp,

Files can be added through the upload button or by dragging them into the chat. Selecting a PDF or image uploads it immediately to the SAGE backend and activates it for the current chat. Extractable PDF text is also used to generate local keywords for inexpensive relevance selection. Text files are inserted into the prompt instead of being uploaded.

Uploaded files are not automatically sent to an LLM with every query. The current request identifies newly attached files, while internal file tools can inspect metadata across all uploaded files, activate relevant files, and send full contents to a supporting model only when needed. Active files can also be managed from the Files sidebar. See File handling for the complete workflow and file-set definitions.

Please note that not all models support file uploads.

📚 Documentation

View full documentation - Browse all available documentation pages.

Core Components

• User-Interface - The main user interface of the OPACA LLM UI.
• Methods - A brief overview of all integrated methods for task-solving.
• User Management & Sessions - How user authentication is done, sessions are handled and messages are stored.
• Backend-API - The RESTful API of the backend.

Task-Solving Methods

• Simple - A simple approach using a single agent.
• Tool-LLM - A more complex approach using 3 distinct agents.
• Orchestration - An orchestration approach dynamically generating multiple AI agents.

Agent Development

• Python Development Guide - Our Python SDK provides a simple way to develop new agent containers.
• Example Agent Containers - A selection of example agent containers for different use cases.

Deployment & Testing

• How to deploy containers - Deploy agent containers to your OPACA platform.
• Configuration - Overview of SAGE configuration options.
• Benchmarks - Performance benchmarks of SAGE.
• Unit-Tests - Tests used for the SAGE CI.

Help

• FAQ - Frequently asked questions.

🔒 Authentication

The OPACA platform that SAGE is connected to can be protected using authentication. To enable authentication on the OPACA platform, set the following environment variables in your .env file or directly in the docker-compose.yml file under the opaca-platform service:

REQUIRE_AUTH=true
SECRET=<YOUR_SECRET_KEY>
PLATFORM_ADMIN_USER=<YOUR_ADMIN_USER>
PLATFORM_ADMIN_PWD=<YOUR_ADMIN_PASSWORD>

Then start SAGE using the platform profile:

docker compose --profile platform up --build

When you then try to connect to the platform with SAGE, you will be prompted to enter your credentials.

An admin can further create additional users for SAGE by using the OPACA platform's Swagger-UI, usually available under http://localhost:8000/swagger-ui/index.html.

📄 Publication

• arXiv Pre-Print: SAGE: Tool-Augmented LLM Task Solving Strategies in Scalable Multi-Agent Environments

Contributors

Acknowledgements

Copyright 2024 - 2026, GT-ARC & DAI-Labor, TU Berlin

• Main Contributors: Robert Strehlow, Tobias Küster, Oskar Kupke, Cedric Braun
• Further contributions by: Daniel Wetzel, Brandon Llanque Kurps, Chenluanxing Liu, Abdullah Kiwan, Benjamin Acar

This repository includes software developed in the course of the project "Offenes Innovationslabor KI zur Förderung gemeinwohlorientierter KI-Anwendungen" (aka Go-KI, https://go-ki.org/) funded by the German Federal Ministry of Labour and Social Affairs (BMAS) under the funding reference number DKI.00.00032.21.