From 0de143d37e7d40607ef82c9aed6eb9225f3218f5 Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Fri, 22 May 2026 20:51:08 -0500 Subject: [PATCH 01/12] feat: Telegram channel integration via secure broker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit End-to-end Telegram support routed through a credential-isolated broker process. The bot token never enters the main app; user-ID allowlist enforcement happens in the broker; the channel and the notifications path both call into the same broker via broker_client.call(). Architecture: 1. sdk/turn/_executor.py + sdk/__init__.py — TurnExecutor and Conversation as SDK primitives with explicit injection points (TurnPersistence protocol, SystemPromptBuilder callable, preloaded_skills sequence). No imports from agents.AgentProfile, conversations, tools.memory, or tools.virtual_computer; channels build their own Agent and supply persistence + a memory-aware prompt builder. 2. conversations._turn_persistence.DiskTurnPersistence — adapter that exposes the on-disk store through the TurnPersistence protocol. 3. tools.memory.memory_prompt_block — formatted memory block that channels prepend to the system prompt; called fresh per turn so memory updates are visible immediately. 4. channels/ — new top-level package. channels/telegram/_runner.py pulls Telegram updates via broker_client.call("next_updates", ...) and sends outbound text via broker_client.call("send_message", ...). Long-poll loop with exponential backoff on transient errors and long backoff on auth/not-connected. 5. integrations/brokers/telegram_broker/ — credential-isolated broker. Owns the aiogram.Bot, runs an UpdatePump that long-polls Telegram and filters each update against the allowlist before enqueueing. RPC verbs: get_me, next_updates(timeout_ms), send_message. send_document declared in the requirement table but not implemented (host-path binding pending). 6. integrations.supervisor._catalog — telegram entry with the Capability.TELEGRAM (new) and env_injection mapping for token and allowed_user_ids. 7. tasks/_notifier.py — drops the direct httpx Telegram API code in favor of broker_client.call("send_message", ...). One broker serves both the bidirectional channel and the notifier. 8. Integrations wizard — providers.js gains a "telegram" entry with authFlow=bot_token. BotTokenSteps.jsx hosts the new three-step wizard (explainer → credentials → verifying). AddIntegrationModal routes the new flow and posts auth_blob {token, allowed_user_ids} with a client-supplied user_suffix derived from a sanitized instance name. server/_integrations_routes.handle_add_integration now accepts a client-supplied user_suffix, falling back to email derivation when absent. Security model: - Token lives only in the broker subprocess. The supervisor passes it via env on spawn; the broker wipes it from os.environ after capture. - Empty user-ID allowlist fails closed at broker boot (exit GENERIC_ERROR) — refuses to start rather than accept all senders. - Unauthorized messages are silently dropped — replying confirms the bot is live and burns outbound rate limits — with aggregate per-sender drop counts logged on a 60-second window. - The channel-side allowlist is gone entirely; the broker is the single chokepoint. Tests: - 80 new unit tests across TurnExecutor injection paths, formatter split branches, ConversationMap semantics, _parse_allowed_user_ids for the broker, UpdatePump filter + drop logging, VerbDispatcher permissions + verb behavior. 1209 pass total. Co-Authored-By: Claude Opus 4.7 (1M context) --- .env.example | 10 +- channels/__init__.py | 6 + channels/telegram/__init__.py | 13 + channels/telegram/_formatter.py | 69 ++ channels/telegram/_profile_map.py | 32 + channels/telegram/_runner.py | 623 ++++++++++++++++++ channels/telegram/_state.py | 44 ++ conversations/__init__.py | 2 + conversations/_turn_persistence.py | 58 ++ .../brokers/telegram_broker/__init__.py | 8 + .../brokers/telegram_broker/__main__.py | 163 +++++ .../brokers/telegram_broker/_updates.py | 188 ++++++ .../brokers/telegram_broker/_verbs.py | 298 +++++++++ integrations/permissions.py | 1 + integrations/supervisor/_catalog.py | 13 + pyproject.toml | 2 + sdk/__init__.py | 14 + sdk/tools/_spawn_agent.py | 96 ++- sdk/turn/__init__.py | 4 + sdk/turn/_executor.py | 306 +++++++++ server/_integrations_routes.py | 22 +- server/aiohttp_app.py | 30 +- server/message_handler.py | 265 ++------ .../add-wizard/AddIntegrationModal.jsx | 94 ++- .../integrations/add-wizard/BotTokenSteps.jsx | 252 +++++++ .../integrations/add-wizard/providers.js | 30 + tasks/_executor.py | 69 +- tasks/_notifier.py | 118 ++-- tests/unit/channels/__init__.py | 0 tests/unit/channels/telegram/__init__.py | 0 .../unit/channels/telegram/test_formatter.py | 128 ++++ .../channels/telegram/test_profile_map.py | 41 ++ tests/unit/channels/telegram/test_state.py | 60 ++ .../brokers/telegram_broker/__init__.py | 0 .../brokers/telegram_broker/test_main.py | 42 ++ .../brokers/telegram_broker/test_updates.py | 276 ++++++++ .../brokers/telegram_broker/test_verbs.py | 400 +++++++++++ .../sdk/events/test_message_handler_bridge.py | 3 +- tests/unit/sdk/turn/test_executor.py | 328 +++++++++ tests/unit/server/test_message_handler.py | 24 +- tests/unit/tasks/test_notifier.py | 157 +++-- tools/memory/__init__.py | 11 +- tools/memory/memory.py | 17 + uv.lock | 37 ++ 44 files changed, 3858 insertions(+), 496 deletions(-) create mode 100644 channels/__init__.py create mode 100644 channels/telegram/__init__.py create mode 100644 channels/telegram/_formatter.py create mode 100644 channels/telegram/_profile_map.py create mode 100644 channels/telegram/_runner.py create mode 100644 channels/telegram/_state.py create mode 100644 conversations/_turn_persistence.py create mode 100644 integrations/brokers/telegram_broker/__init__.py create mode 100644 integrations/brokers/telegram_broker/__main__.py create mode 100644 integrations/brokers/telegram_broker/_updates.py create mode 100644 integrations/brokers/telegram_broker/_verbs.py create mode 100644 sdk/turn/_executor.py create mode 100644 server/ui/src/components/integrations/add-wizard/BotTokenSteps.jsx create mode 100644 tests/unit/channels/__init__.py create mode 100644 tests/unit/channels/telegram/__init__.py create mode 100644 tests/unit/channels/telegram/test_formatter.py create mode 100644 tests/unit/channels/telegram/test_profile_map.py create mode 100644 tests/unit/channels/telegram/test_state.py create mode 100644 tests/unit/integrations/brokers/telegram_broker/__init__.py create mode 100644 tests/unit/integrations/brokers/telegram_broker/test_main.py create mode 100644 tests/unit/integrations/brokers/telegram_broker/test_updates.py create mode 100644 tests/unit/integrations/brokers/telegram_broker/test_verbs.py create mode 100644 tests/unit/sdk/turn/test_executor.py diff --git a/.env.example b/.env.example index 03f625b5..c170ac6f 100644 --- a/.env.example +++ b/.env.example @@ -11,9 +11,11 @@ LLM_API_KEY= # HuggingFace token (for gated models like Flux.1-schnell) HF_TOKEN= -# Telegram notifications for goal completion/failure -# Create a bot via @BotFather, then message it and check -# https://api.telegram.org/bot/getUpdates for your chat ID -TELEGRAM_BOT_TOKEN= +# Telegram notifications for goal completion/failure. The bot token lives +# inside the telegram broker (configured per integration); the notifier only +# needs to know which integration to target and which chat to send to. +# To find your chat ID: send a message to your bot, then visit +# https://api.telegram.org/bot/getUpdates and look for chat.id. +TELEGRAM_INTEGRATION_ID= TELEGRAM_CHAT_ID= diff --git a/channels/__init__.py b/channels/__init__.py new file mode 100644 index 00000000..cbd80484 --- /dev/null +++ b/channels/__init__.py @@ -0,0 +1,6 @@ +"""Channels — entry points that receive user input and run agent turns. + +Each subpackage (``telegram``, future ones) hosts one channel implementation. +A channel constructs an ``Agent``, builds a ``Conversation``, and drives the +turn loop via ``sdk.TurnExecutor``. +""" diff --git a/channels/telegram/__init__.py b/channels/telegram/__init__.py new file mode 100644 index 00000000..d9b10c6e --- /dev/null +++ b/channels/telegram/__init__.py @@ -0,0 +1,13 @@ +"""Telegram channel — receives messages from Telegram chats and processes +them through the agent pipeline, returning results inline. +""" + +from channels.telegram._formatter import TelegramFormatter +from channels.telegram._runner import TelegramChannel +from channels.telegram._state import ConversationMap + +__all__ = [ + "ConversationMap", + "TelegramChannel", + "TelegramFormatter", +] diff --git a/channels/telegram/_formatter.py b/channels/telegram/_formatter.py new file mode 100644 index 00000000..bbd9f46b --- /dev/null +++ b/channels/telegram/_formatter.py @@ -0,0 +1,69 @@ +"""Format agent output for Telegram delivery.""" + +from __future__ import annotations + +import re +from pathlib import Path + +__all__ = ["TelegramFormatter"] + +# Telegram message size limit. +_MSG_LIMIT = 4096 + + +class TelegramFormatter: + """Prepares agent text for Telegram: splitting, escaping, truncating.""" + + # -- public API ----------------------------------------------------- + + @staticmethod + def split(text: str, *, limit: int = _MSG_LIMIT) -> list[str]: + """Split *text* into chunks that fit Telegram's message size limit. + + Prefers paragraph boundaries, then sentence boundaries, then + falls back to hard splitting at *limit*. + """ + if len(text) <= limit: + return [text] + + chunks: list[str] = [] + remaining = text + while remaining: + if len(remaining) <= limit: + chunks.append(remaining) + break + + # Try paragraph break + cut = remaining.rfind("\n\n", 0, limit) + if cut == -1: + # Try single newline + cut = remaining.rfind("\n", 0, limit) + if cut == -1: + # Try sentence boundary + cut = max( + remaining.rfind(". ", 0, limit), + remaining.rfind("! ", 0, limit), + remaining.rfind("? ", 0, limit), + ) + if cut == -1 or cut < limit // 4: + # Hard split + cut = limit + + chunks.append(remaining[:cut].rstrip()) + remaining = remaining[cut:].lstrip("\n") + + return chunks + + @staticmethod + def escape_markdown(text: str) -> str: + """Escape characters that are special in Telegram MarkdownV2.""" + special = r"_*[]()~`>#+-=|{}.!" + return re.sub(r"([%s])" % re.escape(special), r"\\\1", text) + + @staticmethod + def file_caption(path: Path, *, index: int = 0, total: int = 1) -> str: + """Build a short caption for an attached file.""" + name = path.name + if total == 1: + return f"📎 {name}" + return f"📎 {name} ({index + 1}/{total})" \ No newline at end of file diff --git a/channels/telegram/_profile_map.py b/channels/telegram/_profile_map.py new file mode 100644 index 00000000..421add61 --- /dev/null +++ b/channels/telegram/_profile_map.py @@ -0,0 +1,32 @@ +"""Per-chat agent-profile selection. + +A simple ``chat_id -> profile_id`` mapping the channel consults on every +turn. Empty mapping means the channel falls back to its configured default. +Persistence is in-memory only today; a chat that hits a fresh process +restarts at the default until the user picks again. +""" + +from __future__ import annotations + +from typing import MutableMapping + +__all__ = ["ProfileMap"] + + +class ProfileMap: + """Maps Telegram chat IDs to the agent profile ID active for that chat.""" + + def __init__(self) -> None: + self._map: MutableMapping[int, str] = {} + + def get(self, chat_id: int) -> str | None: + """Return the chosen profile ID for ``chat_id`` or ``None``.""" + return self._map.get(chat_id) + + def set(self, chat_id: int, profile_id: str) -> None: + """Record the user's profile choice for this chat.""" + self._map[chat_id] = profile_id + + def clear(self, chat_id: int) -> None: + """Drop the recorded choice so this chat reverts to the default.""" + self._map.pop(chat_id, None) diff --git a/channels/telegram/_runner.py b/channels/telegram/_runner.py new file mode 100644 index 00000000..486c54a2 --- /dev/null +++ b/channels/telegram/_runner.py @@ -0,0 +1,623 @@ +"""Telegram channel — talks to the telegram broker over RPC. + +The channel no longer holds the bot token or the aiogram client. It pulls +incoming messages from the broker via the ``next_updates`` long-poll verb +and sends outbound text via ``send_message``. All sender allowlist +enforcement lives in the broker; this process trusts whatever the broker +forwards. +""" + +from __future__ import annotations + +import asyncio +import logging +from contextlib import suppress +from pathlib import Path +from typing import Any + +from agents import build_agent, get_agent_profile, list_agent_profiles +from agents.types import Agent +from channels.telegram._formatter import TelegramFormatter +from channels.telegram._profile_map import ProfileMap +from channels.telegram._state import ConversationMap +from conversations import DiskTurnPersistence, load_conversation_history +from integrations import supervisor_client +from integrations.broker_client import ( + IntegrationAuthFailed, + IntegrationError, + IntegrationNotConnected, + call as broker_call, +) +from sdk import Conversation, TurnExecutor +from sdk.context import ConversationHistory +from sdk.turn import is_turn_active, request_stop +from tools.memory import forget, memory_prompt_block, remember +from tools.virtual_computer.run_bash_cmd import run_bash_cmd + +logger = logging.getLogger(__name__) + +__all__ = ["TelegramChannel"] + +# Long-poll window the channel asks the broker to hold each call for. +_LONG_POLL_TIMEOUT_MS = 30_000 + +# Exponential backoff bounds for transient broker errors. +_BACKOFF_INITIAL_SECONDS = 1.0 +_BACKOFF_CAP_SECONDS = 30.0 + +# Backoff used when the broker reports a state the user has to fix +# (auth_failed, not_connected). Polling fast doesn't help — wait longer. +_LONG_BACKOFF_SECONDS = 60.0 + +# Telegram's chat-action indicator clears after ~5 seconds. Re-send every 4 +# so the "typing…" header stays visible across a multi-minute turn. +_TYPING_INDICATOR_INTERVAL_SECONDS = 4.0 + +# callback_data prefix encoding profile picks. Keeps the payload format +# self-describing so unrelated callbacks (future verbs) don't collide. +_PROFILE_CALLBACK_PREFIX = "profile:" + +# Initial status text shown while we wait for the first event from the agent. +_STATUS_THINKING = "🤔 Thinking..." +_STATUS_WRITING = "✍️ Writing response..." + + +class TelegramChannel: + """Pulls updates from the telegram broker and dispatches agent turns.""" + + def __init__(self, *, app_sock_path: Path) -> None: + self._app_sock = app_sock_path + self._state = ConversationMap() + self._profiles = ProfileMap() + self._formatter = TelegramFormatter() + self._turn_executor = TurnExecutor() + self._persistence = DiskTurnPersistence() + self._conversations: dict[str, Conversation] = {} + self._integration_id: str = "" + self._default_profile_id: str = "computron" + self._pull_task: asyncio.Task[None] | None = None + self._turn_tasks: set[asyncio.Task[None]] = set() + self._stopping = False + + # -- lifecycle ------------------------------------------------------ + + async def start(self) -> None: + """Auto-discover a Telegram integration and start the pull loop. + + Queries the supervisor for any integration with slug=telegram. If + none is registered, logs and exits — there's nothing to drive. If + more than one exists, binds to the first and notes the choice so the + behavior is explicit. + """ + integration_id = await self._discover_integration() + if integration_id is None: + logger.info("No Telegram integration registered; channel not starting") + return + self._integration_id = integration_id + + # Identity probe — confirms the broker is reachable and authenticated. + try: + me = await broker_call( + integration_id, "get_me", {}, app_sock_path=self._app_sock, + ) + except IntegrationNotConnected: + logger.warning( + "Telegram integration %r vanished between discovery and probe; " + "channel not starting", + integration_id, + ) + return + except IntegrationAuthFailed as exc: + logger.error( + "Telegram integration %r auth failed; channel not starting: %s", + integration_id, exc, + ) + return + except IntegrationError as exc: + logger.error( + "Telegram broker probe failed for %r; channel not starting: %s", + integration_id, exc, + ) + return + + logger.info( + "telegram channel started integration_id=%s bot=@%s (id=%d)", + integration_id, me.get("username"), me.get("id"), + ) + self._pull_task = asyncio.create_task(self._pull_loop(), name="telegram-pull") + + async def _discover_integration(self) -> str | None: + """Pick the telegram integration the channel should bind to. + + Returns the integration_id, or None if no telegram integration is + registered. With multiple registered, the first is chosen and the + choice is logged. + """ + try: + result = await supervisor_client.call( + "list", {}, app_sock_path=self._app_sock, + ) + except (FileNotFoundError, ConnectionRefusedError, OSError) as exc: + logger.warning( + "Supervisor unreachable; Telegram channel not starting: %s", exc, + ) + return None + + telegrams = [ + i for i in result.get("integrations", []) + if i.get("slug") == "telegram" + ] + if not telegrams: + return None + if len(telegrams) > 1: + ids = sorted(i["id"] for i in telegrams) + logger.warning( + "Multiple Telegram integrations registered (%s); binding to %s. " + "Multi-integration support is a future feature.", + ids, telegrams[0]["id"], + ) + return telegrams[0]["id"] + + async def stop(self) -> None: + """Stop the pull loop and any in-flight turns.""" + self._stopping = True + if self._pull_task is not None: + self._pull_task.cancel() + with suppress(asyncio.CancelledError): + await self._pull_task + for task in list(self._turn_tasks): + task.cancel() + if self._turn_tasks: + await asyncio.gather(*self._turn_tasks, return_exceptions=True) + logger.info("telegram channel stopped") + + # -- pull loop ------------------------------------------------------ + + async def _pull_loop(self) -> None: + """Long-poll the broker for incoming messages and dispatch each.""" + backoff = _BACKOFF_INITIAL_SECONDS + while not self._stopping: + try: + result = await broker_call( + self._integration_id, + "next_updates", + {"timeout_ms": _LONG_POLL_TIMEOUT_MS}, + app_sock_path=self._app_sock, + ) + backoff = _BACKOFF_INITIAL_SECONDS + except asyncio.CancelledError: + raise + except IntegrationAuthFailed as exc: + logger.error( + "telegram broker auth failed (will retry in %.0fs): %s", + _LONG_BACKOFF_SECONDS, exc, + ) + await asyncio.sleep(_LONG_BACKOFF_SECONDS) + continue + except IntegrationNotConnected as exc: + logger.warning( + "telegram broker not connected (will retry in %.0fs): %s", + _LONG_BACKOFF_SECONDS, exc, + ) + await asyncio.sleep(_LONG_BACKOFF_SECONDS) + continue + except IntegrationError as exc: + logger.warning( + "telegram broker call failed (retrying in %.1fs): %s", + backoff, exc, + ) + await asyncio.sleep(backoff) + backoff = min(backoff * 2, _BACKOFF_CAP_SECONDS) + continue + + for update in result.get("updates", []): + await self._dispatch(update) + + # -- dispatch ------------------------------------------------------- + + async def _dispatch(self, update: dict[str, Any]) -> None: + """Route a single update to the right handler.""" + kind = update.get("type") + if kind == "message": + await self._dispatch_message(update) + elif kind == "callback_query": + await self._dispatch_callback(update) + + async def _dispatch_message(self, update: dict[str, Any]) -> None: + chat_id = update["chat_id"] + text = update["text"] + + if update.get("is_command"): + cmd = text.split(maxsplit=1)[0][1:] # strip leading "/" + if cmd == "new": + await self._handle_new(chat_id) + elif cmd == "stop": + await self._handle_stop(chat_id) + elif cmd == "profile": + await self._handle_profile(chat_id) + elif cmd == "help": + await self._handle_help(chat_id) + else: + await self._send_text(chat_id, f"Unknown command: /{cmd}") + return + + # Regular text — spawn a turn task if one isn't already running. + conv_id = self._state.get(chat_id) + if is_turn_active(conv_id): + await self._send_text( + chat_id, "⏳ A turn is already running. Use /stop to cancel it.", + ) + return + task = asyncio.create_task(self._run_turn(chat_id, text)) + self._turn_tasks.add(task) + task.add_done_callback(self._turn_tasks.discard) + + async def _dispatch_callback(self, update: dict[str, Any]) -> None: + """Handle an inline-keyboard button tap.""" + callback_id = update["callback_id"] + chat_id = update["chat_id"] + data = update["data"] + + if data.startswith(_PROFILE_CALLBACK_PREFIX): + profile_id = data[len(_PROFILE_CALLBACK_PREFIX):] + await self._select_profile(chat_id, profile_id, callback_id) + return + + # Unknown callback — dismiss the loading spinner anyway so the + # Telegram client doesn't sit on it forever. + await self._answer_callback(callback_id) + + # -- command handlers ----------------------------------------------- + + async def _handle_new(self, chat_id: int) -> None: + conv_id = self._state.reset(chat_id) + self._conversations.pop(conv_id, None) + await self._send_text(chat_id, f"🔄 New conversation started ({conv_id})") + logger.info("telegram /new chat_id=%s conv_id=%s", chat_id, conv_id) + + async def _handle_stop(self, chat_id: int) -> None: + conv_id = self._state.conversation_id_for(chat_id) + if conv_id and is_turn_active(conv_id): + request_stop(conv_id) + await self._send_text(chat_id, "⏹ Stop requested") + logger.info("telegram /stop chat_id=%s conv_id=%s", chat_id, conv_id) + else: + await self._send_text(chat_id, "No active turn to stop") + + async def _handle_profile(self, chat_id: int) -> None: + """Show the profile picker as an inline keyboard.""" + profiles = list_agent_profiles() + if not profiles: + await self._send_text(chat_id, "No agent profiles available.") + return + + current = self._profiles.get(chat_id) or self._default_profile_id + # One button per row keeps long profile lists readable on mobile. + buttons = [ + [{ + "text": f"{'✓ ' if p.id == current else ''}{p.name}", + "data": f"{_PROFILE_CALLBACK_PREFIX}{p.id}", + }] + for p in profiles + ] + try: + await broker_call( + self._integration_id, + "send_message", + { + "chat_id": chat_id, + "text": "Pick an agent profile for this chat:", + "buttons": buttons, + }, + app_sock_path=self._app_sock, + ) + except IntegrationError as exc: + logger.warning("telegram /profile failed chat_id=%s: %s", chat_id, exc) + + async def _handle_help(self, chat_id: int) -> None: + lines = [ + "Commands:", + " /new — start a new conversation", + " /stop — stop the current turn", + " /profile — pick an agent profile for this chat", + " /help — this message", + ] + await self._send_text(chat_id, "\n".join(lines)) + + async def _select_profile( + self, chat_id: int, profile_id: str, callback_id: str, + ) -> None: + """Record the user's profile choice and acknowledge.""" + profile = get_agent_profile(profile_id) + if profile is None or not profile.enabled: + await self._answer_callback(callback_id, text="Profile unavailable") + return + self._profiles.set(chat_id, profile_id) + # Tiny toast on the button tap + a normal reply for the chat history. + await self._answer_callback(callback_id, text=f"Profile: {profile.name}") + await self._send_text(chat_id, f"✅ Profile set to {profile.name}") + logger.info( + "telegram profile selected chat_id=%s profile_id=%s", + chat_id, profile_id, + ) + + # -- turn execution ------------------------------------------------- + + async def _run_turn(self, chat_id: int, text: str) -> None: + """Execute one agent turn for the given chat and deliver the reply.""" + conversation_id = self._state.get(chat_id) + conversation, is_new = self._get_conversation(conversation_id) + + profile = self._resolve_profile(chat_id) + if profile is None: + await self._send_text( + chat_id, "⚠️ No agent profile available. Set one with /profile.", + ) + return + + agent = build_agent(profile, tools=[run_bash_cmd, remember, forget]) + + logger.info( + "telegram turn start chat_id=%s conversation_id=%s is_new=%s profile=%s", + chat_id, conversation_id, is_new, profile.id, + ) + + typing_task = asyncio.create_task( + self._keep_typing(chat_id), name="telegram-typing", + ) + status = _StatusMessage(chat_id, self) + await status.start(_STATUS_THINKING) + collected_text = "" + file_paths: list[str] = [] + wrote_started = False + + try: + async for event in self._turn_executor.execute( + conversation=conversation, + agent=agent, + user_content=text, + is_new_conversation=is_new, + preloaded_skills=profile.skills, + persistence=self._persistence, + build_system_prompt=lambda: memory_prompt_block() + agent.instruction, + profile_name=profile.name, + ): + payload = event.payload + if event.type == "tool_call" and hasattr(payload, "name"): + await status.set(f"🔧 Calling {payload.name}...") + elif event.type == "agent_started" and hasattr(payload, "agent_name"): + await status.set(f"🚀 Spawning {payload.agent_name}...") + elif event.type == "agent_completed": + await status.set(_STATUS_THINKING) + elif event.type == "content" and hasattr(payload, "content"): + if payload.content: + collected_text += payload.content + if not wrote_started: + wrote_started = True + await status.set(_STATUS_WRITING) + elif event.type == "file_output" and hasattr(payload, "path"): + if payload.path: + file_paths.append(payload.path) + except Exception: + logger.exception( + "telegram turn failed chat_id=%s conversation_id=%s", + chat_id, conversation_id, + ) + await status.clear() + await self._send_text( + chat_id, "⚠️ An error occurred while processing your message.", + ) + return + finally: + typing_task.cancel() + with suppress(asyncio.CancelledError, Exception): + await typing_task + + await status.clear() + + if collected_text.strip(): + await self._send_text(chat_id, collected_text) + + for path in file_paths: + await self._send_document(chat_id, path) + + logger.info( + "telegram turn end chat_id=%s conversation_id=%s text_len=%d files=%d", + chat_id, conversation_id, len(collected_text), len(file_paths), + ) + + def _resolve_profile(self, chat_id: int): + """Return the AgentProfile for *chat_id* — picked, default, or None. + + The profile's own ``model`` field is the source of truth. There's no + Telegram-specific model override anymore; pick a different profile + via ``/profile`` if you want a different model on Telegram. + """ + for candidate in (self._profiles.get(chat_id), self._default_profile_id): + if not candidate: + continue + profile = get_agent_profile(candidate) + if profile is None or not profile.enabled: + continue + return profile + return None + + async def _keep_typing(self, chat_id: int) -> None: + """Re-send the typing indicator on a loop while a turn is running. + + Telegram's chat-action lasts ~5 seconds; loop slightly faster so + the indicator never gaps. Stops when the task is cancelled. + """ + try: + while True: + try: + await broker_call( + self._integration_id, + "send_chat_action", + {"chat_id": chat_id, "action": "typing"}, + app_sock_path=self._app_sock, + ) + except IntegrationError as exc: + # Transient broker hiccup — drop this beat and try again. + logger.debug( + "telegram send_chat_action failed (will retry): %s", exc, + ) + await asyncio.sleep(_TYPING_INDICATOR_INTERVAL_SECONDS) + except asyncio.CancelledError: + raise + + # -- conversation cache --------------------------------------------- + + def _get_conversation(self, conversation_id: str) -> tuple[Conversation, bool]: + """Return the conversation for *conversation_id*, creating if needed. + + Returns: + ``(conversation, is_new)`` where ``is_new`` is True only when no + on-disk history existed (a genuine first-time use). + """ + if conversation_id in self._conversations: + return self._conversations[conversation_id], False + + messages = load_conversation_history(conversation_id) + if messages is not None: + conversation = Conversation( + id=conversation_id, + history=ConversationHistory(messages, instance_id=conversation_id), + ) + self._conversations[conversation_id] = conversation + return conversation, False + + conversation = Conversation( + id=conversation_id, + history=ConversationHistory(instance_id=conversation_id), + ) + self._conversations[conversation_id] = conversation + return conversation, True + + # -- outbound ------------------------------------------------------- + + async def _send_text(self, chat_id: int, text: str) -> None: + """Send *text* to *chat_id*, splitting if necessary.""" + for chunk in self._formatter.split(text): + try: + await broker_call( + self._integration_id, + "send_message", + {"chat_id": chat_id, "text": chunk}, + app_sock_path=self._app_sock, + ) + except IntegrationError as exc: + logger.warning( + "telegram send_message failed chat_id=%s: %s", chat_id, exc, + ) + return + + async def _send_document(self, chat_id: int, host_path: str) -> None: + """Upload *host_path* as a Telegram document. + + Reads the file via the broker so the channel never touches the bot + token. Errors are logged and the upload is skipped — the rest of + the reply still goes through. + """ + path = Path(host_path) + caption = self._formatter.file_caption(path) + try: + await broker_call( + self._integration_id, + "send_document", + { + "chat_id": chat_id, + "host_path": str(host_path), + "caption": caption, + "filename": path.name, + }, + app_sock_path=self._app_sock, + ) + except IntegrationError as exc: + logger.warning( + "telegram send_document failed chat_id=%s path=%s: %s", + chat_id, host_path, exc, + ) + + async def _answer_callback( + self, callback_id: str, *, text: str | None = None, + ) -> None: + """Dismiss the loading spinner on an inline-keyboard button tap.""" + args: dict[str, Any] = {"callback_id": callback_id} + if text is not None: + args["text"] = text + try: + await broker_call( + self._integration_id, + "answer_callback_query", + args, + app_sock_path=self._app_sock, + ) + except IntegrationError as exc: + logger.debug("telegram answer_callback_query failed: %s", exc) + + +class _StatusMessage: + """A live status message that the channel edits in place during a turn. + + Sends one Telegram message at turn start (capturing its message_id), + edits the text on state transitions (tool calls, sub-agent spawns, + response writing), and deletes the message before the final reply. + Falls back to silent no-ops if the initial send or any edit fails so + a broker hiccup never breaks the turn itself. + """ + + def __init__(self, chat_id: int, channel: TelegramChannel) -> None: + self._chat_id = chat_id + self._channel = channel + self._message_id: int | None = None + self._current: str | None = None + + async def start(self, text: str) -> None: + """Post the initial status message and capture its message_id.""" + try: + result = await broker_call( + self._channel._integration_id, + "send_message", + {"chat_id": self._chat_id, "text": text}, + app_sock_path=self._channel._app_sock, + ) + self._message_id = result.get("message_id") + self._current = text + except IntegrationError as exc: + logger.debug("telegram status send_message failed: %s", exc) + + async def set(self, text: str) -> None: + """Edit the status to *text* if it differs from the current value.""" + if self._message_id is None or text == self._current: + return + try: + await broker_call( + self._channel._integration_id, + "edit_message_text", + { + "chat_id": self._chat_id, + "message_id": self._message_id, + "text": text, + }, + app_sock_path=self._channel._app_sock, + ) + self._current = text + except IntegrationError as exc: + logger.debug("telegram status edit_message_text failed: %s", exc) + + async def clear(self) -> None: + """Delete the status message. Safe to call multiple times.""" + if self._message_id is None: + return + message_id = self._message_id + self._message_id = None + try: + await broker_call( + self._channel._integration_id, + "delete_message", + {"chat_id": self._chat_id, "message_id": message_id}, + app_sock_path=self._channel._app_sock, + ) + except IntegrationError as exc: + logger.debug("telegram status delete_message failed: %s", exc) diff --git a/channels/telegram/_state.py b/channels/telegram/_state.py new file mode 100644 index 00000000..9f9c5de6 --- /dev/null +++ b/channels/telegram/_state.py @@ -0,0 +1,44 @@ +"""Lightweight mapping between Telegram chat IDs and agent conversation IDs.""" + +from __future__ import annotations + +import uuid +from typing import MutableMapping + +__all__ = ["ConversationMap"] + + +class ConversationMap: + """Maps Telegram chat IDs to agent conversation IDs. + + Default convention: ``telegram_{chat_id}``. When a user issues /new, + a fresh UUID-based suffix is appended so the agent starts a new context. + """ + + def __init__(self) -> None: + self._map: MutableMapping[int, str] = {} + + # -- lookup --------------------------------------------------------- + + def get(self, chat_id: int) -> str: + """Return the conversation ID for *chat_id*, creating a default if absent.""" + return self._map.setdefault(chat_id, f"telegram_{chat_id}") + + # -- reset (used by /new) ------------------------------------------- + + def reset(self, chat_id: int) -> str: + """Assign a brand-new conversation ID for *chat_id*. + + The new ID is ``telegram_{chat_id}_{uuid4_short}`` so it's unique + but still traceable to the originating chat. + """ + short = uuid.uuid4().hex[:8] + conv_id = f"telegram_{chat_id}_{short}" + self._map[chat_id] = conv_id + return conv_id + + # -- introspection -------------------------------------------------- + + def conversation_id_for(self, chat_id: int) -> str | None: + """Return the current conversation ID without side-effects, or ``None``.""" + return self._map.get(chat_id) \ No newline at end of file diff --git a/conversations/__init__.py b/conversations/__init__.py index 57d08068..5e4d49f8 100644 --- a/conversations/__init__.py +++ b/conversations/__init__.py @@ -23,9 +23,11 @@ from ._title_generation import ( generate_conversation_title, ) +from ._turn_persistence import DiskTurnPersistence __all__ = [ "ConversationSummary", + "DiskTurnPersistence", "SummaryRecord", "delete_conversation", "generate_conversation_title", diff --git a/conversations/_turn_persistence.py b/conversations/_turn_persistence.py new file mode 100644 index 00000000..62dedbc6 --- /dev/null +++ b/conversations/_turn_persistence.py @@ -0,0 +1,58 @@ +"""TurnPersistence adapter backed by the on-disk conversation store.""" + +from __future__ import annotations + +import logging +from collections.abc import Iterable +from typing import TYPE_CHECKING + +from ._store import ( + load_loaded_skills, + save_agent_events, + save_conversation_title, + save_loaded_skills, +) +from ._title_generation import generate_conversation_title + +if TYPE_CHECKING: + from sdk.events import AgentEvent + +logger = logging.getLogger(__name__) + +__all__ = ["DiskTurnPersistence"] + + +class DiskTurnPersistence: + """Implements the ``sdk.turn.TurnPersistence`` protocol against the on-disk store.""" + + def load_skills(self, conversation_id: str) -> Iterable[str]: + return load_loaded_skills(conversation_id) + + def save_skills(self, conversation_id: str, skills: Iterable[str]) -> None: + save_loaded_skills(conversation_id, frozenset(skills)) + + def save_events( + self, + conversation_id: str, + events: list[AgentEvent], + ) -> None: + save_agent_events(conversation_id, events) + + async def on_new_conversation( + self, + conversation_id: str, + first_message: str, + ) -> None: + try: + title = await generate_conversation_title(first_message) + save_conversation_title(conversation_id, title) + logger.info( + "Generated title for conversation %s: %r", + conversation_id, + title, + ) + except Exception: + logger.exception( + "Failed to generate title for conversation %s", + conversation_id, + ) diff --git a/integrations/brokers/telegram_broker/__init__.py b/integrations/brokers/telegram_broker/__init__.py new file mode 100644 index 00000000..482df1c1 --- /dev/null +++ b/integrations/brokers/telegram_broker/__init__.py @@ -0,0 +1,8 @@ +"""Telegram broker — holds the bot token and bridges Telegram's HTTP API +to the broker RPC surface. + +The broker process owns the aiogram ``Bot`` instance, runs a long-polling +loop, enforces the per-integration allowlist of sender user IDs, and +exposes a small set of verbs (``next_updates``, ``send_message``, +``get_me``) over the standard UDS RPC framing. +""" diff --git a/integrations/brokers/telegram_broker/__main__.py b/integrations/brokers/telegram_broker/__main__.py new file mode 100644 index 00000000..c9a8df5b --- /dev/null +++ b/integrations/brokers/telegram_broker/__main__.py @@ -0,0 +1,163 @@ +"""Telegram broker entry point: ``python -m integrations.brokers.telegram_broker``. + +The supervisor spawns this with credentials and policy in the environment, +reads ``READY\\n`` from stdout as the signal the broker is serving, and later +sends SIGTERM to shut it down. + +Exit codes: + +- 0: clean shutdown. +- 77: the bot token was rejected by Telegram. The supervisor flips the + integration to ``auth_failed`` and does not restart. +- 1: anything else (env-parse failure, network unreachable, allowlist empty, + internal error). The supervisor transitions to ``error`` and restarts on + backoff. +""" + +from __future__ import annotations + +import asyncio +import logging +import os +import sys +from pathlib import Path +from typing import Any + +from aiogram import Bot +from aiogram.exceptions import TelegramUnauthorizedError + +from integrations._env import env_required +from integrations._perms import PROCESS_UMASK, disable_core_dumps +from integrations._rpc import serve_rpc +from integrations.brokers._common._exit_codes import AUTH_FAIL, CLEAN_SHUTDOWN, GENERIC_ERROR +from integrations.brokers._common._ready import print_ready +from integrations.brokers.telegram_broker._updates import UpdatePump +from integrations.brokers.telegram_broker._verbs import VerbDispatcher +from integrations.permissions import permissions_from_env + +logger = logging.getLogger("telegram_broker") + +# Owner-only by default for everything this process creates. Specific call +# sites still set explicit modes (e.g. 0o660 sockets) — see integrations/_perms.py. +os.umask(PROCESS_UMASK) + +# No core dumps — the broker holds the bot token in memory, and a +# kernel-generated core file would write it to disk. +disable_core_dumps() + + +def _parse_allowed_user_ids(raw: str) -> frozenset[int]: + """Parse a comma-separated user ID list. Bad entries logged and skipped.""" + out: set[int] = set() + for part in raw.split(","): + part = part.strip() + if not part: + continue + try: + out.add(int(part)) + except ValueError: + logger.warning("TELEGRAM_ALLOWED_USER_IDS entry is not an integer: %r", part) + return frozenset(out) + + +async def _run() -> int: + integration_id = env_required("INTEGRATION_ID") + socket_path = Path(env_required("BROKER_SOCKET")) + token = env_required("TELEGRAM_BOT_TOKEN") + permissions = permissions_from_env(env_required("PERMISSIONS")) + allowed = _parse_allowed_user_ids(env_required("TELEGRAM_ALLOWED_USER_IDS")) + + # Wipe the token from the process environ once captured. Best-effort + # hygiene: narrows in-process exposure (debuggers, traceback locals, + # crash-reporter captures). The kernel's /proc//environ snapshot is + # set at exec time and won't update; that file is mode 0400 and the + # agent runs as a different UID, so it's not a concern regardless. + os.environ.pop("TELEGRAM_BOT_TOKEN", None) + os.environ.pop("TELEGRAM_ALLOWED_USER_IDS", None) + + log = logging.getLogger(f"telegram_broker[{integration_id}]") + + if not allowed: + # Fail closed at boot. A Telegram bot has no Telegram-side ACL, so an + # empty allowlist would let anyone who discovers the bot username drive + # the agent. Refuse to start rather than silently accept everyone. + log.error("TELEGRAM_ALLOWED_USER_IDS is empty; refusing to start") + return GENERIC_ERROR + + bot = Bot(token=token) + + # Identity probe: detects an invalid/revoked token before we start polling. + try: + me = await bot.get_me() + except TelegramUnauthorizedError as exc: + log.error("Telegram rejected the bot token: %s", exc) + await bot.session.close() + return AUTH_FAIL + except OSError as exc: + # Network-level failure — supervisor retries on backoff. + log.error("could not reach Telegram: %s", exc) + await bot.session.close() + return GENERIC_ERROR + + log.info( + "authenticated as @%s (id=%d) allowed_user_ids=%s", + me.username, me.id, sorted(allowed), + ) + + pump = UpdatePump(bot, allowed, integration_id=integration_id) + pump_task = asyncio.create_task(pump.run(), name="telegram-update-pump") + + dispatcher = VerbDispatcher(bot, pump, permissions=permissions) + + async def handler(verb: str, args: dict[str, Any]) -> dict[str, Any]: + return await dispatcher.dispatch(verb, args) + + server = await serve_rpc(socket_path, handler) + log.info("listening on %s (permissions=%s)", socket_path, permissions) + + # READY sentinel: the supervisor watches stdout for this exact line and + # flips the integration from ``pending`` to ``active`` on seeing it. + print_ready() + + async with server: + try: + done, _ = await asyncio.wait( + [pump_task, asyncio.create_task(server.serve_forever())], + return_when=asyncio.FIRST_COMPLETED, + ) + except asyncio.CancelledError: + log.info("shutting down") + done = set() + + # If the pump exited cleanly with auth_failed=True, that's the only path + # that should map to a 77 exit; cancellation or server-finished is clean. + if pump.auth_failed: + await bot.session.close() + return AUTH_FAIL + + pump_task.cancel() + try: + await pump_task + except (asyncio.CancelledError, Exception): + pass + + await bot.session.close() + return CLEAN_SHUTDOWN + + +def main() -> None: + """Console entry point — configure logging, run the async body, exit with its return code.""" + logging.basicConfig( + stream=sys.stderr, + level=logging.INFO, + format="[%(name)s] %(asctime)s %(levelname)s %(message)s", + ) + try: + code = asyncio.run(_run()) + except KeyboardInterrupt: + code = CLEAN_SHUTDOWN + sys.exit(code) + + +if __name__ == "__main__": + main() diff --git a/integrations/brokers/telegram_broker/_updates.py b/integrations/brokers/telegram_broker/_updates.py new file mode 100644 index 00000000..936255d6 --- /dev/null +++ b/integrations/brokers/telegram_broker/_updates.py @@ -0,0 +1,188 @@ +"""Long-polling update pump. + +Pulls updates from Telegram in the background and pushes allowed ones onto +an internal queue. Verb handlers drain the queue via ``next_updates``. + +Drops from non-allowlisted senders are silent — replying would confirm the +bot is live and waste outbound rate limits. Drop counts are logged +periodically rather than per-message so a spam burst doesn't spam the log. +""" + +from __future__ import annotations + +import asyncio +import logging +import time +from typing import Any + +from aiogram import Bot +from aiogram.exceptions import TelegramAPIError, TelegramUnauthorizedError + +logger = logging.getLogger(__name__) + +__all__ = ["UpdatePump", "update_to_dict"] + +# How often to surface aggregate drop counts. A spam burst at this interval +# produces at most one log line per chat ID per window. +_DROP_LOG_INTERVAL_SECONDS = 60.0 + +# Long-poll timeout in seconds — how long Telegram holds the request open +# waiting for an update before returning an empty list. 25 is comfortably +# under typical proxy/load-balancer timeouts and matches aiogram's default. +_LONG_POLL_TIMEOUT_SECONDS = 25 + +# Backoff for transient network errors. The first failure waits 1s; each +# subsequent failure doubles up to the cap. Resets on the first success. +_BACKOFF_INITIAL_SECONDS = 1.0 +_BACKOFF_CAP_SECONDS = 30.0 + + +def update_to_dict(update: Any) -> dict[str, Any] | None: + """Flatten an aiogram ``Update`` to the wire shape, or ``None`` to skip. + + Forwards text ``message`` updates and ``callback_query`` updates + (inline-keyboard button taps). Other update kinds are dropped. + """ + message = getattr(update, "message", None) + if message is not None: + return _message_to_dict(message) + callback = getattr(update, "callback_query", None) + if callback is not None: + return _callback_to_dict(callback) + return None + + +def _message_to_dict(message: Any) -> dict[str, Any] | None: + text = getattr(message, "text", None) + if not text: + return None + user = message.from_user + if user is None: + return None + return { + "type": "message", + "message_id": message.message_id, + "chat_id": message.chat.id, + "from_user_id": user.id, + "from_username": getattr(user, "username", None), + "text": text, + "is_command": text.startswith("/"), + "timestamp": int(message.date.timestamp()) if message.date else int(time.time()), + } + + +def _callback_to_dict(callback: Any) -> dict[str, Any] | None: + """Flatten a callback_query (inline-keyboard button tap) to the wire shape.""" + user = getattr(callback, "from_user", None) + if user is None: + return None + data = getattr(callback, "data", None) + if data is None: + return None + msg = getattr(callback, "message", None) + chat_id = msg.chat.id if (msg is not None and msg.chat is not None) else None + message_id = msg.message_id if msg is not None else None + if chat_id is None or message_id is None: + return None + return { + "type": "callback_query", + "callback_id": callback.id, + "from_user_id": user.id, + "from_username": getattr(user, "username", None), + "data": data, + "chat_id": chat_id, + "message_id": message_id, + "timestamp": int(time.time()), + } + + +class UpdatePump: + """Background long-poller. Allowed updates land on ``self.queue``.""" + + def __init__( + self, + bot: Bot, + allowed_user_ids: frozenset[int], + *, + integration_id: str, + ) -> None: + self._bot = bot + self._allowed = allowed_user_ids + self._integration_id = integration_id + self.queue: asyncio.Queue[dict[str, Any]] = asyncio.Queue() + + # Aggregate drop tracking — keyed by sender so a single bad actor + # contributes one log line per window even if they send thousands. + self._drop_counts: dict[int, int] = {} + self._drop_window_started_at: float = time.monotonic() + + # Set to True only after the pump observes an Unauthorized response, + # so __main__ can exit AUTH_FAIL instead of restarting in a loop. + self.auth_failed: bool = False + + async def run(self) -> None: + """Pump loop — long-poll, filter, enqueue. Returns on auth failure or cancel.""" + offset: int | None = None + backoff = _BACKOFF_INITIAL_SECONDS + while True: + try: + updates = await self._bot.get_updates( + offset=offset, + timeout=_LONG_POLL_TIMEOUT_SECONDS, + allowed_updates=["message", "callback_query"], + ) + backoff = _BACKOFF_INITIAL_SECONDS + except TelegramUnauthorizedError as exc: + logger.error("Telegram rejected the bot token: %s", exc) + self.auth_failed = True + return + except TelegramAPIError as exc: + logger.warning( + "Telegram API error in long-poll (retrying in %.1fs): %s", + backoff, exc, + ) + await asyncio.sleep(backoff) + backoff = min(backoff * 2, _BACKOFF_CAP_SECONDS) + continue + except asyncio.CancelledError: + raise + except Exception: # pragma: no cover - defensive + logger.exception( + "Unexpected error in long-poll (retrying in %.1fs)", backoff, + ) + await asyncio.sleep(backoff) + backoff = min(backoff * 2, _BACKOFF_CAP_SECONDS) + continue + + for update in updates: + offset = update.update_id + 1 + self._handle_update(update) + + self._maybe_flush_drop_log() + + def _handle_update(self, update: Any) -> None: + """Filter and enqueue a single update.""" + payload = update_to_dict(update) + if payload is None: + return + if payload["from_user_id"] not in self._allowed: + self._drop_counts[payload["from_user_id"]] = ( + self._drop_counts.get(payload["from_user_id"], 0) + 1 + ) + return + self.queue.put_nowait(payload) + + def _maybe_flush_drop_log(self) -> None: + """Emit aggregate drop counts once per window.""" + now = time.monotonic() + if now - self._drop_window_started_at < _DROP_LOG_INTERVAL_SECONDS: + return + if self._drop_counts: + for user_id, count in sorted(self._drop_counts.items()): + logger.warning( + "dropped %d unauthorized message(s) from user_id=%d " + "in the last %.0fs", + count, user_id, _DROP_LOG_INTERVAL_SECONDS, + ) + self._drop_counts.clear() + self._drop_window_started_at = now diff --git a/integrations/brokers/telegram_broker/_verbs.py b/integrations/brokers/telegram_broker/_verbs.py new file mode 100644 index 00000000..b0ceecd9 --- /dev/null +++ b/integrations/brokers/telegram_broker/_verbs.py @@ -0,0 +1,298 @@ +"""Verb dispatcher for the telegram broker. + +Bridges the RPC frame layer to aiogram's typed Bot methods. Same shape as +the email broker's dispatcher: a ``_VERB_REQUIREMENT`` table gates each +verb by ``(Capability, min Access)``, and the handler table is allowed +to omit verbs that haven't been implemented yet — those return BAD_REQUEST. +""" + +from __future__ import annotations + +import asyncio +from collections.abc import Awaitable, Callable +from pathlib import Path +from typing import Any + +from aiogram import Bot +from aiogram.exceptions import TelegramAPIError +from aiogram.types import FSInputFile, InlineKeyboardButton, InlineKeyboardMarkup + +from integrations._rpc import RpcError +from integrations.brokers.telegram_broker._updates import UpdatePump +from integrations.permissions import Access, Capability, Permissions + +# Per-verb: (capability, minimum_access_required). +_VERB_REQUIREMENT: dict[str, tuple[Capability, Access]] = { + "get_me": (Capability.TELEGRAM, Access.READ), + "next_updates": (Capability.TELEGRAM, Access.READ), + "send_message": (Capability.TELEGRAM, Access.READ_WRITE), + "send_document": (Capability.TELEGRAM, Access.READ_WRITE), + "send_chat_action": (Capability.TELEGRAM, Access.READ_WRITE), + "answer_callback_query": (Capability.TELEGRAM, Access.READ_WRITE), + "edit_message_text": (Capability.TELEGRAM, Access.READ_WRITE), + "delete_message": (Capability.TELEGRAM, Access.READ_WRITE), +} + + +_Handler = Callable[[dict[str, Any]], Awaitable[dict[str, Any]]] + + +class VerbDispatcher: + """Route one RPC verb call to the right Bot method.""" + + def __init__( + self, + bot: Bot, + pump: UpdatePump, + *, + permissions: Permissions, + ) -> None: + self._bot = bot + self._pump = pump + self._permissions = permissions + self._handlers: dict[str, _Handler] = { + "get_me": self._handle_get_me, + "next_updates": self._handle_next_updates, + "send_message": self._handle_send_message, + "send_document": self._handle_send_document, + "send_chat_action": self._handle_send_chat_action, + "answer_callback_query": self._handle_answer_callback_query, + "edit_message_text": self._handle_edit_message_text, + "delete_message": self._handle_delete_message, + } + + async def dispatch(self, verb: str, args: dict[str, Any]) -> dict[str, Any]: + """Entry point called by the RPC layer for every incoming frame.""" + requirement = _VERB_REQUIREMENT.get(verb) + if requirement is None: + msg = f"unknown verb: {verb}" + raise RpcError("BAD_REQUEST", msg) + + cap, min_access = requirement + granted = self._permissions.get(cap, Access.OFF) + if granted < min_access: + msg = ( + f"verb {verb!r} requires {cap.value}:{min_access.name.lower()}, " + f"but this integration has {cap.value}:{granted.name.lower()}" + ) + raise RpcError("PERMISSION_DENIED", msg) + + handler = self._handlers.get(verb) + if handler is None: + msg = f"verb not implemented: {verb}" + raise RpcError("BAD_REQUEST", msg) + + return await handler(args) + + # --- handlers ----------------------------------------------------------- + + async def _handle_get_me(self, _args: dict[str, Any]) -> dict[str, Any]: + """``get_me`` → ``{id, username, first_name}``.""" + try: + me = await self._bot.get_me() + except TelegramAPIError as exc: + raise RpcError("INTERNAL", str(exc)) from exc + return { + "id": me.id, + "username": me.username, + "first_name": me.first_name, + } + + async def _handle_next_updates(self, args: dict[str, Any]) -> dict[str, Any]: + """``next_updates {timeout_ms}`` → ``{updates: [...]}``. + + Long-polls the internal queue: blocks up to ``timeout_ms`` for the + first update, then drains everything currently buffered before + returning. Empty list on timeout. + """ + timeout_ms = _require_int(args, "timeout_ms", default=25_000) + if timeout_ms < 0: + raise RpcError("BAD_REQUEST", "'timeout_ms' must be >= 0") + timeout_s = timeout_ms / 1000.0 + + updates: list[dict[str, Any]] = [] + try: + first = await asyncio.wait_for(self._pump.queue.get(), timeout=timeout_s) + updates.append(first) + except asyncio.TimeoutError: + return {"updates": []} + + # Drain the rest without blocking so the caller gets a full batch. + while True: + try: + updates.append(self._pump.queue.get_nowait()) + except asyncio.QueueEmpty: + break + return {"updates": updates} + + async def _handle_send_message(self, args: dict[str, Any]) -> dict[str, Any]: + """``send_message {chat_id, text, reply_to_message_id?, buttons?}`` → ``{message_id}``. + + ``buttons`` is an optional 2-D array of ``{text, data}`` dicts that + becomes an inline keyboard attached to the message. Each tap fires a + callback_query update with ``data`` matching what was sent. + """ + chat_id = _require_int(args, "chat_id") + text = _require_str(args, "text") + reply_to = args.get("reply_to_message_id") + if reply_to is not None and (isinstance(reply_to, bool) or not isinstance(reply_to, int)): + raise RpcError("BAD_REQUEST", "'reply_to_message_id' must be an integer") + reply_markup = _coerce_inline_keyboard(args.get("buttons")) + try: + msg = await self._bot.send_message( + chat_id=chat_id, + text=text, + reply_to_message_id=reply_to, + reply_markup=reply_markup, + ) + except TelegramAPIError as exc: + raise RpcError("INTERNAL", str(exc)) from exc + return {"message_id": msg.message_id} + + async def _handle_send_document(self, args: dict[str, Any]) -> dict[str, Any]: + """``send_document {chat_id, host_path, caption?, filename?}`` → ``{message_id}``. + + ``host_path`` is an absolute path the broker can read. The broker + currently shares the main app's filesystem and UID, so any path the + agent emits via ``FileOutputPayload`` is reachable directly. + """ + chat_id = _require_int(args, "chat_id") + host_path = _require_str(args, "host_path") + caption = args.get("caption") + if caption is not None and not isinstance(caption, str): + raise RpcError("BAD_REQUEST", "'caption' must be a string") + filename = args.get("filename") + if filename is not None and not isinstance(filename, str): + raise RpcError("BAD_REQUEST", "'filename' must be a string") + + path = Path(host_path) + if not path.is_file(): + raise RpcError("BAD_REQUEST", f"file not found: {host_path}") + + document = FSInputFile(str(path), filename=filename or path.name) + try: + msg = await self._bot.send_document( + chat_id=chat_id, + document=document, + caption=caption, + ) + except TelegramAPIError as exc: + raise RpcError("INTERNAL", str(exc)) from exc + return {"message_id": msg.message_id} + + async def _handle_send_chat_action(self, args: dict[str, Any]) -> dict[str, Any]: + """``send_chat_action {chat_id, action}`` → ``{}``. + + ``action`` is one of Telegram's chat-action values, e.g. ``"typing"``. + The indicator clears after ~5 seconds — callers that want it to + persist should re-send periodically. + """ + chat_id = _require_int(args, "chat_id") + action = _require_str(args, "action") + try: + await self._bot.send_chat_action(chat_id=chat_id, action=action) + except TelegramAPIError as exc: + raise RpcError("INTERNAL", str(exc)) from exc + return {} + + async def _handle_edit_message_text(self, args: dict[str, Any]) -> dict[str, Any]: + """``edit_message_text {chat_id, message_id, text}`` → ``{}``. + + Edits a previously sent message in place. The channel uses this for a + rolling status indicator that names the current activity rather than + the generic "typing..." chat action. + """ + chat_id = _require_int(args, "chat_id") + message_id = _require_int(args, "message_id") + text = _require_str(args, "text") + try: + await self._bot.edit_message_text( + chat_id=chat_id, message_id=message_id, text=text, + ) + except TelegramAPIError as exc: + raise RpcError("INTERNAL", str(exc)) from exc + return {} + + async def _handle_delete_message(self, args: dict[str, Any]) -> dict[str, Any]: + """``delete_message {chat_id, message_id}`` → ``{}``. + + Removes a previously sent message. Used to clear the rolling status + message once the turn ends so it doesn't clutter the chat history. + """ + chat_id = _require_int(args, "chat_id") + message_id = _require_int(args, "message_id") + try: + await self._bot.delete_message(chat_id=chat_id, message_id=message_id) + except TelegramAPIError as exc: + raise RpcError("INTERNAL", str(exc)) from exc + return {} + + async def _handle_answer_callback_query(self, args: dict[str, Any]) -> dict[str, Any]: + """``answer_callback_query {callback_id, text?}`` → ``{}``. + + Acknowledges a callback_query so Telegram dismisses the button-tap + loading spinner. Optional ``text`` shows a small toast to the user. + """ + callback_id = _require_str(args, "callback_id") + text = args.get("text") + if text is not None and not isinstance(text, str): + raise RpcError("BAD_REQUEST", "'text' must be a string") + try: + await self._bot.answer_callback_query( + callback_query_id=callback_id, + text=text, + ) + except TelegramAPIError as exc: + raise RpcError("INTERNAL", str(exc)) from exc + return {} + + +def _coerce_inline_keyboard(buttons: Any) -> InlineKeyboardMarkup | None: + """Turn a wire-format buttons grid into an aiogram InlineKeyboardMarkup. + + Wire format: ``[[{"text": "Label", "data": "payload"}, ...], ...]``. + Returns ``None`` when ``buttons`` is missing or empty. + """ + if buttons is None: + return None + if not isinstance(buttons, list): + raise RpcError("BAD_REQUEST", "'buttons' must be a 2-D array of {text, data} dicts") + rows: list[list[InlineKeyboardButton]] = [] + for row in buttons: + if not isinstance(row, list): + raise RpcError("BAD_REQUEST", "each 'buttons' row must be a list of dicts") + button_row: list[InlineKeyboardButton] = [] + for cell in row: + if not isinstance(cell, dict): + raise RpcError("BAD_REQUEST", "each button must be a {text, data} dict") + label = cell.get("text") + data = cell.get("data") + if not isinstance(label, str) or not label: + raise RpcError("BAD_REQUEST", "button 'text' must be a non-empty string") + if not isinstance(data, str) or not data: + raise RpcError("BAD_REQUEST", "button 'data' must be a non-empty string") + button_row.append(InlineKeyboardButton(text=label, callback_data=data)) + if button_row: + rows.append(button_row) + if not rows: + return None + return InlineKeyboardMarkup(inline_keyboard=rows) + + +def _require_str(args: dict[str, Any], key: str) -> str: + value = args.get(key) + if not isinstance(value, str) or not value: + raise RpcError("BAD_REQUEST", f"{key!r} required (non-empty string)") + return value + + +def _require_int(args: dict[str, Any], key: str, *, default: int | None = None) -> int: + if default is None: + value = args.get(key) + if isinstance(value, bool) or not isinstance(value, int): + raise RpcError("BAD_REQUEST", f"{key!r} required (integer)") + return value + value = args.get(key, default) + if isinstance(value, bool) or not isinstance(value, int): + raise RpcError("BAD_REQUEST", f"{key!r} must be an integer") + return value diff --git a/integrations/permissions.py b/integrations/permissions.py index f468dc28..282adabb 100644 --- a/integrations/permissions.py +++ b/integrations/permissions.py @@ -17,6 +17,7 @@ class Capability(StrEnum): DRIVE = "drive" CONTACTS = "contacts" LLM_PROXY = "llm_proxy" + TELEGRAM = "telegram" class Access(IntEnum): diff --git a/integrations/supervisor/_catalog.py b/integrations/supervisor/_catalog.py index 4de50b1e..221779c1 100644 --- a/integrations/supervisor/_catalog.py +++ b/integrations/supervisor/_catalog.py @@ -199,6 +199,18 @@ def resolve_capabilities(self, auth_blob: dict | None = None) -> dict[Capability ) +_TELEGRAM = CatalogEntry( + slug="telegram", + command=["python", "-m", "integrations.brokers.telegram_broker"], + capabilities={Capability.TELEGRAM: Access.READ_WRITE}, + static_env={}, + env_injection={ + "token": "TELEGRAM_BOT_TOKEN", + "allowed_user_ids": "TELEGRAM_ALLOWED_USER_IDS", + }, +) + + DEFAULT_CATALOG: dict[str, CatalogEntry] = { "icloud": _ICLOUD, "gmail": _GMAIL, @@ -207,6 +219,7 @@ def resolve_capabilities(self, auth_blob: dict | None = None) -> dict[Capability "llm_openrouter": _LLM_OPENROUTER, "llm_openai_compat": _LLM_OPENAI_COMPAT, "google_workspace": _GOOGLE_WORKSPACE, + "telegram": _TELEGRAM, } diff --git a/pyproject.toml b/pyproject.toml index 328a4033..311bfc82 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -23,6 +23,7 @@ dependencies = [ "google-auth>=2.30", "google-auth-oauthlib>=1.2", "google-api-python-client>=2.130", + "aiogram>=3.7", ] [project.optional-dependencies] @@ -148,6 +149,7 @@ build-backend = "setuptools.build_meta" # Explicitly include our flat-layout top-level packages include = [ "agents*", + "channels*", "config*", "migrations*", "models*", diff --git a/sdk/__init__.py b/sdk/__init__.py index 92c15d79..63f064a4 100644 --- a/sdk/__init__.py +++ b/sdk/__init__.py @@ -16,16 +16,30 @@ from .turn import run_turn from .providers import LLMRuntimeStats, llm_runtime_stats +# Imported last so that sdk.context, sdk.hooks, sdk.turn (without _executor) +# and sdk.providers are all fully loaded before the executor module — which +# pulls from all of them — is initialized. +from .turn._executor import ( + Conversation, + SystemPromptBuilder, + TurnExecutor, + TurnPersistence, +) + __all__ = [ "BudgetGuard", "ContextHook", "ContextManager", + "Conversation", "ConversationHistory", "LLMRuntimeStats", "LoggingHook", "LoopDetector", "PersistenceHook", "StopHook", + "SystemPromptBuilder", + "TurnExecutor", + "TurnPersistence", "default_hooks", "llm_runtime_stats", "run_turn", diff --git a/sdk/tools/_spawn_agent.py b/sdk/tools/_spawn_agent.py index 3dda716a..c925c614 100644 --- a/sdk/tools/_spawn_agent.py +++ b/sdk/tools/_spawn_agent.py @@ -8,12 +8,12 @@ from rich.text import Text from agents import AgentProfile, build_agent, get_agent_profile -from sdk.context import ContextManager, ConversationHistory, LLMCompactionStrategy -from sdk.events import agent_span -from sdk.hooks import PersistenceHook, default_hooks +from sdk import Conversation, TurnExecutor +from sdk.context import ConversationHistory +from sdk.events._models import ContentPayload from sdk.skills import AgentState, get_skill, list_skills from sdk.tools._core import get_core_tools -from sdk.turn import StopRequestedError, get_conversation_id, run_turn +from sdk.turn import StopRequestedError, get_conversation_id logger = logging.getLogger(__name__) @@ -149,7 +149,11 @@ async def spawn_agent( _log_spawn_error(agent_name, msg) return msg - agent_state = AgentState(await get_core_tools()) + # Validate skills up-front so a missing one fails the spawn synchronously + # with a clear message rather than mid-turn with a generic warning. The + # executor itself also tolerates missing skills (logs + skips), but the + # tool's "tell the calling LLM what's wrong" surface beats that. + state = AgentState(await get_core_tools()) for skill_name in agent_profile.skills: skill = get_skill(skill_name) if skill is None: @@ -160,9 +164,9 @@ async def spawn_agent( ) _log_spawn_error(agent_name, msg) return msg - agent_state.add(skill) + state.add(skill) - agent = build_agent(agent_profile, tools=agent_state.tools, name=agent_name) + agent = build_agent(agent_profile, tools=state.tools, name=agent_name) logger.info( "Spawning sub-agent '%s' (profile=%s, max_iter=%d, instruction=%.100s)", @@ -170,58 +174,36 @@ async def spawn_agent( ) _log_spawn(agent_name, agent_profile, instructions) - async with agent_span( - agent_name, - instruction=instructions, - agent_state=agent_state, - profile_name=agent_profile.name, - ): - conv_id = get_conversation_id() or "default" - short_id = _uuid.uuid4().hex[:8] - instance_id = f"{conv_id}/{agent_name}_{short_id}" - history = ConversationHistory( - [ - {"role": "system", "content": agent.instruction}, - {"role": "user", "content": instructions}, - ], - instance_id=instance_id, - ) + parent_conv_id = get_conversation_id() or "default" + short_id = _uuid.uuid4().hex[:8] + instance_id = f"{parent_conv_id}/{agent_name}_{short_id}" + conversation = Conversation( + id=instance_id, + history=ConversationHistory(instance_id=instance_id), + ) - ctx_manager = ContextManager( - history=history, - agent_state=agent_state, - context_limit=agent.context_window, - agent_name=agent.name, - strategies=[ - LLMCompactionStrategy(threshold=agent.compaction_threshold), - ], - ) - hooks = default_hooks( - agent, - max_iterations=agent.max_iterations, - ctx_manager=ctx_manager, - ) - hooks.append(PersistenceHook( - conversation_id=conv_id, - history=history, + accumulated: list[str] = [] + try: + async for event in TurnExecutor().execute( + conversation=conversation, + agent=agent, + user_content=instructions, + preloaded_skills=agent_profile.skills, + profile_name=agent_profile.name, sub_agent_name=agent_name, sub_agent_id=short_id, - )) - - try: - result_text = await run_turn( - history=history, - agent=agent, - hooks=hooks, - ) - except StopRequestedError: - logger.info("Spawned agent '%s' stopped by user request", agent_name) - raise - except Exception as exc: - _log_spawn_error(agent_name, str(exc)) - logger.exception("Unexpected error in spawned agent '%s'", agent_name) - raise - - result = (result_text or "").strip() + ): + if event.type == "content" and isinstance(event.payload, ContentPayload): + if event.payload.content: + accumulated.append(event.payload.content) + except StopRequestedError: + logger.info("Spawned agent '%s' stopped by user request", agent_name) + raise + except Exception as exc: + _log_spawn_error(agent_name, str(exc)) + logger.exception("Unexpected error in spawned agent '%s'", agent_name) + raise + + result = "".join(accumulated).strip() _log_spawn_complete(agent_name, result) return result diff --git a/sdk/turn/__init__.py b/sdk/turn/__init__.py index 636abda3..3795d738 100644 --- a/sdk/turn/__init__.py +++ b/sdk/turn/__init__.py @@ -4,6 +4,10 @@ - ``run_turn``: Async function driving the chat/tool loop. - ``turn_scope``: Async context manager for conversation turn lifecycle. - Stop/nudge signaling utilities for user-initiated control. + +The high-level ``TurnExecutor`` and ``Conversation`` are defined in +``sdk.turn._executor`` and re-exported from the top-level ``sdk`` package. +They are not re-exported here to avoid a load-order cycle with ``sdk.context``. """ from ._execution import ToolLoopError, run_turn diff --git a/sdk/turn/_executor.py b/sdk/turn/_executor.py new file mode 100644 index 00000000..0d265261 --- /dev/null +++ b/sdk/turn/_executor.py @@ -0,0 +1,306 @@ +"""High-level turn executor: wraps the agent loop with context management, +skill state, hooks, and optional caller-supplied persistence and prompt +augmentation. + +The caller builds the ``Agent``, creates a ``Conversation``, optionally +provides a ``TurnPersistence`` and a ``SystemPromptBuilder``, then iterates +the yielded events. +""" + +from __future__ import annotations + +import asyncio +import logging +from collections.abc import AsyncGenerator, Callable, Iterable, Sequence +from contextlib import suppress +from dataclasses import dataclass +from typing import Protocol + +from agents.types import Agent +from sdk.context._history import ConversationHistory +from sdk.context._manager import ContextManager +from sdk.context._strategy import LLMCompactionStrategy +from sdk.events._context import agent_span, get_current_dispatcher +from sdk.events._models import AgentEvent +from sdk.hooks._agent_event_buffer import AgentEventBufferHook +from sdk.hooks._default import default_hooks +from sdk.hooks._persistence import PersistenceHook +from sdk.skills import AgentState, get_skill +from sdk.tools._core import get_core_tools +from sdk.turn._execution import run_turn +from sdk.turn._turn import StopRequestedError, turn_scope + +logger = logging.getLogger(__name__) + +# Background tasks held to prevent GC; cleared via done callback. +_background_tasks: set[asyncio.Task] = set() + + +@dataclass +class Conversation: + """Per-conversation state owned by the caller. + + Attributes: + id: Unique conversation identifier. + history: The conversation history. + """ + + id: str + history: ConversationHistory + + +class TurnPersistence(Protocol): + """Optional persistence hooks invoked by ``TurnExecutor``. + + Channels that don't want persistence pass ``None`` instead of an + implementation. Implementations are typically thin wrappers over the + application's on-disk store. + """ + + def load_skills(self, conversation_id: str) -> Iterable[str]: + """Return persisted skill names to restore for this conversation.""" + + def save_skills(self, conversation_id: str, skills: Iterable[str]) -> None: + """Persist the set of currently-loaded skill names.""" + + def save_events( + self, + conversation_id: str, + events: list[AgentEvent], + ) -> None: + """Persist agent lifecycle events captured during the turn.""" + + async def on_new_conversation( + self, + conversation_id: str, + first_message: str, + ) -> None: + """Hook fired once when a conversation runs its first turn. + + Typical use: generate and persist a conversation title. + """ + + +SystemPromptBuilder = Callable[[], str] +"""Caller-supplied function returning the base system prompt for this turn. + +Called fresh each turn so the caller can inject up-to-date state (e.g. a +memory block). If absent, the agent's ``instruction`` is used as-is. +""" + + +class TurnExecutor: + """Executes a single agent turn with setup, hooks, and persistence. + + Callers build the ``Agent`` themselves and supply optional persistence + and prompt-building injection points. The executor is stateless and + safe to share across conversations. + """ + + async def execute( + self, + *, + conversation: Conversation, + agent: Agent, + user_content: str, + is_new_conversation: bool = False, + preloaded_skills: Sequence[str] = (), + persistence: TurnPersistence | None = None, + build_system_prompt: SystemPromptBuilder | None = None, + profile_name: str | None = None, + sub_agent_name: str | None = None, + sub_agent_id: str | None = None, + ) -> AsyncGenerator[AgentEvent, None]: + """Run a single turn and yield events. + + Args: + conversation: Per-conversation state. + agent: The fully-constructed Agent to run. + user_content: The user's message, already augmented if needed. + is_new_conversation: True if this is the conversation's first + turn. Triggers ``persistence.on_new_conversation`` when set; + a no-op without a persistence implementation. + preloaded_skills: Skill names to install before turn start + (e.g. profile-attached skills). + persistence: Optional persistence bundle. ``None`` skips all + persistence calls. + build_system_prompt: Optional callable returning the base system + prompt; re-evaluated each turn so callers can inject + up-to-date state (e.g. a memory block). Falls back to + ``agent.instruction`` when ``None``. + profile_name: Optional metadata threaded through ``agent_span``. + sub_agent_name: When this turn is a sub-agent invocation, the + short uppercase agent name. Forwarded to ``PersistenceHook`` + so the turn writes to the conversation's ``sub_agents/`` + directory instead of overwriting the main history. + sub_agent_id: When this turn is a sub-agent invocation, a short + unique id (UUID hex prefix). Pairs with ``sub_agent_name``. + + Yields: + AgentEvent: Events emitted by the agent during the turn. + """ + conv_id = conversation.id + logger.info( + "Turn started: conv=%s agent=%s message=%.80s", + conv_id, + agent.name, + user_content, + ) + + # Fresh AgentState each turn; pre-load profile skills then restore + # any persisted skills from a previous turn. + agent_state = AgentState(await get_core_tools() + agent.tools) + for skill_name in preloaded_skills: + skill = get_skill(skill_name) + if skill is None: + logger.warning( + "Preloaded skill '%s' not registered; skipping", skill_name, + ) + continue + agent_state.add(skill) + logger.info("Preloaded skill '%s' for conv=%s", skill_name, conv_id) + + if persistence is not None: + for skill_name in persistence.load_skills(conv_id): + if skill_name in agent_state.loaded_skill_names: + continue + skill = get_skill(skill_name) + if skill is None: + logger.warning( + "Persisted skill '%s' for conv=%s not found in registry; skipping", + skill_name, + conv_id, + ) + continue + agent_state.add(skill) + logger.info("Restored skill '%s' for conv=%s", skill_name, conv_id) + + # Fresh ContextManager per turn — it borrows the live agent_state so + # the token estimate reflects the current tool set, and the strategy + # threshold tracks the agent's compaction setting. + ctx_manager = ContextManager( + history=conversation.history, + agent_state=agent_state, + context_limit=agent.context_window, + agent_name=agent.name, + strategies=[ + LLMCompactionStrategy(threshold=agent.compaction_threshold), + ], + ) + + # Bridge published events through a queue so we can yield them + # regardless of how the caller is iterating. + queue: asyncio.Queue[AgentEvent | None] = asyncio.Queue() + + async def _queue_handler(evt: AgentEvent) -> None: + try: + await queue.put(evt) + except Exception: # pragma: no cover - defensive + logger.exception("Failed to enqueue AgentEvent in TurnExecutor") + + async def _producer() -> None: + try: + async with turn_scope( + handler=_queue_handler, + conversation_id=conv_id, + ): + event_buffer = AgentEventBufferHook() + dispatcher = get_current_dispatcher() + if dispatcher: + dispatcher.subscribe(event_buffer.handle_event) + + async with agent_span( + agent.name, + instruction=user_content, + agent_state=agent_state, + profile_name=profile_name, + ): + conversation.history.append( + {"role": "user", "content": user_content}, + ) + + base_prompt = ( + build_system_prompt() + if build_system_prompt is not None + else agent.instruction + ) + skill_prompt = agent_state.build_skill_prompt() + full_prompt = ( + f"{base_prompt}\n{skill_prompt}" + if skill_prompt + else base_prompt + ) + conversation.history.set_system_message(full_prompt) + + hooks = default_hooks( + agent, + max_iterations=agent.max_iterations, + ctx_manager=ctx_manager, + ) + hooks.append( + PersistenceHook( + conversation_id=conv_id, + history=conversation.history, + sub_agent_name=sub_agent_name, + sub_agent_id=sub_agent_id, + ), + ) + + with suppress(StopRequestedError): + await run_turn( + history=conversation.history, + agent=agent, + hooks=hooks, + ) + + if persistence is not None and agent_state.loaded_skill_names: + try: + persistence.save_skills( + conv_id, + agent_state.loaded_skill_names, + ) + except Exception: + logger.exception( + "Failed to save loaded skills for '%s'", conv_id, + ) + + # Yield once so synchronous handlers registered via + # call_soon get to run before we read the buffer. + await asyncio.sleep(0) + + if persistence is not None: + buffered_events = event_buffer.get_events() + if buffered_events: + try: + persistence.save_events(conv_id, buffered_events) + logger.info( + "Saved %d agent events for conv=%s", + len(buffered_events), + conv_id, + ) + except Exception: + logger.exception( + "Failed to save agent events for '%s'", conv_id, + ) + + if is_new_conversation and persistence is not None: + task = asyncio.create_task( + persistence.on_new_conversation(conv_id, user_content), + ) + _background_tasks.add(task) + task.add_done_callback(_background_tasks.discard) + finally: + await queue.put(None) + + producer_task = asyncio.create_task(_producer()) + try: + while True: + item = await queue.get() + if item is None: + break + yield item + finally: + if not producer_task.done(): + producer_task.cancel() + with suppress(Exception): + await producer_task diff --git a/server/_integrations_routes.py b/server/_integrations_routes.py index 5d5488d1..806abab4 100644 --- a/server/_integrations_routes.py +++ b/server/_integrations_routes.py @@ -115,20 +115,26 @@ async def handle_add_integration(request: web.Request) -> web.Response: status=400, ) - # user_suffix is derived from auth_blob.email — clients never set it. - # Keeps integration IDs deterministic and out of the user's mental model. - # LLM integrations are singletons — no suffix, so the ID is just the slug - # and the socket path matches what the provider factory expects. + # Integration ID composition. Three shapes: + # - LLM integrations are singletons — ID is just the slug, no + # user_suffix, no per-capability permissions (the supervisor's add + # verb still requires the dict though). + # - Slugs whose wizard collects an email derive user_suffix from it so + # IDs stay deterministic and out of the user's mental model. + # - Slugs without an email (bot_token, oauth_device) submit + # user_suffix directly from the wizard; the supervisor enforces the + # [a-z0-9_-]{1,48} format. slug = body.get("slug", "") if slug.startswith("llm_"): - # LLM integrations have no per-capability permissions (no email, calendar, - # etc.) but the supervisor's add verb requires the field as a dict. if "permissions" not in body: body["permissions"] = {} - else: + elif not body.get("user_suffix"): derived = _derive_suffix_from_email(body.get("auth_blob")) if not derived: - return error_response("BAD_REQUEST", "Email address is required.") + return error_response( + "BAD_REQUEST", + "An email address or user_suffix is required.", + ) body["user_suffix"] = derived try: diff --git a/server/aiohttp_app.py b/server/aiohttp_app.py index 1ed809b9..c41ac83d 100644 --- a/server/aiohttp_app.py +++ b/server/aiohttp_app.py @@ -44,6 +44,7 @@ from server._setup_routes import register_setup_routes from server._task_routes import register_task_routes from server.message_handler import handle_user_message, resume_conversation +from channels.telegram import TelegramChannel from tools.custom_tools.registry import delete_tool, list_tools from tools.desktop._exec import DesktopExecError from tools.desktop._lifecycle import start_desktop @@ -463,6 +464,7 @@ def create_app(*, client_max_size: int = 10 * 1024**2) -> web.Application: # ``app["ready"]`` and then initializes everything that needed to wait. app.on_startup.append(_start_deferred_subsystems) app.on_cleanup.append(_stop_deferred_subsystems) + app.on_cleanup.append(_stop_telegram_bot) return app @@ -584,6 +586,7 @@ async def _deferred() -> None: await app["ready"].wait() logger.info("Ready — starting deferred subsystems") await _init_task_runner(app) + await _init_telegram_bot(app) except asyncio.CancelledError: raise except Exception: @@ -605,7 +608,10 @@ async def _init_task_runner(app: web.Application) -> None: notifier = None if config.goals.notifications.enabled: - notifier = TelegramNotifier(config.goals.notifications) + notifier = TelegramNotifier( + config.goals.notifications, + app_sock_path=Path(config.integrations.app_sock_path), + ) if not notifier.enabled: notifier = None @@ -624,4 +630,26 @@ async def _stop_deferred_subsystems(app: web.Application) -> None: await runner.stop() +async def _init_telegram_bot(app: web.Application) -> None: + """Start the Telegram channel if a telegram integration is registered. + + The channel auto-discovers the integration via the supervisor — no + config flag, no integration_id field. Adding/removing a Telegram + integration in the wizard is the only enable/disable knob. + """ + config = load_config() + runner = TelegramChannel( + app_sock_path=Path(config.integrations.app_sock_path), + ) + await runner.start() + app["telegram_bot_runner"] = runner + + +async def _stop_telegram_bot(app: web.Application) -> None: + """Stop the Telegram bot runner if present.""" + runner: TelegramChannel | None = app.get("telegram_bot_runner") + if runner: + await runner.stop() + + __all__ = ["create_app"] diff --git a/server/message_handler.py b/server/message_handler.py index 6eac6640..3149553d 100644 --- a/server/message_handler.py +++ b/server/message_handler.py @@ -1,10 +1,8 @@ """Message handler for user prompts.""" -import asyncio import logging from collections import OrderedDict -from collections.abc import AsyncGenerator, Callable, Sequence -from contextlib import suppress +from collections.abc import AsyncGenerator, Sequence from rich.console import Console from rich.panel import Panel @@ -15,36 +13,20 @@ build_agent, get_agent_profile, ) -from agents.types import Agent, Data -from conversations import ( - generate_conversation_title, - load_conversation_history, - load_loaded_skills, - save_agent_events, - save_conversation_title, - save_loaded_skills, -) -from sdk import ( - PersistenceHook, - default_hooks, - run_turn, -) -from sdk.context import ContextManager, ConversationHistory, LLMCompactionStrategy +from agents.types import Data +from conversations import DiskTurnPersistence, load_conversation_history +from sdk import Conversation, TurnExecutor +from sdk.context import ConversationHistory from sdk.events import ( AgentEvent, ContentPayload, TurnEndPayload, - agent_span, - get_current_dispatcher, ) -from sdk.hooks._agent_event_buffer import AgentEventBufferHook -from sdk.skills import AgentState, get_skill -from sdk.tools._core import get_core_tools -from sdk.turn import is_turn_active, turn_scope -from sdk.turn._turn import StopRequestedError +from sdk.turn import is_turn_active from tools.browser.core import release_agent_browser -from tools.memory import load_memory +from tools.memory import forget, memory_prompt_block, remember from tools.virtual_computer.receive_file import receive_attachment +from tools.virtual_computer.run_bash_cmd import run_bash_cmd logger = logging.getLogger(__name__) _console = Console(stderr=True) @@ -105,14 +87,15 @@ def _log_turn_start(profile: AgentProfile) -> None: # state is authoritative; an evicted entry is rehydrated from disk on # next access. _MAX_CACHED_CONVERSATIONS = 25 -_conversations: OrderedDict[str, ConversationHistory] = OrderedDict() +_conversations: OrderedDict[str, Conversation] = OrderedDict() -# Track background tasks to avoid garbage collection (RUF006) -_background_tasks: set[asyncio.Task] = set() +# Shared turn executor — stateless, safe to reuse across conversations. +_turn_executor = TurnExecutor() +_persistence = DiskTurnPersistence() -async def _get_conversation(conversation_id: str) -> tuple[ConversationHistory, bool]: - """Return ``(history, is_new)`` for the given ID, creating it if needed. +async def _get_conversation(conversation_id: str) -> tuple[Conversation, bool]: + """Return ``(conversation, is_new)`` for the given ID, creating it if needed. ``is_new`` is True only when the conversation has no in-memory entry AND no on-disk history — a genuine first-time use. On any cache miss @@ -135,7 +118,10 @@ async def _get_conversation(conversation_id: str) -> tuple[ConversationHistory, is_new = persisted is None if is_new: logger.info("Creating new conversation %s", conversation_id) - _conversations[conversation_id] = ConversationHistory(persisted, instance_id=conversation_id) + _conversations[conversation_id] = Conversation( + id=conversation_id, + history=ConversationHistory(persisted, instance_id=conversation_id), + ) await _evict_lru_conversation(exclude=conversation_id) return _conversations[conversation_id], is_new @@ -181,29 +167,15 @@ async def resume_conversation(conversation_id: str) -> list[dict] | None: if messages is None: return None - _conversations[conversation_id] = ConversationHistory(messages, instance_id=conversation_id) + _conversations[conversation_id] = Conversation( + id=conversation_id, + history=ConversationHistory(messages, instance_id=conversation_id), + ) _conversations.move_to_end(conversation_id) await _evict_lru_conversation(exclude=conversation_id) return messages -def _refresh_system_message(history: ConversationHistory, system_prompt: str) -> None: - """Re-inserts the system message at the start of history with up-to-date memory. - - Called before each model invocation so any memories stored during the previous - turn are visible immediately. - """ - instruction = system_prompt - memory = load_memory() - if memory: - lines = "\n".join(f" {k}: {e.value}" for k, e in memory.items()) - sep = "─" * 64 - memory_block = f"\n── Memory (persisted across sessions) ──────────────────────────\n{lines}\n{sep}\n" - instruction = memory_block + instruction - - history.set_system_message(instruction) - - def _augment_message_with_attachments(message: str, data: Sequence[Data]) -> str: """Write attachments to the virtual computer and return an augmented message.""" file_lines = [] @@ -220,144 +192,6 @@ def _augment_message_with_attachments(message: str, data: Sequence[Data]) -> str return f"{message}\n\n[Attached files written to virtual computer]\n{files_block}" -def _build_agent_from_profile(profile: AgentProfile) -> Agent: - """Construct an Agent from an AgentProfile.""" - from tools.memory import forget, remember - from tools.virtual_computer.run_bash_cmd import run_bash_cmd - - return build_agent(profile, tools=[run_bash_cmd, remember, forget]) - - -async def _run_turn( - *, - history: ConversationHistory, - active_agent: Agent, - profile: AgentProfile, - user_content: str, - conversation_id: str, - handler: Callable[[AgentEvent], object], - is_new_conversation: bool = False, -) -> None: - """Execute a single conversation turn: model calls, tool execution, persistence.""" - logger.info( - "Turn started: conv=%s agent=%s message=%.80s", - conversation_id, - active_agent.name, - user_content, - ) - _log_turn_start(profile) - - conv_id = conversation_id - - # Fresh AgentState each turn, restored from persisted skill names. - # Pre-load skills from the profile. - agent_state = AgentState(await get_core_tools() + active_agent.tools) - for skill_name in profile.skills: - skill = get_skill(skill_name) - if skill is None: - logger.warning("Profile skill '%s' not registered; skipping", skill_name) - continue - agent_state.add(skill) - logger.info("Pre-loaded profile skill '%s' for conv=%s", skill_name, conv_id) - for skill_name in load_loaded_skills(conv_id): - if skill_name in agent_state.loaded_skill_names: - continue - skill = get_skill(skill_name) - if skill is None: - logger.warning( - "Persisted skill '%s' for conv=%s was not found in the skills registry; skipping", - skill_name, - conv_id, - ) - continue - agent_state.add(skill) - logger.info("Restored skill '%s' for conv=%s", skill_name, conv_id) - - ctx_manager = ContextManager( - history=history, - agent_state=agent_state, - context_limit=active_agent.context_window, - agent_name=active_agent.name, - strategies=[ - LLMCompactionStrategy(threshold=active_agent.compaction_threshold), - ], - ) - - async with turn_scope(handler=handler, conversation_id=conversation_id): - # Subscribe event buffer to capture agent lifecycle/preview events - event_buffer = AgentEventBufferHook() - dispatcher = get_current_dispatcher() - if dispatcher: - dispatcher.subscribe(event_buffer.handle_event) - - async with agent_span( - active_agent.name, instruction=user_content, agent_state=agent_state, profile_name=profile.name - ): - history.append({"role": "user", "content": user_content}) - # Build full system prompt: profile prompt + loaded skill prompts - full_prompt = active_agent.instruction - skill_prompt = agent_state.build_skill_prompt() - if skill_prompt: - full_prompt = full_prompt + "\n" + skill_prompt - _refresh_system_message(history, full_prompt) - - hooks = default_hooks( - active_agent, - max_iterations=active_agent.max_iterations, - ctx_manager=ctx_manager, - ) - - hooks.append( - PersistenceHook( - conversation_id=conv_id, - history=history, - ) - ) - - with suppress(StopRequestedError): - await run_turn( - history=history, - agent=active_agent, - hooks=hooks, - ) - - # Persist loaded skills so they survive across turns and restarts - if agent_state.loaded_skill_names: - try: - save_loaded_skills(conv_id, agent_state.loaded_skill_names) - except Exception: - logger.exception("Failed to save loaded skills for '%s'", conv_id) - - # Yield to event loop so call_soon callbacks (sync event handlers) - # have a chance to run before we read the buffer - await asyncio.sleep(0) - - # Save agent events after the turn (outside agent_span so completion is captured) - buffered_events = event_buffer.get_events() - if buffered_events: - try: - save_agent_events(conv_id, buffered_events) - logger.info("Saved %d agent events for conv=%s", len(buffered_events), conv_id) - except Exception: - logger.exception("Failed to save agent events for '%s'", conv_id) - - # Generate a title for new conversations after the first successful turn - if is_new_conversation and conversation_id: - task = asyncio.create_task(_generate_title(conversation_id, user_content)) - _background_tasks.add(task) - task.add_done_callback(_background_tasks.discard) - - -async def _generate_title(conversation_id: str, first_message: str) -> None: - """Generate and save a title for a new conversation.""" - try: - title = await generate_conversation_title(first_message) - save_conversation_title(conversation_id, title) - logger.info("Generated title for conversation %s: %r", conversation_id, title) - except Exception: - logger.exception("Failed to generate title for conversation %s", conversation_id) - - async def handle_user_message( message: str, data: Sequence[Data] | None = None, @@ -379,7 +213,7 @@ async def handle_user_message( if not conversation_id: msg = "conversation_id is required" raise ValueError(msg) - history, is_new_conversation = await _get_conversation(conversation_id) + conversation, is_new_conversation = await _get_conversation(conversation_id) user_content = message if data: @@ -397,45 +231,22 @@ async def handle_user_message( msg = "No model configured. Complete the setup wizard to select a model." raise ValueError(msg) - try: - # Bridge published events via a queue so we can stream them to the caller. - queue: asyncio.Queue[AgentEvent | None] = asyncio.Queue() - - async def _queue_handler(evt: AgentEvent) -> None: - try: - await queue.put(evt) - except Exception: # pragma: no cover - defensive logging - logger.exception("Failed to enqueue AgentEvent in message handler") - - active_agent = _build_agent_from_profile(profile) - - async def _producer() -> None: - try: - await _run_turn( - history=history, - active_agent=active_agent, - profile=profile, - user_content=user_content, - conversation_id=conversation_id, - handler=_queue_handler, - is_new_conversation=is_new_conversation, - ) - finally: - await queue.put(None) - - producer_task = asyncio.create_task(_producer()) - try: - while True: - item = await queue.get() - if item is None: - break - yield item - finally: - if not producer_task.done(): - producer_task.cancel() - with suppress(Exception): - await producer_task + _log_turn_start(profile) + + agent = build_agent(profile, tools=[run_bash_cmd, remember, forget]) + try: + async for event in _turn_executor.execute( + conversation=conversation, + agent=agent, + user_content=user_content, + is_new_conversation=is_new_conversation, + preloaded_skills=profile.skills, + persistence=_persistence, + build_system_prompt=lambda: memory_prompt_block() + agent.instruction, + profile_name=profile.name, + ): + yield event except Exception: logger.exception("Error handling user message") yield AgentEvent( diff --git a/server/ui/src/components/integrations/add-wizard/AddIntegrationModal.jsx b/server/ui/src/components/integrations/add-wizard/AddIntegrationModal.jsx index 929d99b6..9885dbeb 100644 --- a/server/ui/src/components/integrations/add-wizard/AddIntegrationModal.jsx +++ b/server/ui/src/components/integrations/add-wizard/AddIntegrationModal.jsx @@ -1,19 +1,30 @@ import { useEffect, useState } from 'react'; import styles from './add-wizard.module.css'; -import { slugifyEmail } from './providers.js'; +import { slugifyEmail, slugifyLabel } from './providers.js'; import { ProviderPicker, SuccessScreen } from './SharedSteps.jsx'; import { ExplainerStep, CredentialsStep, VerifyingStep } from './AppPasswordSteps.jsx'; +import { + ExplainerStep as BotExplainerStep, + CredentialsStep as BotCredentialsStep, + VerifyingStep as BotVerifyingStep, +} from './BotTokenSteps.jsx'; import { OauthCapabilitiesStep, OauthGcpSetupStep, OauthRedirectStep } from './OAuthSteps.jsx'; export default function AddIntegrationModal({ onClose, onAdded }) { const [provider, setProvider] = useState(null); const [step, setStep] = useState(1); const [form, setForm] = useState({ - email: '', - password: '', + // Shared label: '', permissions: {}, + // App-password flow + email: '', + password: '', + // Bot-token flow + token: '', + allowedUserIds: '', + instanceName: '', }); const [oauth, setOauth] = useState({ clientId: '', @@ -85,6 +96,59 @@ export default function AddIntegrationModal({ onClose, onAdded }) { } }; + const handleBotTokenSubmit = async () => { + setSubmitting(true); + setError(null); + setStep(3); + const token = form.token.trim().replace(/\s+/g, ''); + const allowedUserIds = form.allowedUserIds.trim(); + const instanceName = form.instanceName.trim() || 'personal'; + const userSuffix = slugifyLabel(instanceName); + if (!userSuffix) { + setError({code: 'BAD_REQUEST', message: 'Instance name produced an empty ID'}); + setSubmitting(false); + setStep(2); + return; + } + const label = form.label.trim() || `${provider.title} · ${instanceName}`; + try { + const resp = await fetch('/api/integrations', { + method: 'POST', + headers: {'Content-Type': 'application/json'}, + body: JSON.stringify({ + slug: provider.slug, + user_suffix: userSuffix, + label, + auth_blob: { + token, + allowed_user_ids: allowedUserIds, + }, + permissions: Object.fromEntries( + (provider.capabilities || []).map( + cap => [cap, form.permissions[cap] || 'rw'], + ), + ), + }), + }); + const body = await resp.json().catch(() => ({})); + if (!resp.ok) { + setError({ + code: body?.error?.code || 'ERROR', + message: body?.error?.message || `HTTP ${resp.status}`, + }); + setSubmitting(false); + setStep(2); + return; + } + setResult(body); + setSubmitting(false); + } catch (err) { + setError({code: 'NETWORK', message: err?.message || 'Request failed'}); + setSubmitting(false); + setStep(2); + } + }; + const handleOauthStart = async () => { setSubmitting(true); setError(null); @@ -236,7 +300,9 @@ export default function AddIntegrationModal({ onClose, onAdded }) { setResult(null); setStep(1); setForm({ - email: '', password: '', label: '', permissions: {}, + label: '', permissions: {}, + email: '', password: '', + token: '', allowedUserIds: '', instanceName: '', }); setOauth({ clientId: '', clientSecret: '', @@ -247,6 +313,26 @@ export default function AddIntegrationModal({ onClose, onAdded }) { }} onDone={() => { onAdded?.(); }} /> + ) : provider.authFlow === 'bot_token' ? ( + step === 1 ? ( + setProvider(null)} + onNext={() => setStep(2)} + /> + ) : step === 2 ? ( + setStep(1)} + onCancel={onClose} + onSubmit={handleBotTokenSubmit} + /> + ) : ( + + ) ) : provider.authFlow === 'oauth_device' ? ( step === 1 ? ( + +
+

Connect {provider.title}

+

+ You'll need a bot token from @BotFather and + {' '}your Telegram user ID from @userinfobot. +

+
+ +
+ + Token never leaves the broker process + + + User-ID allowlist enforced server-side + + + Revocable from @BotFather at any time + +
+
+
+
+ +
+ +
+
+ + ); +} + +export function CredentialsStep({ provider, form, setForm, error, onBack, onCancel, onSubmit }) { + const trimmedToken = form.token.trim(); + const trimmedAllowed = form.allowedUserIds.trim(); + const trimmedName = form.instanceName.trim(); + // user_suffix derives from the instance name. Empty defaults to "personal" + // — that's the most common single-bot case and removes a required field. + const effectiveName = trimmedName || 'personal'; + const userSuffix = slugifyLabel(effectiveName); + const canSubmit = trimmedToken && trimmedAllowed && userSuffix; + return ( + <> + +
+

Paste your bot details

+

+ Create a bot with @BotFather (run /newbot) and copy + its token. Get your user ID from @userinfobot. +

+
+ + + Open @BotFather + + t.me/BotFather + + + + Open @userinfobot (get your user ID) + + t.me/userinfobot + + +
+ + setForm(f => ({ ...f, token: e.target.value }))} + data-testid="wizard-bot-token" + /> + + Pasted verbatim from BotFather — whitespace is trimmed. + +
+ +
+ + setForm(f => ({ ...f, allowedUserIds: e.target.value }))} + data-testid="wizard-allowed-user-ids" + /> + + Comma-separated numeric IDs. Only senders on this list can + drive the bot — everyone else is silently dropped. + +
+ +
+ + setForm(f => ({ ...f, instanceName: e.target.value }))} + data-testid="wizard-instance-name" + /> + + How this Telegram setup is identified. Becomes + telegram_{userSuffix || 'personal'} in the + integration list. Leave blank for "personal". + +
+ +
Permissions
+
+ {(provider.capabilities || []).map(cap => ( +
+ + {CAP_LABELS[cap] || cap} + + +
+ ))} +
+ + + {error && (() => { + const copy = errorCopy(error, provider); + return ( + + ); + })()} +
+
+
+ +
+ + +
+
+ + ); +} + +export function VerifyingStep() { + return ( + <> + +
+

Connecting…

+

This usually takes a few seconds.

+
+
+
+
+ +
+
Securing your credentials
+
done
+
+
+
+ +
+
Authenticating with Telegram
+
…
+
+
+
+
+
+ +
+ + +
+
+ + ); +} diff --git a/server/ui/src/components/integrations/add-wizard/providers.js b/server/ui/src/components/integrations/add-wizard/providers.js index f624696f..f34ef3c4 100644 --- a/server/ui/src/components/integrations/add-wizard/providers.js +++ b/server/ui/src/components/integrations/add-wizard/providers.js @@ -25,6 +25,18 @@ export const PROVIDERS = [ emailPlaceholder: 'you@gmail.com', capabilities: ['email'], }, + { + slug: 'telegram', + authFlow: 'bot_token', + category: 'Messaging', + title: 'Telegram', + description: 'Bidirectional bot + push notifications', + icon: 'bi-telegram', + vendor: 'Telegram', + botFatherUrl: 'https://t.me/BotFather', + userInfoBotUrl: 'https://t.me/userinfobot', + capabilities: ['telegram'], + }, { slug: 'google_workspace', authFlow: 'oauth_device', @@ -74,6 +86,7 @@ export const PROVIDERS = [ export function errorCopy(error, provider) { const vendor = provider?.vendor ?? provider?.title ?? 'this provider'; const isOauth = provider?.authFlow === 'oauth_device'; + const isBotToken = provider?.authFlow === 'bot_token'; switch (error?.code) { case 'AUTH': if (isOauth) { @@ -86,6 +99,15 @@ export function errorCopy(error, provider) { + '(Google Auth Platform → Audience → Publish app).', }; } + if (isBotToken) { + return { + title: `${vendor} rejected the bot token`, + description: + 'The token from BotFather may be mistyped or revoked. ' + + 'Open @BotFather, run /mybots, pick your bot, and ' + + '"API Token" to copy a fresh token.', + }; + } return { title: `${vendor} rejected the password`, description: @@ -122,3 +144,11 @@ export function slugifyEmail(email) { const local = email.split('@')[0].toLowerCase(); return local.replace(/[^a-z0-9_-]+/g, '-').replace(/^-+|-+$/g, '').slice(0, 48); } + +export function slugifyLabel(label) { + if (!label) return ''; + return label.toLowerCase() + .replace(/[^a-z0-9_-]+/g, '-') + .replace(/^-+|-+$/g, '') + .slice(0, 48); +} diff --git a/tasks/_executor.py b/tasks/_executor.py index bad97aae..f2a0046e 100644 --- a/tasks/_executor.py +++ b/tasks/_executor.py @@ -7,16 +7,13 @@ from agents import build_agent, get_agent_profile from agents.types import Agent -from sdk import PersistenceHook, default_hooks, run_turn -from sdk.context import ContextManager, ConversationHistory, LLMCompactionStrategy -from sdk.events._context import agent_span, get_current_dispatcher -from sdk.events._models import FileOutputPayload +from sdk import Conversation, TurnExecutor +from sdk.context import ConversationHistory +from sdk.events._models import ContentPayload, FileOutputPayload from sdk.skills import AgentState, get_skill from sdk.tools._core import get_core_tools -from sdk.turn import turn_scope if TYPE_CHECKING: - from sdk.events._models import AgentEvent from tasks._models import Goal, Task, TaskResult from tasks._store import TaskStore @@ -28,6 +25,7 @@ class TaskExecutor: def __init__(self, store: TaskStore) -> None: self._store = store + self._executor = TurnExecutor() async def run(self, task_result: TaskResult, task: Task) -> tuple[str, list[str]]: """Execute a task and return (result_text, file_output_paths).""" @@ -46,45 +44,34 @@ async def run(self, task_result: TaskResult, task: Task) -> tuple[str, list[str] agent = await self._build_agent(task) - history = ConversationHistory( - [ - {"role": "system", "content": agent.instruction}, - {"role": "user", "content": instruction}, - ], - instance_id=conversation_id, + conversation = Conversation( + id=conversation_id, + history=ConversationHistory(instance_id=conversation_id), ) + accumulated_text: list[str] = [] file_paths: list[str] = [] - def _capture_file_output(event: AgentEvent) -> None: - if isinstance(event.payload, FileOutputPayload) and event.payload.path: - file_paths.append(event.payload.path) - - async with turn_scope(conversation_id=conversation_id): - dispatcher = get_current_dispatcher() - if dispatcher: - dispatcher.subscribe(_capture_file_output) - state = AgentState(await get_core_tools() + (agent.tools or [])) - ctx_manager = ContextManager( - history=history, - agent_state=state, - context_limit=agent.context_window, - agent_name=agent.name, - strategies=[ - LLMCompactionStrategy(threshold=agent.compaction_threshold), - ], - ) - hooks = default_hooks(agent, max_iterations=agent.max_iterations, ctx_manager=ctx_manager) - hooks.append( - PersistenceHook(conversation_id=conversation_id, history=history) - ) - async with agent_span(agent.name, instruction=instruction, agent_state=state): - result = await run_turn(history, agent, hooks=hooks) + async for event in self._executor.execute( + conversation=conversation, + agent=agent, + user_content=instruction, + ): + payload = event.payload + if isinstance(payload, ContentPayload) and payload.content: + accumulated_text.append(payload.content) + elif isinstance(payload, FileOutputPayload) and payload.path: + file_paths.append(payload.path) - return result or "", file_paths + return "".join(accumulated_text), file_paths async def _build_agent(self, task: Task) -> Agent: - """Construct an Agent from the task's agent profile.""" + """Construct an Agent from the task's agent profile. + + Pre-validates the profile's skills so a missing one trips the task + runner with a clear message before the turn starts. ``TurnExecutor`` + independently restores the same skills via ``preloaded_skills``. + """ if not task.agent_profile: msg = f"Task {task.id} has no agent_profile set" raise RuntimeError(msg) @@ -93,15 +80,15 @@ async def _build_agent(self, task: Task) -> Agent: msg = f"Agent profile '{task.agent_profile}' not found for task {task.id}" raise RuntimeError(msg) - agent_state = AgentState(await get_core_tools()) + state = AgentState(await get_core_tools()) for skill_name in profile.skills: skill = get_skill(skill_name) if skill is None: msg = f"Profile '{profile.id}' references unregistered skill '{skill_name}'" raise RuntimeError(msg) - agent_state.add(skill) + state.add(skill) - return build_agent(profile, tools=agent_state.tools, name="TASK_AGENT") + return build_agent(profile, tools=state.tools, name="TASK_AGENT") def _build_instruction( self, task_result: TaskResult, task: Task, goal: Goal diff --git a/tasks/_notifier.py b/tasks/_notifier.py index 135028b7..838e91b8 100644 --- a/tasks/_notifier.py +++ b/tasks/_notifier.py @@ -1,4 +1,8 @@ -"""Telegram push notifications for goal run completion/failure.""" +"""Telegram push notifications for goal run completion/failure. + +Sends messages through the telegram broker (same broker the bidirectional +channel uses); no Telegram credentials live in this process. +""" from __future__ import annotations @@ -7,38 +11,54 @@ from pathlib import Path from typing import TYPE_CHECKING -import httpx +from integrations.broker_client import IntegrationError, call as broker_call if TYPE_CHECKING: from config import NotificationsConfig logger = logging.getLogger(__name__) +# Matches Telegram's per-message text cap; we truncate locally so the broker +# returns a clean message_id rather than rejecting a 4097-character payload. _TELEGRAM_MSG_LIMIT = 4096 class TelegramNotifier: - """Sends messages and file attachments to Telegram via the Bot API. + """Sends notification messages to Telegram via the broker. - Reads TELEGRAM_BOT_TOKEN and TELEGRAM_CHAT_ID from environment variables. - If either is missing, the notifier disables itself with a warning. - All public methods are fire-and-forget — errors are logged, never raised. + Reads ``TELEGRAM_INTEGRATION_ID`` and ``TELEGRAM_CHAT_ID`` from the + environment. If either is missing, the notifier disables itself with a + warning. All public methods are fire-and-forget — errors are logged, + never raised. """ - def __init__(self, config: NotificationsConfig) -> None: + def __init__( + self, + config: NotificationsConfig, + *, + app_sock_path: Path, + ) -> None: self._config = config - token = os.environ.get("TELEGRAM_BOT_TOKEN", "") - self._chat_id = os.environ.get("TELEGRAM_CHAT_ID", "") - if not token or not self._chat_id: + self._app_sock = app_sock_path + self._integration_id = os.environ.get("TELEGRAM_INTEGRATION_ID", "") + chat_id_raw = os.environ.get("TELEGRAM_CHAT_ID", "") + try: + self._chat_id: int | None = int(chat_id_raw) if chat_id_raw else None + except ValueError: logger.warning( - "TELEGRAM_BOT_TOKEN or TELEGRAM_CHAT_ID not set — " - "Telegram notifications disabled" + "TELEGRAM_CHAT_ID is not an integer (%r); Telegram notifications " + "disabled", chat_id_raw, + ) + self._chat_id = None + + if not self._integration_id or self._chat_id is None: + logger.warning( + "TELEGRAM_INTEGRATION_ID or TELEGRAM_CHAT_ID not set — " + "Telegram notifications disabled", ) self._disabled = True - self._base_url = "" - return - self._disabled = False - self._base_url = f"https://api.telegram.org/bot{token}" + else: + self._disabled = False @property def enabled(self) -> bool: @@ -54,54 +74,30 @@ async def send( return try: await self._send_message(message) - for path in attachments or []: - await self._send_document(path) except Exception: logger.exception("Failed to send Telegram notification") + return + + for path in attachments or []: + # Document uploads through the broker are pending; surface the + # skip rather than silently dropping the file references. + logger.warning( + "telegram attachment not sent (broker send_document not " + "implemented yet): %s", path, + ) async def _send_message(self, text: str) -> None: if len(text) > _TELEGRAM_MSG_LIMIT: text = text[: _TELEGRAM_MSG_LIMIT - 30] + "\n\n… (truncated)" - async with httpx.AsyncClient(timeout=30) as client: - resp = await client.post( - f"{self._base_url}/sendMessage", - json={ - "chat_id": self._chat_id, - "text": text, - }, - ) - if resp.status_code != 200: - logger.error( - "Telegram sendMessage failed (%d): %s", - resp.status_code, - resp.text, - ) - - async def _send_document(self, path: Path) -> None: - max_bytes = self._config.max_attachment_size_mb * 1024 * 1024 - if not path.is_file(): - logger.warning("Attachment not found, skipping: %s", path) - return - if path.stat().st_size > max_bytes: - logger.warning( - "Attachment too large (%d MB limit), skipping: %s", - self._config.max_attachment_size_mb, - path, + try: + await broker_call( + self._integration_id, + "send_message", + {"chat_id": self._chat_id, "text": text}, + app_sock_path=self._app_sock, ) - return - async with httpx.AsyncClient(timeout=120) as client: - with open(path, "rb") as f: - resp = await client.post( - f"{self._base_url}/sendDocument", - data={"chat_id": self._chat_id}, - files={"document": (path.name, f)}, - ) - if resp.status_code != 200: - logger.error( - "Telegram sendDocument failed (%d): %s", - resp.status_code, - resp.text, - ) + except IntegrationError as exc: + logger.error("Telegram send_message failed: %s", exc) def format_run_completed( @@ -115,8 +111,8 @@ def format_run_completed( ) -> str: """Format a success notification message.""" lines = [ - f"\u2705 Goal completed: {goal_description}", - f"Run #{run_number} \u00b7 {duration} \u00b7 {completed_tasks}/{total_tasks} tasks", + f"✅ Goal completed: {goal_description}", + f"Run #{run_number} · {duration} · {completed_tasks}/{total_tasks} tasks", "", ] if final_output: @@ -140,8 +136,8 @@ def format_run_failed( ) -> str: """Format a failure notification message.""" lines = [ - f"\u274c Goal failed: {goal_description}", - f"Run #{run_number} \u00b7 {duration} \u00b7 {completed_tasks}/{total_tasks} tasks completed", + f"❌ Goal failed: {goal_description}", + f"Run #{run_number} · {duration} · {completed_tasks}/{total_tasks} tasks completed", "", f"Error (task: {failed_task_description}):", error, diff --git a/tests/unit/channels/__init__.py b/tests/unit/channels/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/unit/channels/telegram/__init__.py b/tests/unit/channels/telegram/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/unit/channels/telegram/test_formatter.py b/tests/unit/channels/telegram/test_formatter.py new file mode 100644 index 00000000..ba272ad7 --- /dev/null +++ b/tests/unit/channels/telegram/test_formatter.py @@ -0,0 +1,128 @@ +"""Tests for channels.telegram._formatter.TelegramFormatter.""" + +from pathlib import Path + +import pytest + +from channels.telegram._formatter import TelegramFormatter + + +@pytest.mark.unit +class TestSplit: + """Tests for the split() chunker — paragraph/sentence/hard-split branches.""" + + def test_short_text_returned_unchanged(self): + chunks = TelegramFormatter.split("hello world", limit=4096) + assert chunks == ["hello world"] + + def test_empty_string_returns_single_empty(self): + # An empty string is below the limit, so it comes back as a single chunk. + assert TelegramFormatter.split("", limit=100) == [""] + + def test_exact_limit_not_split(self): + text = "x" * 100 + chunks = TelegramFormatter.split(text, limit=100) + assert chunks == [text] + + def test_paragraph_break_preferred(self): + """When a double-newline exists before the limit, the first split + happens at the paragraph boundary.""" + text = "para one is here.\n\npara two is short." + chunks = TelegramFormatter.split(text, limit=30) + assert len(chunks) == 2 + assert chunks[0] == "para one is here." + assert chunks[1].startswith("para two") + + def test_single_newline_when_no_paragraph_break(self): + """Falls through to a single-newline split when no double-newline fits.""" + text = "line one of text\nline two of text\nline three" + chunks = TelegramFormatter.split(text, limit=20) + # Should split on a newline before position 20. + assert len(chunks) >= 2 + assert all(len(c) <= 20 for c in chunks) + # No chunk should contain a leading newline (lstrip\n applied). + assert all(not c.startswith("\n") for c in chunks) + + def test_sentence_boundary_fallback(self): + """Falls through to a sentence end when no newline before the limit. + + The splitter's sentence cut sits *before* the punctuation, so the + next chunk starts with the punctuation. Assert on the partitioning + rather than the boundary character. + """ + text = "First sentence ends here. Second sentence is right after! Third?" + chunks = TelegramFormatter.split(text, limit=30) + assert len(chunks) >= 2 + # Both halves reflect the original content split somewhere mid-string. + joined = "".join(chunks) + # Joining loses the whitespace that was stripped at the boundaries; + # confirm the meaningful tokens survive. + for token in ("First", "Second", "Third"): + assert token in joined + + def test_hard_split_when_no_boundary(self): + """Long unbroken text gets a hard split at the limit.""" + text = "x" * 50 + chunks = TelegramFormatter.split(text, limit=10) + # 50 / 10 = 5 hard-split chunks. + assert len(chunks) == 5 + assert all(len(c) == 10 for c in chunks) + + def test_hard_split_when_only_late_boundary(self): + """Sentence break later than limit//4 wins; earlier than that is too short, hard-split instead.""" + # Sentence break is at position 5, limit is 20, limit//4 is 5 — boundary + # >= limit//4 wins. Make it boundary < limit//4 to force hard split. + text = "ab. " + "x" * 40 # period at index 2, limit//4 = 5 -> too early + chunks = TelegramFormatter.split(text, limit=20) + # First chunk should be ~20 chars, not "ab." + assert len(chunks[0]) > 5 + + def test_chunks_join_to_original_modulo_whitespace_trim(self): + """Chunks reassemble back to the original input ignoring per-chunk + leading-newline strip and trailing-whitespace trim.""" + text = ( + "Paragraph one with some content.\n\n" + "Paragraph two has a fair bit more text so it crosses the limit boundary.\n\n" + "Paragraph three." + ) + chunks = TelegramFormatter.split(text, limit=60) + joined = "\n\n".join(chunks) + # Whitespace normalization makes exact equality tricky; just confirm + # each paragraph survives somewhere in the output. + assert "Paragraph one" in joined + assert "Paragraph two" in joined + assert "Paragraph three" in joined + + +@pytest.mark.unit +class TestEscapeMarkdown: + """Tests for MarkdownV2 escaping.""" + + def test_escapes_all_special_chars(self): + out = TelegramFormatter.escape_markdown("hello *world* (yes)") + assert "\\*" in out + assert "\\(" in out + assert "\\)" in out + + def test_plain_text_passthrough(self): + out = TelegramFormatter.escape_markdown("plain text") + assert out == "plain text" + + +@pytest.mark.unit +class TestFileCaption: + """Tests for file_caption().""" + + def test_single_file_caption(self): + caption = TelegramFormatter.file_caption(Path("report.pdf")) + assert caption == "📎 report.pdf" + + def test_multi_file_caption_includes_index(self): + caption = TelegramFormatter.file_caption( + Path("/tmp/a.txt"), index=0, total=3, + ) + assert caption == "📎 a.txt (1/3)" + caption = TelegramFormatter.file_caption( + Path("/tmp/b.txt"), index=2, total=3, + ) + assert caption == "📎 b.txt (3/3)" diff --git a/tests/unit/channels/telegram/test_profile_map.py b/tests/unit/channels/telegram/test_profile_map.py new file mode 100644 index 00000000..44da53b1 --- /dev/null +++ b/tests/unit/channels/telegram/test_profile_map.py @@ -0,0 +1,41 @@ +"""Tests for channels.telegram._profile_map.ProfileMap.""" + +import pytest + +from channels.telegram._profile_map import ProfileMap + + +@pytest.mark.unit +class TestProfileMap: + + def test_unknown_chat_returns_none(self): + pmap = ProfileMap() + assert pmap.get(404) is None + + def test_set_then_get(self): + pmap = ProfileMap() + pmap.set(7, "code_expert") + assert pmap.get(7) == "code_expert" + + def test_set_overwrites_previous_choice(self): + pmap = ProfileMap() + pmap.set(7, "first") + pmap.set(7, "second") + assert pmap.get(7) == "second" + + def test_clear_drops_choice(self): + pmap = ProfileMap() + pmap.set(7, "x") + pmap.clear(7) + assert pmap.get(7) is None + + def test_clear_unknown_chat_is_a_no_op(self): + pmap = ProfileMap() + pmap.clear(404) # should not raise + + def test_independent_chats(self): + pmap = ProfileMap() + pmap.set(1, "a") + pmap.set(2, "b") + assert pmap.get(1) == "a" + assert pmap.get(2) == "b" diff --git a/tests/unit/channels/telegram/test_state.py b/tests/unit/channels/telegram/test_state.py new file mode 100644 index 00000000..f2a6966c --- /dev/null +++ b/tests/unit/channels/telegram/test_state.py @@ -0,0 +1,60 @@ +"""Tests for channels.telegram._state.ConversationMap.""" + +import pytest + +from channels.telegram._state import ConversationMap + + +@pytest.mark.unit +class TestConversationMap: + + def test_get_creates_default_conv_id_on_first_access(self): + cmap = ConversationMap() + cid = cmap.get(12345) + assert cid == "telegram_12345" + + def test_get_is_idempotent_for_same_chat(self): + cmap = ConversationMap() + first = cmap.get(7) + second = cmap.get(7) + assert first == second + + def test_different_chats_get_distinct_ids(self): + cmap = ConversationMap() + a = cmap.get(1) + b = cmap.get(2) + assert a != b + + def test_reset_assigns_new_unique_id_with_chat_id_prefix(self): + cmap = ConversationMap() + original = cmap.get(42) + reset = cmap.reset(42) + + assert reset != original + # Still keyed back to the chat — the prefix preserves traceability. + assert reset.startswith("telegram_42_") + # Subsequent get returns the reset id, not the default. + assert cmap.get(42) == reset + + def test_reset_is_unique_across_resets_on_same_chat(self): + cmap = ConversationMap() + cmap.get(99) + first_reset = cmap.reset(99) + second_reset = cmap.reset(99) + assert first_reset != second_reset + + def test_conversation_id_for_returns_none_when_unknown(self): + cmap = ConversationMap() + assert cmap.conversation_id_for(404) is None + + def test_conversation_id_for_returns_current_id_without_creating(self): + cmap = ConversationMap() + assert cmap.conversation_id_for(8) is None # not created + cmap.get(8) + assert cmap.conversation_id_for(8) == "telegram_8" + + def test_conversation_id_for_reflects_reset(self): + cmap = ConversationMap() + cmap.get(15) + new = cmap.reset(15) + assert cmap.conversation_id_for(15) == new diff --git a/tests/unit/integrations/brokers/telegram_broker/__init__.py b/tests/unit/integrations/brokers/telegram_broker/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/unit/integrations/brokers/telegram_broker/test_main.py b/tests/unit/integrations/brokers/telegram_broker/test_main.py new file mode 100644 index 00000000..126990e9 --- /dev/null +++ b/tests/unit/integrations/brokers/telegram_broker/test_main.py @@ -0,0 +1,42 @@ +"""Tests for integrations.brokers.telegram_broker.__main__ helpers.""" + +import pytest + +from integrations.brokers.telegram_broker.__main__ import _parse_allowed_user_ids + + +@pytest.mark.unit +class TestParseAllowedUserIds: + + def test_empty_string_yields_empty_set(self): + assert _parse_allowed_user_ids("") == frozenset() + + def test_whitespace_only_yields_empty_set(self): + assert _parse_allowed_user_ids(" , ,") == frozenset() + + def test_single_id(self): + assert _parse_allowed_user_ids("12345") == frozenset({12345}) + + def test_comma_separated_ids(self): + assert _parse_allowed_user_ids("1,2,3") == frozenset({1, 2, 3}) + + def test_strips_whitespace_around_entries(self): + assert _parse_allowed_user_ids(" 1 , 2,3 ") == frozenset({1, 2, 3}) + + def test_skips_blank_entries(self): + assert _parse_allowed_user_ids("1,,2,,,3") == frozenset({1, 2, 3}) + + def test_skips_non_integer_entries(self, caplog): + out = _parse_allowed_user_ids("1,bogus,2,also-bad,3") + assert out == frozenset({1, 2, 3}) + # Confirm we logged each bad entry (presence, not specific text). + assert any("bogus" in rec.getMessage() for rec in caplog.records) + assert any("also-bad" in rec.getMessage() for rec in caplog.records) + + def test_deduplicates(self): + assert _parse_allowed_user_ids("7,7,7,42") == frozenset({7, 42}) + + def test_negative_ids_accepted(self): + # Group chat IDs are negative in Telegram; we don't filter them out + # at the parser level — that's policy belonging higher up. + assert _parse_allowed_user_ids("-100,42") == frozenset({-100, 42}) diff --git a/tests/unit/integrations/brokers/telegram_broker/test_updates.py b/tests/unit/integrations/brokers/telegram_broker/test_updates.py new file mode 100644 index 00000000..1275fe0a --- /dev/null +++ b/tests/unit/integrations/brokers/telegram_broker/test_updates.py @@ -0,0 +1,276 @@ +"""Tests for integrations.brokers.telegram_broker._updates.""" + +from __future__ import annotations + +import asyncio +import time +from dataclasses import dataclass, field +from datetime import datetime, timezone +from typing import Any +from unittest.mock import MagicMock + +import pytest + +from integrations.brokers.telegram_broker._updates import ( + UpdatePump, + update_to_dict, +) + + +# --------------------------------------------------------------------------- +# Fake update objects — shape matches the aiogram attributes we read. +# --------------------------------------------------------------------------- + + +@dataclass +class _FakeUser: + id: int + username: str | None = None + + +@dataclass +class _FakeChat: + id: int + + +@dataclass +class _FakeMessage: + message_id: int + chat: _FakeChat + from_user: _FakeUser | None + text: str | None + date: datetime | None = field(default_factory=lambda: datetime(2026, 5, 22, tzinfo=timezone.utc)) + + +@dataclass +class _FakeCallbackQuery: + id: str + from_user: _FakeUser | None + data: str | None + message: _FakeMessage | None + + +@dataclass +class _FakeUpdate: + update_id: int + message: _FakeMessage | None = None + callback_query: _FakeCallbackQuery | None = None + + +# --------------------------------------------------------------------------- +# update_to_dict +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +class TestUpdateToDict: + + def test_text_message_maps_to_wire_dict(self): + msg = _FakeMessage( + message_id=10, + chat=_FakeChat(id=42), + from_user=_FakeUser(id=99, username="alice"), + text="hello", + ) + out = update_to_dict(_FakeUpdate(update_id=1, message=msg)) + assert out is not None + assert out["type"] == "message" + assert out["message_id"] == 10 + assert out["chat_id"] == 42 + assert out["from_user_id"] == 99 + assert out["from_username"] == "alice" + assert out["text"] == "hello" + assert out["is_command"] is False + assert isinstance(out["timestamp"], int) + + def test_slash_text_marked_as_command(self): + msg = _FakeMessage( + message_id=1, chat=_FakeChat(id=1), + from_user=_FakeUser(id=1), text="/new", + ) + out = update_to_dict(_FakeUpdate(update_id=1, message=msg)) + assert out is not None + assert out["is_command"] is True + + def test_no_message_returns_none(self): + # Update without a message field (e.g. a callback_query in real Telegram). + out = update_to_dict(_FakeUpdate(update_id=1, message=None)) + assert out is None + + def test_message_without_text_returns_none(self): + # Photo/document/sticker — no .text, dropped at this layer. + msg = _FakeMessage( + message_id=1, chat=_FakeChat(id=1), + from_user=_FakeUser(id=1), text=None, + ) + assert update_to_dict(_FakeUpdate(update_id=1, message=msg)) is None + + def test_message_without_from_user_returns_none(self): + # Service messages (channel posts, etc.) can lack from_user. + msg = _FakeMessage( + message_id=1, chat=_FakeChat(id=1), + from_user=None, text="hi", + ) + assert update_to_dict(_FakeUpdate(update_id=1, message=msg)) is None + + def test_falls_back_to_now_when_no_date(self): + msg = _FakeMessage( + message_id=1, chat=_FakeChat(id=1), + from_user=_FakeUser(id=1), text="hi", date=None, + ) + before = int(time.time()) + out = update_to_dict(_FakeUpdate(update_id=1, message=msg)) + after = int(time.time()) + assert out is not None + assert before <= out["timestamp"] <= after + + def test_callback_query_maps_to_wire_dict(self): + carrier_msg = _FakeMessage( + message_id=200, chat=_FakeChat(id=42), + from_user=_FakeUser(id=99), text="Pick:", + ) + cb = _FakeCallbackQuery( + id="cbq-1", + from_user=_FakeUser(id=99, username="alice"), + data="profile:foo", + message=carrier_msg, + ) + out = update_to_dict(_FakeUpdate(update_id=1, callback_query=cb)) + assert out is not None + assert out["type"] == "callback_query" + assert out["callback_id"] == "cbq-1" + assert out["from_user_id"] == 99 + assert out["from_username"] == "alice" + assert out["data"] == "profile:foo" + assert out["chat_id"] == 42 + assert out["message_id"] == 200 + + def test_callback_without_from_user_is_dropped(self): + carrier = _FakeMessage( + message_id=1, chat=_FakeChat(id=1), + from_user=_FakeUser(id=1), text="x", + ) + cb = _FakeCallbackQuery(id="x", from_user=None, data="d", message=carrier) + assert update_to_dict(_FakeUpdate(update_id=1, callback_query=cb)) is None + + def test_callback_without_data_is_dropped(self): + carrier = _FakeMessage( + message_id=1, chat=_FakeChat(id=1), + from_user=_FakeUser(id=1), text="x", + ) + cb = _FakeCallbackQuery( + id="x", from_user=_FakeUser(id=1), data=None, message=carrier, + ) + assert update_to_dict(_FakeUpdate(update_id=1, callback_query=cb)) is None + + def test_callback_without_carrier_message_is_dropped(self): + cb = _FakeCallbackQuery( + id="x", from_user=_FakeUser(id=1), data="d", message=None, + ) + assert update_to_dict(_FakeUpdate(update_id=1, callback_query=cb)) is None + + +# --------------------------------------------------------------------------- +# UpdatePump._handle_update — the security filter +# --------------------------------------------------------------------------- + + +def _make_pump(allowed_ids: frozenset[int]) -> UpdatePump: + """UpdatePump with a stubbed Bot — _handle_update doesn't use the bot.""" + return UpdatePump( + bot=MagicMock(), + allowed_user_ids=allowed_ids, + integration_id="telegram_test", + ) + + +def _msg_update(*, user_id: int, text: str = "hi", update_id: int = 1) -> _FakeUpdate: + return _FakeUpdate( + update_id=update_id, + message=_FakeMessage( + message_id=update_id, + chat=_FakeChat(id=user_id), # DM: chat.id == user.id + from_user=_FakeUser(id=user_id), + text=text, + ), + ) + + +@pytest.mark.unit +class TestUpdatePumpFilter: + + def test_allowed_user_is_enqueued(self): + pump = _make_pump(frozenset({100})) + pump._handle_update(_msg_update(user_id=100, text="hello")) + assert pump.queue.qsize() == 1 + out = pump.queue.get_nowait() + assert out["from_user_id"] == 100 + assert out["text"] == "hello" + + def test_disallowed_user_is_dropped(self): + pump = _make_pump(frozenset({100})) + pump._handle_update(_msg_update(user_id=999, text="spam")) + assert pump.queue.qsize() == 0 + # Drop accounted for. + assert pump._drop_counts == {999: 1} + + def test_disallowed_drops_accumulate_per_user(self): + pump = _make_pump(frozenset({100})) + for i in range(3): + pump._handle_update(_msg_update(user_id=999, text=f"spam{i}", update_id=i)) + for i in range(2): + pump._handle_update(_msg_update(user_id=888, text=f"x{i}", update_id=100 + i)) + assert pump.queue.qsize() == 0 + assert pump._drop_counts == {999: 3, 888: 2} + + def test_skippable_update_is_neither_queued_nor_counted(self): + # No-message update — update_to_dict returns None and the filter + # path is bypassed entirely. Drops counter must not bump. + pump = _make_pump(frozenset({100})) + pump._handle_update(_FakeUpdate(update_id=1, message=None)) + assert pump.queue.qsize() == 0 + assert pump._drop_counts == {} + + def test_empty_allowlist_drops_everyone(self): + pump = _make_pump(frozenset()) + pump._handle_update(_msg_update(user_id=1)) + pump._handle_update(_msg_update(user_id=2)) + assert pump.queue.qsize() == 0 + assert sum(pump._drop_counts.values()) == 2 + + +@pytest.mark.unit +class TestUpdatePumpDropLog: + + def test_drop_log_does_not_flush_within_window(self, caplog): + pump = _make_pump(frozenset({100})) + pump._handle_update(_msg_update(user_id=999)) + # Calling flush immediately (window not elapsed) should not log + # and should leave the counter intact. + pump._maybe_flush_drop_log() + assert pump._drop_counts == {999: 1} + + def test_drop_log_flushes_and_resets_after_window(self, caplog, monkeypatch): + pump = _make_pump(frozenset({100})) + pump._handle_update(_msg_update(user_id=999)) + pump._handle_update(_msg_update(user_id=888)) + + # Pretend the window started 9999 seconds ago so the flush fires. + pump._drop_window_started_at = time.monotonic() - 9999.0 + + with caplog.at_level("WARNING"): + pump._maybe_flush_drop_log() + + # Drop counters cleared, a log line emitted per offender. + assert pump._drop_counts == {} + messages = " ".join(rec.getMessage() for rec in caplog.records) + assert "user_id=999" in messages + assert "user_id=888" in messages + + def test_drop_log_no_op_when_nothing_to_flush(self, caplog): + pump = _make_pump(frozenset({100})) + pump._drop_window_started_at = time.monotonic() - 9999.0 + with caplog.at_level("WARNING"): + pump._maybe_flush_drop_log() + # Window timer should still reset even if there was nothing to log. + assert pump._drop_counts == {} diff --git a/tests/unit/integrations/brokers/telegram_broker/test_verbs.py b/tests/unit/integrations/brokers/telegram_broker/test_verbs.py new file mode 100644 index 00000000..8f04927a --- /dev/null +++ b/tests/unit/integrations/brokers/telegram_broker/test_verbs.py @@ -0,0 +1,400 @@ +"""Tests for integrations.brokers.telegram_broker._verbs.VerbDispatcher.""" + +from __future__ import annotations + +import asyncio +from dataclasses import dataclass +from unittest.mock import AsyncMock, MagicMock + +import pytest + +from integrations._rpc import RpcError +from integrations.brokers.telegram_broker._updates import UpdatePump +from integrations.brokers.telegram_broker._verbs import VerbDispatcher +from integrations.permissions import Access, Capability + + +_READ_WRITE = {Capability.TELEGRAM: Access.READ_WRITE} +_READ_ONLY = {Capability.TELEGRAM: Access.READ} +_NONE = {Capability.TELEGRAM: Access.OFF} + + +def _make_pump() -> UpdatePump: + return UpdatePump( + bot=MagicMock(), + allowed_user_ids=frozenset({1}), + integration_id="telegram_test", + ) + + +def _make_dispatcher(*, permissions=_READ_WRITE, bot=None, pump=None) -> VerbDispatcher: + return VerbDispatcher( + bot=bot or MagicMock(), + pump=pump or _make_pump(), + permissions=permissions, + ) + + +# --------------------------------------------------------------------------- +# Verb gating +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +class TestDispatchGating: + + async def test_unknown_verb_raises_bad_request(self): + dispatcher = _make_dispatcher() + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch("nope", {}) + assert exc.value.code == "BAD_REQUEST" + + async def test_insufficient_access_raises_permission_denied(self): + dispatcher = _make_dispatcher(permissions=_READ_ONLY) + # send_message requires READ_WRITE. + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch("send_message", {"chat_id": 1, "text": "x"}) + assert exc.value.code == "PERMISSION_DENIED" + + async def test_off_capability_blocks_read_verbs_too(self): + dispatcher = _make_dispatcher(permissions=_NONE) + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch("get_me", {}) + assert exc.value.code == "PERMISSION_DENIED" + + async def test_write_verb_blocked_by_permission_gate(self): + # The permission gate must fire before any handler logic runs. + dispatcher = _make_dispatcher(permissions=_READ_ONLY) + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch( + "send_document", {"chat_id": 1, "host_path": "/tmp/whatever"}, + ) + assert exc.value.code == "PERMISSION_DENIED" + + +# --------------------------------------------------------------------------- +# get_me +# --------------------------------------------------------------------------- + + +@dataclass +class _FakeMe: + id: int + username: str | None + first_name: str + + +@pytest.mark.unit +class TestGetMe: + + async def test_returns_identity_fields(self): + bot = MagicMock() + bot.get_me = AsyncMock(return_value=_FakeMe(id=42, username="testbot", first_name="Test")) + dispatcher = _make_dispatcher(bot=bot) + out = await dispatcher.dispatch("get_me", {}) + assert out == {"id": 42, "username": "testbot", "first_name": "Test"} + + +# --------------------------------------------------------------------------- +# next_updates +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +class TestNextUpdates: + + async def test_returns_empty_on_timeout(self): + pump = _make_pump() + dispatcher = _make_dispatcher(pump=pump) + out = await dispatcher.dispatch("next_updates", {"timeout_ms": 50}) + assert out == {"updates": []} + + async def test_drains_buffered_batch(self): + pump = _make_pump() + pump.queue.put_nowait({"type": "message", "text": "a", "message_id": 1}) + pump.queue.put_nowait({"type": "message", "text": "b", "message_id": 2}) + pump.queue.put_nowait({"type": "message", "text": "c", "message_id": 3}) + dispatcher = _make_dispatcher(pump=pump) + out = await dispatcher.dispatch("next_updates", {"timeout_ms": 50}) + assert [u["text"] for u in out["updates"]] == ["a", "b", "c"] + assert pump.queue.empty() + + async def test_blocks_until_an_update_arrives(self): + """The handler waits up to timeout_ms for the first update.""" + pump = _make_pump() + dispatcher = _make_dispatcher(pump=pump) + + async def _deliver_later() -> None: + await asyncio.sleep(0.02) + pump.queue.put_nowait({"type": "message", "text": "late", "message_id": 1}) + + deliver_task = asyncio.create_task(_deliver_later()) + out = await dispatcher.dispatch("next_updates", {"timeout_ms": 1000}) + await deliver_task + assert len(out["updates"]) == 1 + assert out["updates"][0]["text"] == "late" + + async def test_rejects_negative_timeout(self): + dispatcher = _make_dispatcher() + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch("next_updates", {"timeout_ms": -1}) + assert exc.value.code == "BAD_REQUEST" + + async def test_default_timeout_used_when_missing(self): + pump = _make_pump() + pump.queue.put_nowait({"type": "message", "text": "x", "message_id": 1}) + dispatcher = _make_dispatcher(pump=pump) + out = await dispatcher.dispatch("next_updates", {}) + # With an item ready the handler returns immediately regardless of + # default timeout — assert the item came through. + assert len(out["updates"]) == 1 + + +# --------------------------------------------------------------------------- +# send_message +# --------------------------------------------------------------------------- + + +@dataclass +class _FakeSentMessage: + message_id: int + + +@pytest.mark.unit +class TestSendMessage: + + async def test_sends_and_returns_message_id(self): + bot = MagicMock() + bot.send_message = AsyncMock(return_value=_FakeSentMessage(message_id=777)) + dispatcher = _make_dispatcher(bot=bot) + out = await dispatcher.dispatch( + "send_message", {"chat_id": 42, "text": "hi"}, + ) + assert out == {"message_id": 777} + bot.send_message.assert_awaited_once_with( + chat_id=42, text="hi", reply_to_message_id=None, reply_markup=None, + ) + + async def test_passes_reply_to_message_id_when_provided(self): + bot = MagicMock() + bot.send_message = AsyncMock(return_value=_FakeSentMessage(message_id=1)) + dispatcher = _make_dispatcher(bot=bot) + await dispatcher.dispatch( + "send_message", + {"chat_id": 1, "text": "hi", "reply_to_message_id": 5}, + ) + assert bot.send_message.await_args.kwargs["reply_to_message_id"] == 5 + + @pytest.mark.parametrize("bad_chat_id", [None, "42", 1.5, True, False]) + async def test_rejects_non_integer_chat_id(self, bad_chat_id): + dispatcher = _make_dispatcher() + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch("send_message", {"chat_id": bad_chat_id, "text": "hi"}) + assert exc.value.code == "BAD_REQUEST" + + @pytest.mark.parametrize("bad_text", [None, "", 0, []]) + async def test_rejects_empty_or_non_string_text(self, bad_text): + dispatcher = _make_dispatcher() + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch("send_message", {"chat_id": 1, "text": bad_text}) + assert exc.value.code == "BAD_REQUEST" + + async def test_rejects_non_integer_reply_to(self): + dispatcher = _make_dispatcher() + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch( + "send_message", + {"chat_id": 1, "text": "hi", "reply_to_message_id": "5"}, + ) + assert exc.value.code == "BAD_REQUEST" + + async def test_aiogram_api_error_becomes_internal_rpc_error(self): + from aiogram.exceptions import TelegramBadRequest + + bot = MagicMock() + bot.send_message = AsyncMock(side_effect=TelegramBadRequest( + method=MagicMock(), message="chat not found", + )) + dispatcher = _make_dispatcher(bot=bot) + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch("send_message", {"chat_id": 1, "text": "hi"}) + assert exc.value.code == "INTERNAL" + + async def test_passes_inline_keyboard_when_buttons_provided(self): + from aiogram.types import InlineKeyboardMarkup + + bot = MagicMock() + bot.send_message = AsyncMock(return_value=_FakeSentMessage(message_id=1)) + dispatcher = _make_dispatcher(bot=bot) + await dispatcher.dispatch( + "send_message", + { + "chat_id": 1, + "text": "Pick:", + "buttons": [ + [{"text": "A", "data": "a"}, {"text": "B", "data": "b"}], + [{"text": "C", "data": "c"}], + ], + }, + ) + markup = bot.send_message.await_args.kwargs["reply_markup"] + assert isinstance(markup, InlineKeyboardMarkup) + # Two rows; first has two buttons, second has one. + assert [len(row) for row in markup.inline_keyboard] == [2, 1] + assert markup.inline_keyboard[0][0].text == "A" + assert markup.inline_keyboard[0][0].callback_data == "a" + assert markup.inline_keyboard[1][0].callback_data == "c" + + @pytest.mark.parametrize("bad_buttons", [ + "not a list", + [{"not": "a list"}], # row isn't a list + [[{"text": "a"}]], # missing data + [[{"data": "x"}]], # missing text + [[{"text": "", "data": "x"}]], # empty text + [[{"text": "a", "data": ""}]], # empty data + ]) + async def test_rejects_malformed_buttons(self, bad_buttons): + dispatcher = _make_dispatcher() + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch( + "send_message", {"chat_id": 1, "text": "x", "buttons": bad_buttons}, + ) + assert exc.value.code == "BAD_REQUEST" + + +# --------------------------------------------------------------------------- +# send_chat_action +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +class TestSendChatAction: + + async def test_calls_bot_send_chat_action(self): + bot = MagicMock() + bot.send_chat_action = AsyncMock() + dispatcher = _make_dispatcher(bot=bot) + out = await dispatcher.dispatch( + "send_chat_action", {"chat_id": 7, "action": "typing"}, + ) + assert out == {} + bot.send_chat_action.assert_awaited_once_with(chat_id=7, action="typing") + + async def test_requires_action(self): + dispatcher = _make_dispatcher() + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch("send_chat_action", {"chat_id": 7}) + assert exc.value.code == "BAD_REQUEST" + + +# --------------------------------------------------------------------------- +# answer_callback_query +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +class TestAnswerCallbackQuery: + + async def test_calls_bot_answer(self): + bot = MagicMock() + bot.answer_callback_query = AsyncMock() + dispatcher = _make_dispatcher(bot=bot) + await dispatcher.dispatch( + "answer_callback_query", + {"callback_id": "cbq-1", "text": "Picked!"}, + ) + bot.answer_callback_query.assert_awaited_once_with( + callback_query_id="cbq-1", text="Picked!", + ) + + async def test_text_optional(self): + bot = MagicMock() + bot.answer_callback_query = AsyncMock() + dispatcher = _make_dispatcher(bot=bot) + await dispatcher.dispatch( + "answer_callback_query", {"callback_id": "cbq-1"}, + ) + bot.answer_callback_query.assert_awaited_once_with( + callback_query_id="cbq-1", text=None, + ) + + +# --------------------------------------------------------------------------- +# send_document +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +class TestSendDocument: + + async def test_sends_existing_file(self, tmp_path): + f = tmp_path / "report.pdf" + f.write_bytes(b"hello") + + bot = MagicMock() + bot.send_document = AsyncMock(return_value=_FakeSentMessage(message_id=99)) + dispatcher = _make_dispatcher(bot=bot) + + out = await dispatcher.dispatch( + "send_document", + {"chat_id": 1, "host_path": str(f), "caption": "📎 report.pdf"}, + ) + assert out == {"message_id": 99} + kwargs = bot.send_document.await_args.kwargs + assert kwargs["chat_id"] == 1 + assert kwargs["caption"] == "📎 report.pdf" + # The document arg is an FSInputFile pointing at the host path. + assert kwargs["document"].path == str(f) + + async def test_missing_file_rejected(self): + dispatcher = _make_dispatcher() + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch( + "send_document", {"chat_id": 1, "host_path": "/no/such/file.bin"}, + ) + assert exc.value.code == "BAD_REQUEST" + + +# --------------------------------------------------------------------------- +# edit_message_text + delete_message — live status message support +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +class TestEditMessageText: + + async def test_calls_bot_edit(self): + bot = MagicMock() + bot.edit_message_text = AsyncMock() + dispatcher = _make_dispatcher(bot=bot) + out = await dispatcher.dispatch( + "edit_message_text", + {"chat_id": 7, "message_id": 100, "text": "🔧 Calling search..."}, + ) + assert out == {} + bot.edit_message_text.assert_awaited_once_with( + chat_id=7, message_id=100, text="🔧 Calling search...", + ) + + @pytest.mark.parametrize("missing", ["chat_id", "message_id", "text"]) + async def test_required_args(self, missing): + args = {"chat_id": 1, "message_id": 2, "text": "x"} + args.pop(missing) + dispatcher = _make_dispatcher() + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch("edit_message_text", args) + assert exc.value.code == "BAD_REQUEST" + + +@pytest.mark.unit +class TestDeleteMessage: + + async def test_calls_bot_delete(self): + bot = MagicMock() + bot.delete_message = AsyncMock() + dispatcher = _make_dispatcher(bot=bot) + out = await dispatcher.dispatch( + "delete_message", {"chat_id": 7, "message_id": 100}, + ) + assert out == {} + bot.delete_message.assert_awaited_once_with(chat_id=7, message_id=100) diff --git a/tests/unit/sdk/events/test_message_handler_bridge.py b/tests/unit/sdk/events/test_message_handler_bridge.py index 56134a22..bf5e18bb 100644 --- a/tests/unit/sdk/events/test_message_handler_bridge.py +++ b/tests/unit/sdk/events/test_message_handler_bridge.py @@ -50,6 +50,7 @@ async def _fake_tool_loop(**_: Any) -> str | None: return "done" import server.message_handler as mh + import sdk.turn._executor as executor_mod mock_profile = AgentProfile( id="computron", @@ -60,7 +61,7 @@ async def _fake_tool_loop(**_: Any) -> str | None: skills=[], ) monkeypatch.setattr(mh, "get_agent_profile", lambda _pid: mock_profile) - monkeypatch.setattr(mh, "run_turn", _fake_tool_loop) + monkeypatch.setattr(executor_mod, "run_turn", _fake_tool_loop) seen: list[AgentEvent] = [] async for ev in handle_user_message( "hi", data=None, profile_id="computron", conversation_id="test-conv", diff --git a/tests/unit/sdk/turn/test_executor.py b/tests/unit/sdk/turn/test_executor.py new file mode 100644 index 00000000..45a1af66 --- /dev/null +++ b/tests/unit/sdk/turn/test_executor.py @@ -0,0 +1,328 @@ +"""Tests for sdk.turn._executor.TurnExecutor — injection points + persistence.""" + +from __future__ import annotations + +from typing import Any +from unittest.mock import AsyncMock, MagicMock, patch + +import pytest + +from agents.types import Agent +from sdk import Conversation, TurnExecutor +from sdk.context import ConversationHistory +from sdk.skills._registry import Skill + +_MOD = "sdk.turn._executor" + + +def _make_agent(**overrides: Any) -> Agent: + defaults = { + "name": "test-agent", + "description": "test", + "instruction": "BASE INSTRUCTION", + "provider": "ollama", + "model": "test-model", + "options": {}, + "tools": [], + "think": False, + "context_window": 0, + "compaction_threshold": 0.75, + "max_iterations": 0, + } + defaults.update(overrides) + return Agent(**defaults) + + +def _make_conversation(conv_id: str = "c1") -> Conversation: + return Conversation( + id=conv_id, + history=ConversationHistory(instance_id=conv_id), + ) + + +class _FakePersistence: + """Implements the TurnPersistence protocol with recording stubs.""" + + def __init__(self, persisted_skills: list[str] | None = None) -> None: + self._persisted_skills = persisted_skills or [] + self.load_skills_calls: list[str] = [] + self.save_skills_calls: list[tuple[str, list[str]]] = [] + self.save_events_calls: list[tuple[str, list[Any]]] = [] + self.on_new_calls: list[tuple[str, str]] = [] + + def load_skills(self, conversation_id: str): + self.load_skills_calls.append(conversation_id) + return list(self._persisted_skills) + + def save_skills(self, conversation_id: str, skills) -> None: + self.save_skills_calls.append((conversation_id, sorted(skills))) + + def save_events(self, conversation_id: str, events) -> None: + self.save_events_calls.append((conversation_id, list(events))) + + async def on_new_conversation(self, conversation_id: str, first_message: str) -> None: + self.on_new_calls.append((conversation_id, first_message)) + + +async def _drain(executor_call): + """Iterate the async-generator to completion and collect events.""" + return [evt async for evt in executor_call] + + +# --------------------------------------------------------------------------- +# build_system_prompt +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +async def test_build_system_prompt_result_is_used_as_base(): + conv = _make_conversation() + agent = _make_agent(instruction="DEFAULT") + + with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): + await _drain(TurnExecutor().execute( + conversation=conv, + agent=agent, + user_content="hi", + build_system_prompt=lambda: "FRESH PROMPT", + )) + + # The system message ends up at index 0 of history. + system = conv.history.messages[0] + assert system["role"] == "system" + assert system["content"] == "FRESH PROMPT" + + +@pytest.mark.unit +async def test_no_builder_falls_back_to_agent_instruction(): + conv = _make_conversation() + agent = _make_agent(instruction="DEFAULT") + + with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): + await _drain(TurnExecutor().execute( + conversation=conv, + agent=agent, + user_content="hi", + )) + + assert conv.history.messages[0]["content"] == "DEFAULT" + + +@pytest.mark.unit +async def test_builder_is_called_fresh_each_turn(): + """The closure is re-evaluated per turn so live state (memory) lands fresh.""" + conv = _make_conversation() + agent = _make_agent() + + call_count = 0 + def _builder() -> str: + nonlocal call_count + call_count += 1 + return f"prompt-{call_count}" + + with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): + await _drain(TurnExecutor().execute( + conversation=conv, agent=agent, user_content="t1", + build_system_prompt=_builder, + )) + await _drain(TurnExecutor().execute( + conversation=conv, agent=agent, user_content="t2", + build_system_prompt=_builder, + )) + + assert call_count == 2 + assert conv.history.messages[0]["content"] == "prompt-2" + + +# --------------------------------------------------------------------------- +# user_content append + system prompt placement +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +async def test_user_message_appended_to_history(): + conv = _make_conversation() + agent = _make_agent() + + with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): + await _drain(TurnExecutor().execute( + conversation=conv, agent=agent, user_content="hello world", + )) + + roles = [m["role"] for m in conv.history.messages] + assert "user" in roles + user_msg = next(m for m in conv.history.messages if m["role"] == "user") + assert user_msg["content"] == "hello world" + + +# --------------------------------------------------------------------------- +# Skill preloading + persistence restore +# --------------------------------------------------------------------------- + + +def _fake_skill(name: str) -> Skill: + return Skill(name=name, description=f"{name} skill", prompt=f"{name}-prompt", tools=[]) + + +@pytest.mark.unit +async def test_preloaded_skills_added_via_registry(): + conv = _make_conversation() + agent = _make_agent() + + skills_lookup = {"foo": _fake_skill("foo"), "bar": _fake_skill("bar")} + + with ( + patch(f"{_MOD}.run_turn", new_callable=AsyncMock), + patch(f"{_MOD}.get_skill", side_effect=lambda n: skills_lookup.get(n)), + ): + await _drain(TurnExecutor().execute( + conversation=conv, agent=agent, user_content="hi", + preloaded_skills=["foo", "bar"], + )) + + # Skill names land in the system prompt via build_skill_prompt. + system = conv.history.messages[0]["content"] + assert "foo-prompt" in system + assert "bar-prompt" in system + + +@pytest.mark.unit +async def test_preloaded_skill_not_in_registry_is_logged_and_skipped(caplog): + conv = _make_conversation() + agent = _make_agent() + + with ( + patch(f"{_MOD}.run_turn", new_callable=AsyncMock), + patch(f"{_MOD}.get_skill", return_value=None), + caplog.at_level("WARNING"), + ): + await _drain(TurnExecutor().execute( + conversation=conv, agent=agent, user_content="hi", + preloaded_skills=["ghost"], + )) + + # Logged a warning; no crash. + assert any("ghost" in rec.getMessage() for rec in caplog.records) + + +@pytest.mark.unit +async def test_persistence_load_skills_restores_them(): + conv = _make_conversation() + agent = _make_agent() + persistence = _FakePersistence(persisted_skills=["restored_a", "restored_b"]) + skills = {n: _fake_skill(n) for n in ("restored_a", "restored_b")} + + with ( + patch(f"{_MOD}.run_turn", new_callable=AsyncMock), + patch(f"{_MOD}.get_skill", side_effect=lambda n: skills.get(n)), + ): + await _drain(TurnExecutor().execute( + conversation=conv, agent=agent, user_content="hi", + persistence=persistence, + )) + + assert persistence.load_skills_calls == [conv.id] + system = conv.history.messages[0]["content"] + assert "restored_a-prompt" in system + assert "restored_b-prompt" in system + + +# --------------------------------------------------------------------------- +# Persistence write paths +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +async def test_persistence_save_skills_called_with_loaded_names(): + conv = _make_conversation() + agent = _make_agent() + persistence = _FakePersistence() + skill = _fake_skill("alpha") + + with ( + patch(f"{_MOD}.run_turn", new_callable=AsyncMock), + patch(f"{_MOD}.get_skill", return_value=skill), + ): + await _drain(TurnExecutor().execute( + conversation=conv, agent=agent, user_content="hi", + preloaded_skills=["alpha"], + persistence=persistence, + )) + + assert persistence.save_skills_calls == [(conv.id, ["alpha"])] + + +@pytest.mark.unit +async def test_persistence_save_skills_skipped_when_no_skills_loaded(): + conv = _make_conversation() + agent = _make_agent() + persistence = _FakePersistence() + + with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): + await _drain(TurnExecutor().execute( + conversation=conv, agent=agent, user_content="hi", + persistence=persistence, + )) + + assert persistence.save_skills_calls == [] + + +@pytest.mark.unit +async def test_on_new_conversation_fires_only_when_flagged(): + conv = _make_conversation() + agent = _make_agent() + persistence = _FakePersistence() + + with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): + await _drain(TurnExecutor().execute( + conversation=conv, agent=agent, user_content="first-turn", + is_new_conversation=True, + persistence=persistence, + )) + + # on_new_conversation runs as a background task — give it a turn. + import asyncio + await asyncio.sleep(0) + assert persistence.on_new_calls == [(conv.id, "first-turn")] + + +@pytest.mark.unit +async def test_on_new_conversation_skipped_on_continuation(): + conv = _make_conversation() + agent = _make_agent() + persistence = _FakePersistence() + + with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): + await _drain(TurnExecutor().execute( + conversation=conv, agent=agent, user_content="t", + is_new_conversation=False, + persistence=persistence, + )) + + import asyncio + await asyncio.sleep(0) + assert persistence.on_new_calls == [] + + +# --------------------------------------------------------------------------- +# persistence=None — every persistence call must be skipped +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +async def test_no_persistence_is_safe(): + conv = _make_conversation() + agent = _make_agent() + skill = _fake_skill("alpha") + + with ( + patch(f"{_MOD}.run_turn", new_callable=AsyncMock), + patch(f"{_MOD}.get_skill", return_value=skill), + ): + # Nothing should raise, even with preloaded skills and is_new=True. + await _drain(TurnExecutor().execute( + conversation=conv, agent=agent, user_content="hi", + is_new_conversation=True, + preloaded_skills=["alpha"], + persistence=None, + )) diff --git a/tests/unit/server/test_message_handler.py b/tests/unit/server/test_message_handler.py index 5399afb3..5dcc21c1 100644 --- a/tests/unit/server/test_message_handler.py +++ b/tests/unit/server/test_message_handler.py @@ -9,6 +9,7 @@ import pytest from conversations._store import save_conversation_history +from sdk import Conversation from sdk.context import ConversationHistory from server import message_handler as mh @@ -31,8 +32,8 @@ def _stub_browser_release(): async def test_get_conversation_cold_cache_no_disk_creates_empty_and_marks_new() -> None: """No in-memory entry, no on-disk history -> empty + is_new=True.""" conv, is_new = await mh._get_conversation("brand-new-id") - assert len(conv) == 0 - assert conv.instance_id == "brand-new-id" + assert len(conv.history) == 0 + assert conv.history.instance_id == "brand-new-id" assert is_new is True @@ -45,8 +46,8 @@ async def test_get_conversation_cold_cache_with_disk_hydrates_and_marks_not_new( conv, is_new = await mh._get_conversation("existing") - assert len(conv) == 2 - loaded = conv.messages + assert len(conv.history) == 2 + loaded = conv.history.messages assert loaded[0]["content"] == "hello" assert loaded[1]["content"] == "hi" assert is_new is False @@ -54,9 +55,12 @@ async def test_get_conversation_cold_cache_with_disk_hydrates_and_marks_not_new( async def test_get_conversation_warm_cache_does_not_reread_disk() -> None: """An in-memory entry wins over whatever is on disk and is_new=False.""" - cached = ConversationHistory( - [{"role": "user", "content": "from-memory"}], - instance_id="cid", + cached = Conversation( + id="cid", + history=ConversationHistory( + [{"role": "user", "content": "from-memory"}], + instance_id="cid", + ), ) mh._conversations["cid"] = cached save_conversation_history("cid", [{"role": "user", "content": "from-disk"}]) @@ -64,12 +68,12 @@ async def test_get_conversation_warm_cache_does_not_reread_disk() -> None: conv, is_new = await mh._get_conversation("cid") assert conv is cached - assert conv.messages[0]["content"] == "from-memory" + assert conv.history.messages[0]["content"] == "from-memory" assert is_new is False async def test_get_conversation_subsequent_call_returns_same_instance() -> None: - """Two calls for the same id return the same ConversationHistory object.""" + """Two calls for the same id return the same Conversation object.""" first, first_new = await mh._get_conversation("same-id") second, second_new = await mh._get_conversation("same-id") assert first is second @@ -94,7 +98,7 @@ async def test_get_conversation_corrupted_history_falls_back_to_empty(tmp_path: conv, is_new = await mh._get_conversation(cid) - assert len(conv) == 0 + assert len(conv.history) == 0 assert is_new is True diff --git a/tests/unit/tasks/test_notifier.py b/tests/unit/tasks/test_notifier.py index 5acf52c2..83843c7a 100644 --- a/tests/unit/tasks/test_notifier.py +++ b/tests/unit/tasks/test_notifier.py @@ -1,4 +1,4 @@ -"""Tests for tasks._notifier — Telegram push notifications.""" +"""Tests for tasks._notifier — Telegram push notifications via the broker.""" import os from pathlib import Path @@ -6,6 +6,7 @@ import pytest +from integrations.broker_client import IntegrationError from tasks._notifier import ( TelegramNotifier, format_run_completed, @@ -20,106 +21,102 @@ def _make_config(**overrides): return NotificationsConfig(**overrides) +_ENABLED_ENV = { + "TELEGRAM_INTEGRATION_ID": "telegram_personal", + "TELEGRAM_CHAT_ID": "42", +} + +_APP_SOCK = Path("/tmp/test_app.sock") + + @pytest.mark.unit class TestTelegramNotifier: """Test TelegramNotifier init and send behavior.""" def test_enables_when_env_vars_present(self): """Notifier is enabled when both env vars are set.""" - with patch.dict(os.environ, {"TELEGRAM_BOT_TOKEN": "tok", "TELEGRAM_CHAT_ID": "123"}): - notifier = TelegramNotifier(_make_config()) + with patch.dict(os.environ, _ENABLED_ENV): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) assert notifier.enabled + def test_disabled_when_env_vars_missing(self): + """Notifier disables itself with a warning when either env is unset.""" + with patch.dict(os.environ, {}, clear=True): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + assert not notifier.enabled + + def test_disabled_when_chat_id_not_numeric(self): + """A non-integer chat ID disables the notifier.""" + with patch.dict( + os.environ, + {"TELEGRAM_INTEGRATION_ID": "telegram_personal", "TELEGRAM_CHAT_ID": "not-a-number"}, + ): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + assert not notifier.enabled + async def test_send_noop_when_disabled(self): - """Sending on a disabled notifier is a silent no-op.""" + """Sending on a disabled notifier is a silent no-op (no broker call).""" with patch.dict(os.environ, {}, clear=True): - notifier = TelegramNotifier(_make_config()) - # Should not raise + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: await notifier.send("hello") + mock_call.assert_not_called() - async def test_send_calls_telegram_api(self): - """Sends a message via the Telegram Bot API.""" - with patch.dict(os.environ, {"TELEGRAM_BOT_TOKEN": "tok", "TELEGRAM_CHAT_ID": "42"}): - notifier = TelegramNotifier(_make_config()) - - mock_response = AsyncMock() - mock_response.status_code = 200 - - with patch("tasks._notifier.httpx.AsyncClient") as mock_client_cls: - mock_client = AsyncMock() - mock_client.post.return_value = mock_response - mock_client.__aenter__ = AsyncMock(return_value=mock_client) - mock_client.__aexit__ = AsyncMock(return_value=False) - mock_client_cls.return_value = mock_client + async def test_send_calls_broker(self): + """Sends a message via broker_client.call('send_message', ...).""" + with patch.dict(os.environ, _ENABLED_ENV): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: + mock_call.return_value = {"message_id": 1} await notifier.send("test message") - mock_client.post.assert_called_once() - call_args = mock_client.post.call_args - assert "/sendMessage" in call_args[0][0] - assert call_args[1]["json"]["chat_id"] == "42" - assert call_args[1]["json"]["text"] == "test message" - - async def test_send_document(self, tmp_path): - """Sends a file attachment via sendDocument.""" + mock_call.assert_awaited_once_with( + "telegram_personal", + "send_message", + {"chat_id": 42, "text": "test message"}, + app_sock_path=_APP_SOCK, + ) + + async def test_send_truncates_long_messages(self): + """Messages over the Telegram limit are truncated before sending.""" + with patch.dict(os.environ, _ENABLED_ENV): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: + mock_call.return_value = {"message_id": 1} + + await notifier.send("x" * 5000) + + args = mock_call.await_args.args + sent_text = args[2]["text"] + assert len(sent_text) <= 4096 + assert sent_text.endswith("… (truncated)") + + async def test_send_skips_attachments_for_now(self, tmp_path): + """Attachments are logged-and-skipped until broker send_document lands.""" test_file = tmp_path / "report.pdf" test_file.write_bytes(b"fake pdf content") - with patch.dict(os.environ, {"TELEGRAM_BOT_TOKEN": "tok", "TELEGRAM_CHAT_ID": "42"}): - notifier = TelegramNotifier(_make_config()) - - mock_response = AsyncMock() - mock_response.status_code = 200 - - with patch("tasks._notifier.httpx.AsyncClient") as mock_client_cls: - mock_client = AsyncMock() - mock_client.post.return_value = mock_response - mock_client.__aenter__ = AsyncMock(return_value=mock_client) - mock_client.__aexit__ = AsyncMock(return_value=False) - mock_client_cls.return_value = mock_client + with patch.dict(os.environ, _ENABLED_ENV): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: + mock_call.return_value = {"message_id": 1} await notifier.send("msg", attachments=[test_file]) - assert mock_client.post.call_count == 2 - send_doc_call = mock_client.post.call_args_list[1] - assert "/sendDocument" in send_doc_call[0][0] - - async def test_skips_large_files(self, tmp_path): - """Files exceeding max_attachment_size_mb are skipped.""" - test_file = tmp_path / "huge.bin" - test_file.write_bytes(b"x" * 100) - - with patch.dict(os.environ, {"TELEGRAM_BOT_TOKEN": "tok", "TELEGRAM_CHAT_ID": "42"}): - # Set max to 0 MB so the 100-byte file exceeds it - notifier = TelegramNotifier(_make_config(max_attachment_size_mb=0)) - - mock_response = AsyncMock() - mock_response.status_code = 200 - - with patch("tasks._notifier.httpx.AsyncClient") as mock_client_cls: - mock_client = AsyncMock() - mock_client.post.return_value = mock_response - mock_client.__aenter__ = AsyncMock(return_value=mock_client) - mock_client.__aexit__ = AsyncMock(return_value=False) - mock_client_cls.return_value = mock_client - - await notifier.send("msg", attachments=[test_file]) - - # Only sendMessage, no sendDocument - assert mock_client.post.call_count == 1 - - async def test_send_does_not_raise_on_error(self): - """Errors in send are logged, never raised.""" - with patch.dict(os.environ, {"TELEGRAM_BOT_TOKEN": "tok", "TELEGRAM_CHAT_ID": "42"}): - notifier = TelegramNotifier(_make_config()) - - with patch("tasks._notifier.httpx.AsyncClient") as mock_client_cls: - mock_client = AsyncMock() - mock_client.post.side_effect = ConnectionError("offline") - mock_client.__aenter__ = AsyncMock(return_value=mock_client) - mock_client.__aexit__ = AsyncMock(return_value=False) - mock_client_cls.return_value = mock_client - + # Only the text send_message call; no document call. + assert mock_call.await_count == 1 + assert mock_call.await_args.args[1] == "send_message" + + async def test_send_does_not_raise_on_broker_error(self): + """Errors from the broker are logged, never raised.""" + with patch.dict(os.environ, _ENABLED_ENV): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + with patch( + "tasks._notifier.broker_call", + new_callable=AsyncMock, + side_effect=IntegrationError("broker offline"), + ): # Should not raise await notifier.send("test") diff --git a/tools/memory/__init__.py b/tools/memory/__init__.py index 4c3847f2..cd240309 100644 --- a/tools/memory/__init__.py +++ b/tools/memory/__init__.py @@ -1,5 +1,12 @@ """Persistent key-value memory tools for COMPUTRON.""" -from .memory import MemoryEntry, forget, load_memory, remember, set_key_hidden +from .memory import MemoryEntry, forget, load_memory, memory_prompt_block, remember, set_key_hidden -__all__ = ["MemoryEntry", "forget", "load_memory", "remember", "set_key_hidden"] +__all__ = [ + "MemoryEntry", + "forget", + "load_memory", + "memory_prompt_block", + "remember", + "set_key_hidden", +] diff --git a/tools/memory/memory.py b/tools/memory/memory.py index 883c1c93..1da19de0 100644 --- a/tools/memory/memory.py +++ b/tools/memory/memory.py @@ -58,6 +58,23 @@ def load_memory() -> dict[str, MemoryEntry]: return _load_raw() +def memory_prompt_block() -> str: + """Return a formatted memory block to prepend to a system prompt. + + Empty string when no memories are stored — callers can concatenate + unconditionally. + """ + memory = _load_raw() + if not memory: + return "" + lines = "\n".join(f" {k}: {e.value}" for k, e in memory.items()) + sep = "─" * 64 + return ( + f"\n── Memory (persisted across sessions) " + f"──────────────────────────────────────────\n{lines}\n{sep}\n" + ) + + def set_key_hidden(key: str, hidden: bool) -> None: """Mark a memory key as hidden or visible in the UI.""" data = _load_raw() diff --git a/uv.lock b/uv.lock index f7469fdd..1f4af395 100644 --- a/uv.lock +++ b/uv.lock @@ -6,6 +6,32 @@ resolution-markers = [ "python_full_version < '3.13'", ] +[[package]] +name = "aiofiles" +version = "25.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/41/c3/534eac40372d8ee36ef40df62ec129bee4fdb5ad9706e58a29be53b2c970/aiofiles-25.1.0.tar.gz", hash = "sha256:a8d728f0a29de45dc521f18f07297428d56992a742f0cd2701ba86e44d23d5b2", size = 46354, upload-time = "2025-10-09T20:51:04.358Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/bc/8a/340a1555ae33d7354dbca4faa54948d76d89a27ceef032c8c3bc661d003e/aiofiles-25.1.0-py3-none-any.whl", hash = "sha256:abe311e527c862958650f9438e859c1fa7568a141b22abcd015e120e86a85695", size = 14668, upload-time = "2025-10-09T20:51:03.174Z" }, +] + +[[package]] +name = "aiogram" +version = "3.28.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "aiofiles" }, + { name = "aiohttp" }, + { name = "certifi" }, + { name = "magic-filter" }, + { name = "pydantic" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/db/3c/72e62c25a7fffedd7dd869e193797b0ed5207041ca90de16bf56fd99e41a/aiogram-3.28.2.tar.gz", hash = "sha256:140913a569516811c4ce576a292491024c8d8a739c47150d64a05aa4adc35310", size = 1855159, upload-time = "2026-05-10T14:20:40.384Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c9/7b/8bc61691b1fc9d1653e12697436b2742dd1408f14c479d15f46d1ab34be1/aiogram-3.28.2-py3-none-any.whl", hash = "sha256:8a90cdf75a64ba629468f73b6e999662943fff39db213bb406afe114a3e8c0f2", size = 751693, upload-time = "2026-05-10T14:20:38.265Z" }, +] + [[package]] name = "aiohappyeyeballs" version = "2.6.1" @@ -306,6 +332,7 @@ name = "computron-9000" version = "0.1.0" source = { editable = "." } dependencies = [ + { name = "aiogram" }, { name = "aiohttp" }, { name = "anthropic" }, { name = "beautifulsoup4" }, @@ -344,6 +371,7 @@ test = [ [package.metadata] requires-dist = [ + { name = "aiogram", specifier = ">=3.7" }, { name = "aiohttp" }, { name = "anthropic", specifier = ">=0.52.0" }, { name = "beautifulsoup4" }, @@ -1008,6 +1036,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d9/76/7ffc1d3005cf7749123bc47cb3ea343cd97b0ac2211bab40f57283577d0e/lxml_html_clean-0.4.4-py3-none-any.whl", hash = "sha256:ce2ef506614ecb85ee1c5fe0a2aa45b06a19514ec7949e9c8f34f06925cfabcb", size = 14565, upload-time = "2026-02-27T09:35:51.86Z" }, ] +[[package]] +name = "magic-filter" +version = "1.0.12" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e6/08/da7c2cc7398cc0376e8da599d6330a437c01d3eace2f2365f300e0f3f758/magic_filter-1.0.12.tar.gz", hash = "sha256:4751d0b579a5045d1dc250625c4c508c18c3def5ea6afaf3957cb4530d03f7f9", size = 11071, upload-time = "2023-10-01T12:33:19.006Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/cc/75/f620449f0056eff0ec7c1b1e088f71068eb4e47a46eb54f6c065c6ad7675/magic_filter-1.0.12-py3-none-any.whl", hash = "sha256:e5929e544f310c2b1f154318db8c5cdf544dd658efa998172acd2e4ba0f6c6a6", size = 11335, upload-time = "2023-10-01T12:33:17.711Z" }, +] + [[package]] name = "markdown-it-py" version = "4.0.0" From bcc302006a03d722d1fdd00e6c2365ee22e8063f Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Sun, 24 May 2026 21:30:03 -0500 Subject: [PATCH 02/12] =?UTF-8?q?feat:=20reactive=20Telegram=20channel=20d?= =?UTF-8?q?iscovery=20=E2=80=94=20wake=20on=20integration=20add?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Channels boot once at app start, but the Telegram integration is added later via the wizard. Previous shape did a one-shot discovery at boot and required an app restart to pick up a freshly-added integration. Replace the one-shot with an event-driven wait: - TelegramChannel exposes ``notify_integration_added(slug)``. Sets an internal asyncio.Event when slug == "telegram"; ignores everything else so other integrations don't thrash the channel. - The supervise task tries discovery once. If nothing is found, it parks on the event. No polling. - server/_integrations_routes.handle_add_integration calls ``notify_integration_added`` after a successful add. The channel wakes, re-runs discovery, finds the new integration, probes the broker, and transitions into the pull loop. All without an app restart. - stop() also sets the event so a shutdown while waiting unblocks cleanly. Co-Authored-By: Claude Opus 4.7 (1M context) --- channels/telegram/_runner.py | 61 +++++++++++++++++++++++++++++----- server/_integrations_routes.py | 6 ++++ 2 files changed, 58 insertions(+), 9 deletions(-) diff --git a/channels/telegram/_runner.py b/channels/telegram/_runner.py index 486c54a2..ce2ec47d 100644 --- a/channels/telegram/_runner.py +++ b/channels/telegram/_runner.py @@ -78,24 +78,43 @@ def __init__(self, *, app_sock_path: Path) -> None: self._pull_task: asyncio.Task[None] | None = None self._turn_tasks: set[asyncio.Task[None]] = set() self._stopping = False + # Signalled by ``notify_integration_added`` so the supervise loop + # wakes the moment the user finishes the wizard. Default-low; the + # supervise loop clears it before each retry. + self._integration_added_event = asyncio.Event() + + def notify_integration_added(self, slug: str) -> None: + """Wake the supervise loop after a relevant integration is added. + + The integrations HTTP route calls this on every successful add. + Filters by slug so adding Gmail or iCloud doesn't trigger any + Telegram-side work. + """ + if slug == "telegram": + self._integration_added_event.set() # -- lifecycle ------------------------------------------------------ async def start(self) -> None: - """Auto-discover a Telegram integration and start the pull loop. + """Launch the supervisor task; discovery happens lazily. - Queries the supervisor for any integration with slug=telegram. If - none is registered, logs and exits — there's nothing to drive. If - more than one exists, binds to the first and notes the choice so the - behavior is explicit. + Channels boot once at app start, but integrations can be added at any + time via the wizard. So discovery runs inside a background task that + polls until a Telegram integration appears, then transitions into the + pull loop. ``start()`` itself returns immediately. """ - integration_id = await self._discover_integration() + self._pull_task = asyncio.create_task( + self._supervise(), name="telegram-supervise", + ) + + async def _supervise(self) -> None: + """Wait for a Telegram integration to exist, probe it, then pull.""" + integration_id = await self._wait_for_integration() if integration_id is None: - logger.info("No Telegram integration registered; channel not starting") + # Stopping was requested before discovery succeeded. return self._integration_id = integration_id - # Identity probe — confirms the broker is reachable and authenticated. try: me = await broker_call( integration_id, "get_me", {}, app_sock_path=self._app_sock, @@ -124,7 +143,27 @@ async def start(self) -> None: "telegram channel started integration_id=%s bot=@%s (id=%d)", integration_id, me.get("username"), me.get("id"), ) - self._pull_task = asyncio.create_task(self._pull_loop(), name="telegram-pull") + await self._pull_loop() + + async def _wait_for_integration(self) -> str | None: + """Discover or wait for a Telegram integration. + + No polling: discovery runs once. If nothing's registered, the + supervise task parks on an event that the integrations HTTP route + sets when a Telegram integration is added. Returns the integration + id, or ``None`` if the channel was stopped while waiting. + """ + while not self._stopping: + integration_id = await self._discover_integration() + if integration_id is not None: + return integration_id + logger.info( + "No Telegram integration registered; channel waiting for one " + "to be added via the wizard", + ) + self._integration_added_event.clear() + await self._integration_added_event.wait() + return None async def _discover_integration(self) -> str | None: """Pick the telegram integration the channel should bind to. @@ -161,6 +200,10 @@ async def _discover_integration(self) -> str | None: async def stop(self) -> None: """Stop the pull loop and any in-flight turns.""" self._stopping = True + # Wake the supervise loop if it's parked waiting for an integration — + # cancel alone is enough, but setting the event keeps the code path + # symmetric with the add-side wake. + self._integration_added_event.set() if self._pull_task is not None: self._pull_task.cancel() with suppress(asyncio.CancelledError): diff --git a/server/_integrations_routes.py b/server/_integrations_routes.py index 806abab4..66b4345c 100644 --- a/server/_integrations_routes.py +++ b/server/_integrations_routes.py @@ -166,6 +166,12 @@ async def handle_add_integration(request: web.Request) -> web.Response: result.get("state") or "running", ) + # Wake channels that were waiting on a relevant integration to appear. + # Each channel filters by slug so unrelated adds are no-ops. + telegram_runner = request.app.get("telegram_bot_runner") + if telegram_runner is not None: + telegram_runner.notify_integration_added(slug) + return web.json_response(result, status=201) From 2d55683137691eb130e801110dc8b6c611bcafda Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Sun, 24 May 2026 21:39:01 -0500 Subject: [PATCH 03/12] fix: survive broker death mid-call in broker_client + channel pull loop Two unhandled exception classes were crashing the Telegram pull task when an integration was removed and re-added (broker process killed + socket unlinked, then respawned with a fresh socket): - broker_client._rpc_one_shot didn't catch the asyncio.IncompleteReadError fired when the broker dies mid-call, nor the FileNotFoundError / ConnectionRefusedError that fires when the socket has been unlinked. Wrap both in IntegrationError so callers' normal retry-with-backoff paths handle them. - TelegramChannel._pull_loop gains an except Exception catch-all that logs with traceback and backs off, so any future surprise exception doesn't silently end the pull task and leave the bot unreachable until app restart. Symptom that surfaced this: removing the Telegram integration while the channel was inside a long-poll left the pull task dead even after re-adding the integration spawned a fresh broker on the same socket. The fix lets the loop catch the disconnect, backoff, and reconnect on the next iteration. Co-Authored-By: Claude Opus 4.7 (1M context) --- channels/telegram/_runner.py | 11 +++++++++++ integrations/broker_client/_call.py | 25 +++++++++++++++++++++---- 2 files changed, 32 insertions(+), 4 deletions(-) diff --git a/channels/telegram/_runner.py b/channels/telegram/_runner.py index ce2ec47d..3d5d86a0 100644 --- a/channels/telegram/_runner.py +++ b/channels/telegram/_runner.py @@ -252,6 +252,17 @@ async def _pull_loop(self) -> None: await asyncio.sleep(backoff) backoff = min(backoff * 2, _BACKOFF_CAP_SECONDS) continue + except Exception: + # Defensive: anything else is a bug, but a crashed pull task + # silently stops the bot until app restart. Log with traceback + # and back off rather than die. + logger.exception( + "telegram pull loop: unexpected error (retrying in %.1fs)", + backoff, + ) + await asyncio.sleep(backoff) + backoff = min(backoff * 2, _BACKOFF_CAP_SECONDS) + continue for update in result.get("updates", []): await self._dispatch(update) diff --git a/integrations/broker_client/_call.py b/integrations/broker_client/_call.py index 80d4b4a6..081a4f4d 100644 --- a/integrations/broker_client/_call.py +++ b/integrations/broker_client/_call.py @@ -19,6 +19,7 @@ from __future__ import annotations import asyncio +from contextlib import suppress from pathlib import Path from typing import Any @@ -106,12 +107,20 @@ async def _rpc_one_shot( """Open a UDS, send one frame, read one frame, close. Wraps the framing helpers in ``integrations._rpc`` so the two hops above - aren't repeating the same 8 lines of connection plumbing. + aren't repeating the same 8 lines of connection plumbing. Maps every + connection/IO failure to ``IntegrationError`` so callers don't have to + catch low-level asyncio exceptions to survive a broker dying mid-call + or a socket being unlinked between integration remove/re-add cycles. """ - reader, writer = await asyncio.open_unix_connection(str(socket_path)) try: - await write_frame(writer, frame) + reader, writer = await asyncio.open_unix_connection(str(socket_path)) + except (FileNotFoundError, ConnectionRefusedError, OSError) as exc: + raise IntegrationError( + f"connect to {socket_path} failed: {exc}", + ) from exc + try: try: + await write_frame(writer, frame) return await read_frame(reader) except RpcError as exc: # A malformed response from the broker / supervisor is a protocol @@ -121,6 +130,14 @@ async def _rpc_one_shot( raise IntegrationError( f"malformed response from {socket_path}: {exc.code}: {exc.message}", ) from exc + except (asyncio.IncompleteReadError, ConnectionError, OSError) as exc: + # Broker died (or socket was unlinked) after we connected — common + # during integration remove/re-add or a broker crash + supervisor + # respawn. Let the caller's normal retry-with-backoff path handle it. + raise IntegrationError( + f"connection to {socket_path} dropped: {exc}", + ) from exc finally: writer.close() - await writer.wait_closed() + with suppress(Exception): + await writer.wait_closed() From 58f827fe08064f0d44b8c293947984bcc42c5435 Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Sun, 24 May 2026 21:44:06 -0500 Subject: [PATCH 04/12] =?UTF-8?q?fix:=20use=20payload.type,=20not=20event.?= =?UTF-8?q?type=20=E2=80=94=20AgentEvent=20has=20no=20.type=20field?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Real-bot test traceback: AttributeError: 'AgentEvent' object has no attribute 'type' The Telegram channel's event-dispatch loop and the spawn_agent event-accumulation loop both reached for ``event.type`` to switch on event kind. AgentEvent is just the envelope — type lives on the discriminated ``payload`` (``event.payload.type``). Why tests didn't catch it: the unit tests construct AgentEvent with real payload subclasses, but the dispatch code paths weren't exercised end-to-end against a real event stream until a Telegram turn actually ran in the manual-test container. Unit coverage for "channel dispatches on event kind" would have caught this. Co-Authored-By: Claude Opus 4.7 (1M context) --- channels/telegram/_runner.py | 11 ++++++----- sdk/tools/_spawn_agent.py | 5 ++--- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/channels/telegram/_runner.py b/channels/telegram/_runner.py index 3d5d86a0..9aa2c6e4 100644 --- a/channels/telegram/_runner.py +++ b/channels/telegram/_runner.py @@ -437,19 +437,20 @@ async def _run_turn(self, chat_id: int, text: str) -> None: profile_name=profile.name, ): payload = event.payload - if event.type == "tool_call" and hasattr(payload, "name"): + ptype = payload.type + if ptype == "tool_call" and hasattr(payload, "name"): await status.set(f"🔧 Calling {payload.name}...") - elif event.type == "agent_started" and hasattr(payload, "agent_name"): + elif ptype == "agent_started" and hasattr(payload, "agent_name"): await status.set(f"🚀 Spawning {payload.agent_name}...") - elif event.type == "agent_completed": + elif ptype == "agent_completed": await status.set(_STATUS_THINKING) - elif event.type == "content" and hasattr(payload, "content"): + elif ptype == "content" and hasattr(payload, "content"): if payload.content: collected_text += payload.content if not wrote_started: wrote_started = True await status.set(_STATUS_WRITING) - elif event.type == "file_output" and hasattr(payload, "path"): + elif ptype == "file_output" and hasattr(payload, "path"): if payload.path: file_paths.append(payload.path) except Exception: diff --git a/sdk/tools/_spawn_agent.py b/sdk/tools/_spawn_agent.py index bf687bea..f6e4a232 100644 --- a/sdk/tools/_spawn_agent.py +++ b/sdk/tools/_spawn_agent.py @@ -205,9 +205,8 @@ async def spawn_agent( sub_agent_id=short_id, correlation_id=correlation_id, ): - if event.type == "content" and isinstance(event.payload, ContentPayload): - if event.payload.content: - accumulated.append(event.payload.content) + if isinstance(event.payload, ContentPayload) and event.payload.content: + accumulated.append(event.payload.content) except StopRequestedError: logger.info("Spawned agent '%s' stopped by user request", agent_name) raise From b41d6e40f08a135c9648a5b14a7657c5b262c653 Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Mon, 25 May 2026 09:08:51 -0500 Subject: [PATCH 05/12] feat: Telegram inbound photos and documents MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The broker downloads photo/document attachments eagerly to the shared "downloads" host-path role (same place email attachments and browser saves land). The channel sees a local path on each update and feeds the agent an augmented user message that names the files — same shape SSE attachments use, so the agent has one mental model. Broker: - catalog entry gains a host_paths binding: role="downloads", env_var="DOWNLOADS_DIR", mode="write". - __main__ reads DOWNLOADS_DIR and threads it through to UpdatePump. - _updates._message_to_dict forwards messages with text, caption, or any supported attachment (was: text-only). - _extract_attachment_meta picks the largest photo size and any document, returns {kind, file_id, file_name, mime_type, ...}. - Filenames sanitized — path separators stripped, unsafe chars collapsed to "_", length-capped at 120. - UpdatePump._handle_update is now async. Downloads each attachment via bot.get_file + bot.download into DOWNLOADS_DIR before enqueueing. Download failures drop the attachment from the wire shape but keep the message (caption + successful peers still reach the agent). Channel: - _dispatch_message tolerates attachments-without-text and text-with-attachments. Builds the agent-facing user_content via a new _build_user_content helper that mirrors SSE's _augment_message_with_attachments shape. - Existing turn flow unchanged; the augmented text goes straight into TurnExecutor.execute(user_content=...). Out of scope (future): - Voice notes / video / stickers / animations - File size limits (Telegram caps at 20MB via getFile) - Cleanup of accumulated files in the downloads dir - Caption-only message with no attachments uses the text-only path (already worked) Co-Authored-By: Claude Opus 4.7 (1M context) --- channels/telegram/_runner.py | 43 +++++- .../brokers/telegram_broker/__main__.py | 7 +- .../brokers/telegram_broker/_updates.py | 138 ++++++++++++++++-- integrations/supervisor/_catalog.py | 7 + .../brokers/telegram_broker/test_updates.py | 41 +++--- .../brokers/telegram_broker/test_verbs.py | 2 + 6 files changed, 206 insertions(+), 32 deletions(-) diff --git a/channels/telegram/_runner.py b/channels/telegram/_runner.py index 9aa2c6e4..caff918e 100644 --- a/channels/telegram/_runner.py +++ b/channels/telegram/_runner.py @@ -279,7 +279,8 @@ async def _dispatch(self, update: dict[str, Any]) -> None: async def _dispatch_message(self, update: dict[str, Any]) -> None: chat_id = update["chat_id"] - text = update["text"] + text = update.get("text") + attachments = update.get("attachments") or [] if update.get("is_command"): cmd = text.split(maxsplit=1)[0][1:] # strip leading "/" @@ -295,14 +296,22 @@ async def _dispatch_message(self, update: dict[str, Any]) -> None: await self._send_text(chat_id, f"Unknown command: /{cmd}") return - # Regular text — spawn a turn task if one isn't already running. + # Text only, attachments only, or both — all flow through one turn. + # Build the augmented user_content here so the agent sees a single + # human-readable description rather than a flag dict. + user_content = _build_user_content(text, attachments) + if not user_content: + # Nothing actionable (would be unusual — broker already filters + # bare messages with neither text nor attachments). + return + conv_id = self._state.get(chat_id) if is_turn_active(conv_id): await self._send_text( chat_id, "⏳ A turn is already running. Use /stop to cancel it.", ) return - task = asyncio.create_task(self._run_turn(chat_id, text)) + task = asyncio.create_task(self._run_turn(chat_id, user_content)) self._turn_tasks.add(task) task.add_done_callback(self._turn_tasks.discard) @@ -612,6 +621,34 @@ async def _answer_callback( logger.debug("telegram answer_callback_query failed: %s", exc) +def _build_user_content( + text: str | None, attachments: list[dict[str, Any]], +) -> str: + """Compose the agent-facing user message from text + attachment paths. + + Mirrors the SSE channel's ``_augment_message_with_attachments`` shape so + the agent sees consistent file references regardless of how the message + arrived (web upload vs. Telegram document). + """ + body = (text or "").strip() + if not attachments: + return body + lines: list[str] = [] + for att in attachments: + path = att.get("path") + if not path: + continue + name = att.get("file_name") or path.rsplit("/", 1)[-1] + mime = att.get("mime_type") or "application/octet-stream" + lines.append(f" - {name} ({mime}) -> {path}") + if not lines: + return body + files_block = "\n".join(lines) + if body: + return f"{body}\n\n[Attached files written to virtual computer]\n{files_block}" + return f"[Attached files written to virtual computer]\n{files_block}" + + class _StatusMessage: """A live status message that the channel edits in place during a turn. diff --git a/integrations/brokers/telegram_broker/__main__.py b/integrations/brokers/telegram_broker/__main__.py index c9a8df5b..2d05dcd9 100644 --- a/integrations/brokers/telegram_broker/__main__.py +++ b/integrations/brokers/telegram_broker/__main__.py @@ -66,6 +66,7 @@ async def _run() -> int: token = env_required("TELEGRAM_BOT_TOKEN") permissions = permissions_from_env(env_required("PERMISSIONS")) allowed = _parse_allowed_user_ids(env_required("TELEGRAM_ALLOWED_USER_IDS")) + downloads_dir = Path(env_required("DOWNLOADS_DIR")) # Wipe the token from the process environ once captured. Best-effort # hygiene: narrows in-process exposure (debuggers, traceback locals, @@ -104,7 +105,11 @@ async def _run() -> int: me.username, me.id, sorted(allowed), ) - pump = UpdatePump(bot, allowed, integration_id=integration_id) + pump = UpdatePump( + bot, allowed, + integration_id=integration_id, + downloads_dir=downloads_dir, + ) pump_task = asyncio.create_task(pump.run(), name="telegram-update-pump") dispatcher = VerbDispatcher(bot, pump, permissions=permissions) diff --git a/integrations/brokers/telegram_broker/_updates.py b/integrations/brokers/telegram_broker/_updates.py index 936255d6..d12faedd 100644 --- a/integrations/brokers/telegram_broker/_updates.py +++ b/integrations/brokers/telegram_broker/_updates.py @@ -6,13 +6,18 @@ Drops from non-allowlisted senders are silent — replying would confirm the bot is live and waste outbound rate limits. Drop counts are logged periodically rather than per-message so a spam burst doesn't spam the log. + +Photo and document attachments are downloaded eagerly before the update is +enqueued — the wire shape carries a local path the channel can read. """ from __future__ import annotations import asyncio import logging +import re import time +from pathlib import Path from typing import Any from aiogram import Bot @@ -36,12 +41,23 @@ _BACKOFF_INITIAL_SECONDS = 1.0 _BACKOFF_CAP_SECONDS = 30.0 +# Sanitize document filenames before writing — strip path separators and any +# character that isn't safe in a filename. Length-cap so a hostile sender +# can't blow out the directory listing. +_FILENAME_UNSAFE = re.compile(r"[^A-Za-z0-9._-]+") +_FILENAME_MAX_LEN = 120 + def update_to_dict(update: Any) -> dict[str, Any] | None: """Flatten an aiogram ``Update`` to the wire shape, or ``None`` to skip. - Forwards text ``message`` updates and ``callback_query`` updates - (inline-keyboard button taps). Other update kinds are dropped. + Forwards text/caption ``message`` updates (including photo/document + attachments) and ``callback_query`` updates. Other update kinds are + dropped. + + Attachments are returned with metadata + a ``file_id`` placeholder; + ``UpdatePump`` downloads the bytes asynchronously before the update is + queued and populates ``path`` on each attachment. """ message = getattr(update, "message", None) if message is not None: @@ -53,22 +69,86 @@ def update_to_dict(update: Any) -> dict[str, Any] | None: def _message_to_dict(message: Any) -> dict[str, Any] | None: - text = getattr(message, "text", None) - if not text: - return None + """Map a ``message`` update to the wire shape. + + A message qualifies for forwarding if it has text, a caption, or at + least one supported attachment. ``text`` is the user-typed content + (either ``message.text`` for plain messages or ``message.caption`` when + a photo/document was sent with explanatory text). + """ user = message.from_user if user is None: return None - return { + text = getattr(message, "text", None) or getattr(message, "caption", None) + attachments = _extract_attachment_meta(message) + if not text and not attachments: + return None + payload: dict[str, Any] = { "type": "message", "message_id": message.message_id, "chat_id": message.chat.id, "from_user_id": user.id, "from_username": getattr(user, "username", None), "text": text, - "is_command": text.startswith("/"), + "is_command": bool(text and text.startswith("/")), "timestamp": int(message.date.timestamp()) if message.date else int(time.time()), } + if attachments: + payload["attachments"] = attachments + return payload + + +def _extract_attachment_meta(message: Any) -> list[dict[str, Any]]: + """Return a list of attachment-metadata dicts for this message. + + Photos arrive as multiple sizes; we keep the largest. Documents arrive + as a single object with its own filename and mime type. Other kinds + (voice, video, sticker, animation) are intentionally dropped today. + """ + out: list[dict[str, Any]] = [] + + photos = getattr(message, "photo", None) or [] + if photos: + # Photo sizes are returned smallest-to-largest; pick the last. + biggest = photos[-1] + out.append({ + "kind": "photo", + "file_id": biggest.file_id, + "file_unique_id": getattr(biggest, "file_unique_id", ""), + "file_size": getattr(biggest, "file_size", None), + "file_name": _photo_filename(biggest), + "mime_type": "image/jpeg", + "width": getattr(biggest, "width", None), + "height": getattr(biggest, "height", None), + }) + + document = getattr(message, "document", None) + if document is not None: + out.append({ + "kind": "document", + "file_id": document.file_id, + "file_unique_id": getattr(document, "file_unique_id", ""), + "file_size": getattr(document, "file_size", None), + "file_name": _safe_filename( + getattr(document, "file_name", None) or f"document_{document.file_id}", + ), + "mime_type": getattr(document, "mime_type", None) or "application/octet-stream", + }) + + return out + + +def _photo_filename(photo: Any) -> str: + """Telegram photos have no user-supplied name; derive a stable one.""" + unique = getattr(photo, "file_unique_id", None) or getattr(photo, "file_id", "photo") + return _safe_filename(f"photo_{unique}.jpg") + + +def _safe_filename(name: str) -> str: + """Strip path separators and unsafe characters; cap length.""" + base = Path(name).name # drop any leading path + cleaned = _FILENAME_UNSAFE.sub("_", base).strip("._") or "file" + return cleaned[:_FILENAME_MAX_LEN] def _callback_to_dict(callback: Any) -> dict[str, Any] | None: @@ -105,10 +185,12 @@ def __init__( allowed_user_ids: frozenset[int], *, integration_id: str, + downloads_dir: Path, ) -> None: self._bot = bot self._allowed = allowed_user_ids self._integration_id = integration_id + self._downloads_dir = downloads_dir self.queue: asyncio.Queue[dict[str, Any]] = asyncio.Queue() # Aggregate drop tracking — keyed by sender so a single bad actor @@ -156,12 +238,12 @@ async def run(self) -> None: for update in updates: offset = update.update_id + 1 - self._handle_update(update) + await self._handle_update(update) self._maybe_flush_drop_log() - def _handle_update(self, update: Any) -> None: - """Filter and enqueue a single update.""" + async def _handle_update(self, update: Any) -> None: + """Filter, download any attachments, and enqueue a single update.""" payload = update_to_dict(update) if payload is None: return @@ -170,8 +252,44 @@ def _handle_update(self, update: Any) -> None: self._drop_counts.get(payload["from_user_id"], 0) + 1 ) return + if payload.get("attachments"): + await self._download_attachments(payload) self.queue.put_nowait(payload) + async def _download_attachments(self, payload: dict[str, Any]) -> None: + """Download each attachment's bytes; populate ``path`` on each. + + Failures drop the attachment from the wire shape rather than the + whole message — the agent still gets the text/caption and any + successful peers. + """ + kept: list[dict[str, Any]] = [] + for att in payload["attachments"]: + try: + local_path = await self._download_one(att) + except Exception: + logger.exception( + "telegram attachment download failed file_id=%s kind=%s", + att.get("file_id"), att.get("kind"), + ) + continue + att["path"] = str(local_path) + att.pop("file_id", None) # drop the internal handle from the wire shape + kept.append(att) + payload["attachments"] = kept + + async def _download_one(self, att: dict[str, Any]) -> Path: + """Pull the file's bytes from Telegram into the downloads dir.""" + file_id = att["file_id"] + file_name = att["file_name"] + tg_file = await self._bot.get_file(file_id) + dest = self._downloads_dir / file_name + # If the dir doesn't exist yet, create it — the supervisor's + # entrypoint normally sets perms but be defensive. + dest.parent.mkdir(parents=True, exist_ok=True) + await self._bot.download(tg_file, destination=dest) + return dest + def _maybe_flush_drop_log(self) -> None: """Emit aggregate drop counts once per window.""" now = time.monotonic() diff --git a/integrations/supervisor/_catalog.py b/integrations/supervisor/_catalog.py index 221779c1..3ed053f6 100644 --- a/integrations/supervisor/_catalog.py +++ b/integrations/supervisor/_catalog.py @@ -208,6 +208,13 @@ def resolve_capabilities(self, auth_blob: dict | None = None) -> dict[Capability "token": "TELEGRAM_BOT_TOKEN", "allowed_user_ids": "TELEGRAM_ALLOWED_USER_IDS", }, + host_paths=( + # Inbound photos/documents land in the shared "downloads" role — + # same place email attachments and browser saves are written. The + # broker writes; the main app process (and the agent's virtual + # computer view) reads from the same path. + HostPathBinding(role="downloads", env_var="DOWNLOADS_DIR", mode="write"), + ), ) diff --git a/tests/unit/integrations/brokers/telegram_broker/test_updates.py b/tests/unit/integrations/brokers/telegram_broker/test_updates.py index 1275fe0a..9f2e308f 100644 --- a/tests/unit/integrations/brokers/telegram_broker/test_updates.py +++ b/tests/unit/integrations/brokers/telegram_broker/test_updates.py @@ -175,12 +175,13 @@ def test_callback_without_carrier_message_is_dropped(self): # --------------------------------------------------------------------------- -def _make_pump(allowed_ids: frozenset[int]) -> UpdatePump: +def _make_pump(allowed_ids: frozenset[int], *, downloads_dir=None) -> UpdatePump: """UpdatePump with a stubbed Bot — _handle_update doesn't use the bot.""" return UpdatePump( bot=MagicMock(), allowed_user_ids=allowed_ids, integration_id="telegram_test", + downloads_dir=downloads_dir or __import__("pathlib").Path("/tmp"), ) @@ -199,42 +200,46 @@ def _msg_update(*, user_id: int, text: str = "hi", update_id: int = 1) -> _FakeU @pytest.mark.unit class TestUpdatePumpFilter: - def test_allowed_user_is_enqueued(self): + async def test_allowed_user_is_enqueued(self): pump = _make_pump(frozenset({100})) - pump._handle_update(_msg_update(user_id=100, text="hello")) + await pump._handle_update(_msg_update(user_id=100, text="hello")) assert pump.queue.qsize() == 1 out = pump.queue.get_nowait() assert out["from_user_id"] == 100 assert out["text"] == "hello" - def test_disallowed_user_is_dropped(self): + async def test_disallowed_user_is_dropped(self): pump = _make_pump(frozenset({100})) - pump._handle_update(_msg_update(user_id=999, text="spam")) + await pump._handle_update(_msg_update(user_id=999, text="spam")) assert pump.queue.qsize() == 0 # Drop accounted for. assert pump._drop_counts == {999: 1} - def test_disallowed_drops_accumulate_per_user(self): + async def test_disallowed_drops_accumulate_per_user(self): pump = _make_pump(frozenset({100})) for i in range(3): - pump._handle_update(_msg_update(user_id=999, text=f"spam{i}", update_id=i)) + await pump._handle_update( + _msg_update(user_id=999, text=f"spam{i}", update_id=i), + ) for i in range(2): - pump._handle_update(_msg_update(user_id=888, text=f"x{i}", update_id=100 + i)) + await pump._handle_update( + _msg_update(user_id=888, text=f"x{i}", update_id=100 + i), + ) assert pump.queue.qsize() == 0 assert pump._drop_counts == {999: 3, 888: 2} - def test_skippable_update_is_neither_queued_nor_counted(self): + async def test_skippable_update_is_neither_queued_nor_counted(self): # No-message update — update_to_dict returns None and the filter # path is bypassed entirely. Drops counter must not bump. pump = _make_pump(frozenset({100})) - pump._handle_update(_FakeUpdate(update_id=1, message=None)) + await pump._handle_update(_FakeUpdate(update_id=1, message=None)) assert pump.queue.qsize() == 0 assert pump._drop_counts == {} - def test_empty_allowlist_drops_everyone(self): + async def test_empty_allowlist_drops_everyone(self): pump = _make_pump(frozenset()) - pump._handle_update(_msg_update(user_id=1)) - pump._handle_update(_msg_update(user_id=2)) + await pump._handle_update(_msg_update(user_id=1)) + await pump._handle_update(_msg_update(user_id=2)) assert pump.queue.qsize() == 0 assert sum(pump._drop_counts.values()) == 2 @@ -242,18 +247,18 @@ def test_empty_allowlist_drops_everyone(self): @pytest.mark.unit class TestUpdatePumpDropLog: - def test_drop_log_does_not_flush_within_window(self, caplog): + async def test_drop_log_does_not_flush_within_window(self, caplog): pump = _make_pump(frozenset({100})) - pump._handle_update(_msg_update(user_id=999)) + await pump._handle_update(_msg_update(user_id=999)) # Calling flush immediately (window not elapsed) should not log # and should leave the counter intact. pump._maybe_flush_drop_log() assert pump._drop_counts == {999: 1} - def test_drop_log_flushes_and_resets_after_window(self, caplog, monkeypatch): + async def test_drop_log_flushes_and_resets_after_window(self, caplog, monkeypatch): pump = _make_pump(frozenset({100})) - pump._handle_update(_msg_update(user_id=999)) - pump._handle_update(_msg_update(user_id=888)) + await pump._handle_update(_msg_update(user_id=999)) + await pump._handle_update(_msg_update(user_id=888)) # Pretend the window started 9999 seconds ago so the flush fires. pump._drop_window_started_at = time.monotonic() - 9999.0 diff --git a/tests/unit/integrations/brokers/telegram_broker/test_verbs.py b/tests/unit/integrations/brokers/telegram_broker/test_verbs.py index 8f04927a..d70b44a8 100644 --- a/tests/unit/integrations/brokers/telegram_broker/test_verbs.py +++ b/tests/unit/integrations/brokers/telegram_broker/test_verbs.py @@ -20,10 +20,12 @@ def _make_pump() -> UpdatePump: + import pathlib return UpdatePump( bot=MagicMock(), allowed_user_ids=frozenset({1}), integration_id="telegram_test", + downloads_dir=pathlib.Path("/tmp"), ) From 6cf7f0be3c744382f2393ba4378ae19a1ec09952 Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Mon, 25 May 2026 10:57:17 -0500 Subject: [PATCH 06/12] fix: race in Telegram dispatch double-spawned turns on batched messages MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two Telegram messages arriving in one broker batch both passed the "is a turn already running?" guard because is_turn_active() reads a ContextVar that doesn't flip True until the turn task actually enters turn_scope. Between asyncio.create_task and the task running, the var is unset, so the dispatcher's second iteration in the same await window saw "no turn running" and spawned a duplicate. Replace the ContextVar check with a per-chat in-flight tasks dict the channel owns directly. Membership is updated synchronously before any await yields, so the second dispatch sees the first task immediately. The /stop and the channel's own use of is_turn_active() are unchanged (those want the "actually executing in turn_scope" semantics). Reproed by sending a file with a caption — Telegram delivered the media and the text as separate updates in one getUpdates batch, both of which went on to spawn turns for the same chat. Co-Authored-By: Claude Opus 4.7 (1M context) --- channels/telegram/_runner.py | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/channels/telegram/_runner.py b/channels/telegram/_runner.py index caff918e..04ecb2f8 100644 --- a/channels/telegram/_runner.py +++ b/channels/telegram/_runner.py @@ -77,6 +77,12 @@ def __init__(self, *, app_sock_path: Path) -> None: self._default_profile_id: str = "computron" self._pull_task: asyncio.Task[None] | None = None self._turn_tasks: set[asyncio.Task[None]] = set() + # Per-chat in-flight turn tracker. Owned by the dispatch loop so the + # "one turn at a time" check is race-free — ContextVar-based + # ``is_turn_active`` doesn't flip True until the task actually runs, + # so two messages in the same broker batch would both pass that + # check and double-spawn. + self._turn_by_chat: dict[int, asyncio.Task[None]] = {} self._stopping = False # Signalled by ``notify_integration_added`` so the supervise loop # wakes the moment the user finishes the wizard. Default-low; the @@ -305,15 +311,25 @@ async def _dispatch_message(self, update: dict[str, Any]) -> None: # bare messages with neither text nor attachments). return - conv_id = self._state.get(chat_id) - if is_turn_active(conv_id): + existing = self._turn_by_chat.get(chat_id) + if existing is not None and not existing.done(): await self._send_text( chat_id, "⏳ A turn is already running. Use /stop to cancel it.", ) return + task = asyncio.create_task(self._run_turn(chat_id, user_content)) self._turn_tasks.add(task) - task.add_done_callback(self._turn_tasks.discard) + self._turn_by_chat[chat_id] = task + + def _on_done(t: asyncio.Task[None]) -> None: + self._turn_tasks.discard(t) + # Only clear if this is still the current one — a /stop + new + # message could have already swapped a fresh task in. + if self._turn_by_chat.get(chat_id) is t: + self._turn_by_chat.pop(chat_id, None) + + task.add_done_callback(_on_done) async def _dispatch_callback(self, update: dict[str, Any]) -> None: """Handle an inline-keyboard button tap.""" From 70575d07fc0c33d4588c4c68d3457c79f529289a Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Mon, 25 May 2026 11:28:29 -0500 Subject: [PATCH 07/12] =?UTF-8?q?feat:=20/list=20command=20=E2=80=94=20sea?= =?UTF-8?q?rch=20and=20resume=20past=20Telegram=20conversations?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Inline-keyboard picker driven entirely by the existing on-disk conversation store. No new state, no new broker verb. The same mechanism the web UI's conversation list reads. Channel-side: - New /list [query] command. Lists conversations belonging to this chat (default ``telegram_`` plus any /new-spawned uuid variants), most-recent first, capped at 10. Optional query is a case-insensitive substring match against title and first message. - Tapping a row fires a callback with a new "conv:" prefix; the channel binds this chat to the picked conversation via a new ConversationMap.set() method, drops the in-memory cache so the next turn rehydrates from disk, and confirms. - _belongs_to_chat uses a strict-prefix check (``telegram_`` or ``telegram__…``) to avoid the false-positive a naive startswith hits when one chat_id is a prefix of another. - _resume_conversation validates the picked id actually belongs to this chat — a malicious callback payload can't trick the channel into binding to someone else's conversation. - /help updated to mention /list. Backed by: - New ``ConversationMap.set(chat_id, conv_id)`` — bind without resetting. - ``conversations.list_conversations()`` (existing) — single source of truth for the chat's conversation history. Co-Authored-By: Claude Opus 4.7 (1M context) --- channels/telegram/_runner.py | 150 +++++++++++++++++- channels/telegram/_state.py | 11 ++ .../channels/telegram/test_list_helpers.py | 100 ++++++++++++ tests/unit/channels/telegram/test_state.py | 12 ++ 4 files changed, 267 insertions(+), 6 deletions(-) create mode 100644 tests/unit/channels/telegram/test_list_helpers.py diff --git a/channels/telegram/_runner.py b/channels/telegram/_runner.py index 04ecb2f8..7266cac4 100644 --- a/channels/telegram/_runner.py +++ b/channels/telegram/_runner.py @@ -20,7 +20,12 @@ from channels.telegram._formatter import TelegramFormatter from channels.telegram._profile_map import ProfileMap from channels.telegram._state import ConversationMap -from conversations import DiskTurnPersistence, load_conversation_history +from conversations import ( + DiskTurnPersistence, + list_conversations, + load_conversation_history, +) +from conversations._models import ConversationSummary from integrations import supervisor_client from integrations.broker_client import ( IntegrationAuthFailed, @@ -57,6 +62,13 @@ # self-describing so unrelated callbacks (future verbs) don't collide. _PROFILE_CALLBACK_PREFIX = "profile:" +# callback_data prefix for picking a past conversation to resume. +_CONV_CALLBACK_PREFIX = "conv:" + +# How many past conversations to show in a /list reply. Keeps the inline +# keyboard within Telegram's reasonable-rendering range and the chat tidy. +_LIST_MAX_RESULTS = 10 + # Initial status text shown while we wait for the first event from the agent. _STATUS_THINKING = "🤔 Thinking..." _STATUS_WRITING = "✍️ Writing response..." @@ -289,13 +301,17 @@ async def _dispatch_message(self, update: dict[str, Any]) -> None: attachments = update.get("attachments") or [] if update.get("is_command"): - cmd = text.split(maxsplit=1)[0][1:] # strip leading "/" + parts = text.split(maxsplit=1) + cmd = parts[0][1:] # strip leading "/" + arg = parts[1] if len(parts) > 1 else "" if cmd == "new": await self._handle_new(chat_id) elif cmd == "stop": await self._handle_stop(chat_id) elif cmd == "profile": await self._handle_profile(chat_id) + elif cmd == "list": + await self._handle_list(chat_id, arg) elif cmd == "help": await self._handle_help(chat_id) else: @@ -342,6 +358,11 @@ async def _dispatch_callback(self, update: dict[str, Any]) -> None: await self._select_profile(chat_id, profile_id, callback_id) return + if data.startswith(_CONV_CALLBACK_PREFIX): + target = data[len(_CONV_CALLBACK_PREFIX):] + await self._resume_conversation(chat_id, target, callback_id) + return + # Unknown callback — dismiss the loading spinner anyway so the # Telegram client doesn't sit on it forever. await self._answer_callback(callback_id) @@ -396,13 +417,93 @@ async def _handle_profile(self, chat_id: int) -> None: async def _handle_help(self, chat_id: int) -> None: lines = [ "Commands:", - " /new — start a new conversation", - " /stop — stop the current turn", - " /profile — pick an agent profile for this chat", - " /help — this message", + " /new — start a new conversation", + " /stop — stop the current turn", + " /profile — pick an agent profile for this chat", + " /list [query] — list past conversations, optionally filtered", + " /help — this message", ] await self._send_text(chat_id, "\n".join(lines)) + async def _handle_list(self, chat_id: int, query: str) -> None: + """Show past conversations for this chat as an inline keyboard. + + Filters to conversations belonging to *chat_id* (default id + + ``/new``-spawned uuids). Optional ``query`` is a case-insensitive + substring match on title and first message — handy for "find that + conversation about taxes" without scrolling. + """ + query = query.strip() + summaries = list_conversations() + matches = [ + s for s in summaries + if _belongs_to_chat(s.conversation_id, chat_id) + and (not query or _matches_query(s, query)) + ][:_LIST_MAX_RESULTS] + + if not matches: + msg = ( + f"No conversations match '{query}'." + if query + else "No past conversations yet." + ) + await self._send_text(chat_id, msg) + return + + current = self._state.conversation_id_for(chat_id) + buttons = [ + [{ + "text": _row_label(s, is_current=s.conversation_id == current), + "data": f"{_CONV_CALLBACK_PREFIX}{s.conversation_id}", + }] + for s in matches + ] + header = ( + f"Conversations matching '{query}':" + if query + else "Recent conversations:" + ) + try: + await broker_call( + self._integration_id, + "send_message", + {"chat_id": chat_id, "text": header, "buttons": buttons}, + app_sock_path=self._app_sock, + ) + except IntegrationError as exc: + logger.warning("telegram /list failed chat_id=%s: %s", chat_id, exc) + + async def _resume_conversation( + self, chat_id: int, conv_id: str, callback_id: str, + ) -> None: + """Bind this chat to *conv_id* so the next message continues it. + + Validates that the picked conversation actually belongs to this + chat — a malicious callback payload can't trick the channel into + binding to an unrelated user's conversation. + """ + if not _belongs_to_chat(conv_id, chat_id): + logger.warning( + "telegram resume rejected — conv_id=%s not for chat_id=%s", + conv_id, chat_id, + ) + await self._answer_callback(callback_id, text="Not your conversation") + return + if load_conversation_history(conv_id) is None: + await self._answer_callback(callback_id, text="Conversation not found") + return + + self._state.set(chat_id, conv_id) + # Drop the cached in-memory Conversation so the next turn rehydrates + # the picked one's history from disk. + self._conversations.pop(conv_id, None) + await self._answer_callback(callback_id, text="Resumed") + await self._send_text(chat_id, f"📂 Resumed conversation: {conv_id}") + logger.info( + "telegram resumed conversation chat_id=%s conv_id=%s", + chat_id, conv_id, + ) + async def _select_profile( self, chat_id: int, profile_id: str, callback_id: str, ) -> None: @@ -637,6 +738,43 @@ async def _answer_callback( logger.debug("telegram answer_callback_query failed: %s", exc) +def _belongs_to_chat(conv_id: str, chat_id: int) -> bool: + """True iff ``conv_id`` was created by this Telegram chat. + + Default conv id is ``telegram_``; ``/new`` appends an + underscore + uuid. Strict prefix match on either form avoids the + false-positive a naive startswith would hit when one chat_id is a + prefix of another (e.g. ``8617`` vs ``8617268723``). + """ + prefix = f"telegram_{chat_id}" + return conv_id == prefix or conv_id.startswith(prefix + "_") + + +def _matches_query(summary: ConversationSummary, query: str) -> bool: + """Case-insensitive substring match on title and first message.""" + needle = query.lower() + return ( + needle in (summary.title or "").lower() + or needle in (summary.first_message or "").lower() + ) + + +def _row_label(summary: ConversationSummary, *, is_current: bool) -> str: + """Human-readable label for a /list inline-keyboard row.""" + if summary.title: + body = summary.title + elif summary.first_message: + # Some conversations don't get a title (turn errored before + # title-generation fired). Fall back to a clipped first message. + body = f"({summary.first_message[:50]}…)" + else: + body = f"(empty • {summary.conversation_id})" + # Telegram button labels render best within ~64 chars. + body = body[:60] + prefix = "✓ " if is_current else "" + return f"{prefix}{body}" + + def _build_user_content( text: str | None, attachments: list[dict[str, Any]], ) -> str: diff --git a/channels/telegram/_state.py b/channels/telegram/_state.py index 9f9c5de6..5291b4ce 100644 --- a/channels/telegram/_state.py +++ b/channels/telegram/_state.py @@ -24,6 +24,17 @@ def get(self, chat_id: int) -> str: """Return the conversation ID for *chat_id*, creating a default if absent.""" return self._map.setdefault(chat_id, f"telegram_{chat_id}") + # -- explicit bind (used by /list to resume a past conversation) ---- + + def set(self, chat_id: int, conversation_id: str) -> None: + """Bind ``chat_id`` to an existing conversation id. + + The next ``get`` returns this value; the channel hydrates the + history from disk on the cache miss. No effect on the conversation + store itself — only on which conversation this chat resolves to. + """ + self._map[chat_id] = conversation_id + # -- reset (used by /new) ------------------------------------------- def reset(self, chat_id: int) -> str: diff --git a/tests/unit/channels/telegram/test_list_helpers.py b/tests/unit/channels/telegram/test_list_helpers.py new file mode 100644 index 00000000..cb0ec99b --- /dev/null +++ b/tests/unit/channels/telegram/test_list_helpers.py @@ -0,0 +1,100 @@ +"""Tests for the /list helpers in channels.telegram._runner.""" + +from __future__ import annotations + +import pytest + +from channels.telegram._runner import _belongs_to_chat, _matches_query, _row_label +from conversations._models import ConversationSummary + + +def _summary( + *, + conversation_id: str = "telegram_42", + title: str = "", + first_message: str = "", +) -> ConversationSummary: + return ConversationSummary( + conversation_id=conversation_id, + title=title, + first_message=first_message, + started_at="2026-05-25T00:00:00+00:00", + turn_count=1, + ) + + +@pytest.mark.unit +class TestBelongsToChat: + + def test_default_id_matches(self): + assert _belongs_to_chat("telegram_42", 42) is True + + def test_uuid_suffix_matches(self): + assert _belongs_to_chat("telegram_42_a1b2c3d4", 42) is True + + def test_partial_chat_id_does_not_match(self): + # ``telegram_8617`` must NOT match chat_id 8617268723 — naive + # startswith would. + assert _belongs_to_chat("telegram_8617", 8617268723) is False + + def test_longer_chat_id_does_not_match(self): + # The other direction: ``telegram_86172687230`` (extra digit) + # must not match chat_id 8617268723. + assert _belongs_to_chat("telegram_86172687230", 8617268723) is False + + def test_unrelated_id_does_not_match(self): + assert _belongs_to_chat("chat_abc", 42) is False + assert _belongs_to_chat("telegram_99", 42) is False + + def test_empty_string_does_not_match(self): + assert _belongs_to_chat("", 42) is False + + +@pytest.mark.unit +class TestMatchesQuery: + + def test_matches_title_case_insensitive(self): + s = _summary(title="Trip planning") + assert _matches_query(s, "trip") is True + assert _matches_query(s, "TRIP") is True + assert _matches_query(s, "PLAN") is True + + def test_matches_first_message(self): + s = _summary(first_message="What's the weather in Paris?") + assert _matches_query(s, "weather") is True + assert _matches_query(s, "paris") is True + + def test_no_match(self): + s = _summary(title="Trip planning", first_message="Looking at flights") + assert _matches_query(s, "taxes") is False + + def test_empty_fields_do_not_match_arbitrary_query(self): + s = _summary() + assert _matches_query(s, "anything") is False + + +@pytest.mark.unit +class TestRowLabel: + + def test_title_used_when_present(self): + s = _summary(title="Trip planning") + assert _row_label(s, is_current=False) == "Trip planning" + + def test_first_message_fallback_when_no_title(self): + s = _summary(first_message="hello world") + assert _row_label(s, is_current=False) == "(hello world…)" + + def test_empty_fallback(self): + s = _summary(conversation_id="telegram_42_x") + assert _row_label(s, is_current=False) == "(empty • telegram_42_x)" + + def test_current_marker_prepended(self): + s = _summary(title="Now this one") + assert _row_label(s, is_current=True) == "✓ Now this one" + + def test_label_clipped_to_telegram_button_safe_length(self): + s = _summary(title="x" * 200) + assert len(_row_label(s, is_current=False)) <= 64 + s_current = _summary(title="x" * 200) + # current adds ✓ and a space; still keeps body to 60 chars. + assert _row_label(s_current, is_current=True).startswith("✓ ") diff --git a/tests/unit/channels/telegram/test_state.py b/tests/unit/channels/telegram/test_state.py index f2a6966c..7ed22822 100644 --- a/tests/unit/channels/telegram/test_state.py +++ b/tests/unit/channels/telegram/test_state.py @@ -58,3 +58,15 @@ def test_conversation_id_for_reflects_reset(self): cmap.get(15) new = cmap.reset(15) assert cmap.conversation_id_for(15) == new + + def test_set_binds_chat_to_existing_id(self): + cmap = ConversationMap() + cmap.set(42, "telegram_42_abc123") + assert cmap.get(42) == "telegram_42_abc123" + + def test_set_overwrites_any_previous_binding(self): + cmap = ConversationMap() + cmap.get(42) # default id + cmap.reset(42) # uuid id + cmap.set(42, "telegram_42_explicit") # explicit override + assert cmap.conversation_id_for(42) == "telegram_42_explicit" From 12e5f420d269f6c57f1b230d72b75b4ebf6c5513 Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Mon, 25 May 2026 11:32:47 -0500 Subject: [PATCH 08/12] refactor: /list shows all conversations, not telegram-only MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit User expectation was that /list surfaces every past conversation regardless of origin — the conversation store is the single source of truth and matches the web UI's listing behavior. Removed the per-chat prefix filter from /list and from the resume safety check. Callback data is bot-controlled (only valid conversation ids ever get put on buttons) so no per-chat prefix check is needed on the resume path. The disk-existence check still catches a stale callback whose conversation was deleted between list-and-tap. Dropped the now-unused _belongs_to_chat helper and its tests. Co-Authored-By: Claude Opus 4.7 (1M context) --- channels/telegram/_runner.py | 39 +++++-------------- .../channels/telegram/test_list_helpers.py | 29 +------------- 2 files changed, 11 insertions(+), 57 deletions(-) diff --git a/channels/telegram/_runner.py b/channels/telegram/_runner.py index 7266cac4..40361448 100644 --- a/channels/telegram/_runner.py +++ b/channels/telegram/_runner.py @@ -426,19 +426,18 @@ async def _handle_help(self, chat_id: int) -> None: await self._send_text(chat_id, "\n".join(lines)) async def _handle_list(self, chat_id: int, query: str) -> None: - """Show past conversations for this chat as an inline keyboard. + """Show past conversations as an inline keyboard. - Filters to conversations belonging to *chat_id* (default id + - ``/new``-spawned uuids). Optional ``query`` is a case-insensitive - substring match on title and first message — handy for "find that - conversation about taxes" without scrolling. + Lists every conversation in the store (Telegram, web, anywhere) — + the conversation store is the single source of truth and the user + shouldn't have to remember where they started. Optional ``query`` + is a case-insensitive substring match on title and first message. """ query = query.strip() summaries = list_conversations() matches = [ s for s in summaries - if _belongs_to_chat(s.conversation_id, chat_id) - and (not query or _matches_query(s, query)) + if not query or _matches_query(s, query) ][:_LIST_MAX_RESULTS] if not matches: @@ -478,17 +477,11 @@ async def _resume_conversation( ) -> None: """Bind this chat to *conv_id* so the next message continues it. - Validates that the picked conversation actually belongs to this - chat — a malicious callback payload can't trick the channel into - binding to an unrelated user's conversation. + Callback data is whatever the bot put on the button — only valid + conversation ids are sent — so no per-chat prefix validation is + needed here. The disk check below catches a stale callback whose + conversation was deleted between list-and-tap. """ - if not _belongs_to_chat(conv_id, chat_id): - logger.warning( - "telegram resume rejected — conv_id=%s not for chat_id=%s", - conv_id, chat_id, - ) - await self._answer_callback(callback_id, text="Not your conversation") - return if load_conversation_history(conv_id) is None: await self._answer_callback(callback_id, text="Conversation not found") return @@ -738,18 +731,6 @@ async def _answer_callback( logger.debug("telegram answer_callback_query failed: %s", exc) -def _belongs_to_chat(conv_id: str, chat_id: int) -> bool: - """True iff ``conv_id`` was created by this Telegram chat. - - Default conv id is ``telegram_``; ``/new`` appends an - underscore + uuid. Strict prefix match on either form avoids the - false-positive a naive startswith would hit when one chat_id is a - prefix of another (e.g. ``8617`` vs ``8617268723``). - """ - prefix = f"telegram_{chat_id}" - return conv_id == prefix or conv_id.startswith(prefix + "_") - - def _matches_query(summary: ConversationSummary, query: str) -> bool: """Case-insensitive substring match on title and first message.""" needle = query.lower() diff --git a/tests/unit/channels/telegram/test_list_helpers.py b/tests/unit/channels/telegram/test_list_helpers.py index cb0ec99b..8c2752d6 100644 --- a/tests/unit/channels/telegram/test_list_helpers.py +++ b/tests/unit/channels/telegram/test_list_helpers.py @@ -4,7 +4,7 @@ import pytest -from channels.telegram._runner import _belongs_to_chat, _matches_query, _row_label +from channels.telegram._runner import _matches_query, _row_label from conversations._models import ConversationSummary @@ -23,33 +23,6 @@ def _summary( ) -@pytest.mark.unit -class TestBelongsToChat: - - def test_default_id_matches(self): - assert _belongs_to_chat("telegram_42", 42) is True - - def test_uuid_suffix_matches(self): - assert _belongs_to_chat("telegram_42_a1b2c3d4", 42) is True - - def test_partial_chat_id_does_not_match(self): - # ``telegram_8617`` must NOT match chat_id 8617268723 — naive - # startswith would. - assert _belongs_to_chat("telegram_8617", 8617268723) is False - - def test_longer_chat_id_does_not_match(self): - # The other direction: ``telegram_86172687230`` (extra digit) - # must not match chat_id 8617268723. - assert _belongs_to_chat("telegram_86172687230", 8617268723) is False - - def test_unrelated_id_does_not_match(self): - assert _belongs_to_chat("chat_abc", 42) is False - assert _belongs_to_chat("telegram_99", 42) is False - - def test_empty_string_does_not_match(self): - assert _belongs_to_chat("", 42) is False - - @pytest.mark.unit class TestMatchesQuery: From c7d4fceeaf5826394eecbf983fca3f549498e21b Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Tue, 26 May 2026 20:49:42 -0500 Subject: [PATCH 09/12] refactor: extract ConversationCache used by both the web handler and Telegram channel MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both call sites kept their own in-memory map of Conversation objects, but the implementations had drifted: the SSE handler had a proper LRU with a 25-entry cap and active-turn-skip eviction, while the Telegram channel kept a plain unbounded dict that grew for the lifetime of the process. This was a latent memory leak — every chat the bot interacted with stayed cached forever — and an open invitation for a third channel to copy whichever shape it found first. New ``conversations.ConversationCache``: - Owns the LRU OrderedDict, the max-size cap, disk-hydrate on miss, and the active-turn-skip eviction logic. - ``is_active`` and ``on_evict`` are injected callbacks so the cache doesn't know about ``sdk.turn.is_turn_active`` or the SSE handler's per-conversation Playwright release directly. - API: ``get(id) -> (Conversation, is_new)``, ``resume(id) -> Conversation | None`` (force-reload from disk), ``pop(id)``, ``clear()``, plus ``__contains__`` and ``__iter__`` for test-friendly access. Wiring: - ``server/message_handler.py`` constructs a module-level cache with ``is_active=is_turn_active`` and an ``on_evict`` that releases the evicted conversation's browser context — preserving the eviction-time Playwright cleanup the inline implementation had. - ``channels/telegram/_runner.py`` constructs its own instance with ``is_active=is_turn_active`` (no ``on_evict``, no per-conv browser). This is where the bounded-cache behavior is genuinely new. Module structure: - Pulled the ``Conversation`` dataclass out of ``sdk/turn/_executor.py`` into ``sdk/turn/_conversation.py`` so importers below the SDK (the new cache) can grab the dataclass without pulling in ``TurnExecutor`` and its transitive deps — that triggered a circular import. Tests: - Moved all cache-behavior tests out of ``test_message_handler.py`` into ``tests/unit/conversations/test_cache.py`` and rewrote them against the cache class directly. Net +1 test (added an explicit ``on_evict`` coverage case). - ``test_message_handler.py`` shrinks to a smoke test for ``resume_conversation``'s dict-return shape. Co-Authored-By: Claude Opus 4.7 (1M context) --- channels/telegram/_runner.py | 42 +--- conversations/__init__.py | 2 + conversations/_cache.py | 167 ++++++++++++++++ sdk/__init__.py | 12 +- sdk/turn/_conversation.py | 29 +++ sdk/turn/_executor.py | 16 +- server/message_handler.py | 100 ++-------- tests/unit/conversations/__init__.py | 0 tests/unit/conversations/test_cache.py | 232 ++++++++++++++++++++++ tests/unit/server/test_message_handler.py | 202 ++++--------------- 10 files changed, 497 insertions(+), 305 deletions(-) create mode 100644 conversations/_cache.py create mode 100644 sdk/turn/_conversation.py create mode 100644 tests/unit/conversations/__init__.py create mode 100644 tests/unit/conversations/test_cache.py diff --git a/channels/telegram/_runner.py b/channels/telegram/_runner.py index 40361448..3354142d 100644 --- a/channels/telegram/_runner.py +++ b/channels/telegram/_runner.py @@ -21,6 +21,7 @@ from channels.telegram._profile_map import ProfileMap from channels.telegram._state import ConversationMap from conversations import ( + ConversationCache, DiskTurnPersistence, list_conversations, load_conversation_history, @@ -33,8 +34,7 @@ IntegrationNotConnected, call as broker_call, ) -from sdk import Conversation, TurnExecutor -from sdk.context import ConversationHistory +from sdk import TurnExecutor from sdk.turn import is_turn_active, request_stop from tools.memory import forget, memory_prompt_block, remember from tools.virtual_computer.run_bash_cmd import run_bash_cmd @@ -84,7 +84,9 @@ def __init__(self, *, app_sock_path: Path) -> None: self._formatter = TelegramFormatter() self._turn_executor = TurnExecutor() self._persistence = DiskTurnPersistence() - self._conversations: dict[str, Conversation] = {} + # Same bounded LRU the web/SSE side uses; on_evict is None because + # the channel doesn't own per-conversation Playwright contexts. + self._cache = ConversationCache(is_active=is_turn_active) self._integration_id: str = "" self._default_profile_id: str = "computron" self._pull_task: asyncio.Task[None] | None = None @@ -371,7 +373,7 @@ async def _dispatch_callback(self, update: dict[str, Any]) -> None: async def _handle_new(self, chat_id: int) -> None: conv_id = self._state.reset(chat_id) - self._conversations.pop(conv_id, None) + self._cache.pop(conv_id) await self._send_text(chat_id, f"🔄 New conversation started ({conv_id})") logger.info("telegram /new chat_id=%s conv_id=%s", chat_id, conv_id) @@ -489,7 +491,7 @@ async def _resume_conversation( self._state.set(chat_id, conv_id) # Drop the cached in-memory Conversation so the next turn rehydrates # the picked one's history from disk. - self._conversations.pop(conv_id, None) + self._cache.pop(conv_id) await self._answer_callback(callback_id, text="Resumed") await self._send_text(chat_id, f"📂 Resumed conversation: {conv_id}") logger.info( @@ -519,7 +521,7 @@ async def _select_profile( async def _run_turn(self, chat_id: int, text: str) -> None: """Execute one agent turn for the given chat and deliver the reply.""" conversation_id = self._state.get(chat_id) - conversation, is_new = self._get_conversation(conversation_id) + conversation, is_new = await self._cache.get(conversation_id) profile = self._resolve_profile(chat_id) if profile is None: @@ -640,34 +642,6 @@ async def _keep_typing(self, chat_id: int) -> None: except asyncio.CancelledError: raise - # -- conversation cache --------------------------------------------- - - def _get_conversation(self, conversation_id: str) -> tuple[Conversation, bool]: - """Return the conversation for *conversation_id*, creating if needed. - - Returns: - ``(conversation, is_new)`` where ``is_new`` is True only when no - on-disk history existed (a genuine first-time use). - """ - if conversation_id in self._conversations: - return self._conversations[conversation_id], False - - messages = load_conversation_history(conversation_id) - if messages is not None: - conversation = Conversation( - id=conversation_id, - history=ConversationHistory(messages, instance_id=conversation_id), - ) - self._conversations[conversation_id] = conversation - return conversation, False - - conversation = Conversation( - id=conversation_id, - history=ConversationHistory(instance_id=conversation_id), - ) - self._conversations[conversation_id] = conversation - return conversation, True - # -- outbound ------------------------------------------------------- async def _send_text(self, chat_id: int, text: str) -> None: diff --git a/conversations/__init__.py b/conversations/__init__.py index aec78dcf..f41c36f7 100644 --- a/conversations/__init__.py +++ b/conversations/__init__.py @@ -22,12 +22,14 @@ save_sub_agent_history, save_summary_record, ) +from ._cache import ConversationCache from ._title_generation import ( generate_conversation_title, ) from ._turn_persistence import DiskTurnPersistence __all__ = [ + "ConversationCache", "ConversationSummary", "DiskTurnPersistence", "SummaryRecord", diff --git a/conversations/_cache.py b/conversations/_cache.py new file mode 100644 index 00000000..ff1d384a --- /dev/null +++ b/conversations/_cache.py @@ -0,0 +1,167 @@ +"""Bounded LRU of in-memory ``Conversation`` objects. + +Each channel/handler owns one. The on-disk conversation store is +authoritative; the cache exists so consecutive turns within the same +conversation don't re-read ``history.json`` every time, and so the +``PersistenceHook``'s in-place mutations of history land in something +the next turn reuses. + +Eviction is LRU but skips conversations whose turn is currently in +flight — dropping an in-flight ``Conversation`` would leave the running +turn writing to a referent the next caller can't find, producing two +parallel writers when the next chat hydrates a fresh copy from disk. +""" + +from __future__ import annotations + +import logging +from collections import OrderedDict +from collections.abc import Awaitable, Callable + +from sdk.context._history import ConversationHistory +from sdk.turn._conversation import Conversation + +from ._store import load_conversation_history + +logger = logging.getLogger(__name__) + +__all__ = ["ConversationCache"] + + +# Default size — modest, sized for "active conversations in a session" +# rather than "every conversation ever." Disk is authoritative on miss. +_DEFAULT_MAX_SIZE = 25 + + +class ConversationCache: + """Bounded LRU of ``Conversation`` objects with disk-hydrate on miss. + + Args: + max_size: Maximum entries before eviction kicks in. + is_active: Predicate that returns True if the named conversation + has a turn currently running. The eviction loop skips entries + for which this is True. Defaults to "never active" — safe but + allows in-flight turns to be evicted, so callers that run real + turns should always pass ``sdk.turn.is_turn_active``. + on_evict: Optional async callback invoked AFTER each eviction with + the evicted conversation id. Used for channel-specific cleanup + (e.g. releasing per-conversation browser contexts). + """ + + def __init__( + self, + *, + max_size: int = _DEFAULT_MAX_SIZE, + is_active: Callable[[str], bool] | None = None, + on_evict: Callable[[str], Awaitable[None]] | None = None, + ) -> None: + self._max_size = max_size + self._is_active = is_active or (lambda _id: False) + self._on_evict = on_evict + self._entries: OrderedDict[str, Conversation] = OrderedDict() + + async def get(self, conversation_id: str) -> tuple[Conversation, bool]: + """Return ``(conversation, is_new)`` for the given id. + + Cache hit: moves to most-recently-used, returns ``is_new=False``. + Cache miss: hydrates from disk (or creates empty if no history), + inserts, then triggers eviction of the oldest non-active entries + if we're over the cap. ``is_new=True`` only when **no on-disk + history existed** — a genuine first-time use. + """ + if not conversation_id: + msg = "conversation_id is required" + raise ValueError(msg) + if conversation_id in self._entries: + self._entries.move_to_end(conversation_id) + return self._entries[conversation_id], False + persisted = load_conversation_history(conversation_id) + is_new = persisted is None + if is_new: + logger.info("Creating new conversation %s", conversation_id) + self._entries[conversation_id] = Conversation( + id=conversation_id, + history=ConversationHistory(persisted, instance_id=conversation_id), + ) + await self._evict_others(exclude=conversation_id) + return self._entries[conversation_id], is_new + + async def resume(self, conversation_id: str) -> Conversation | None: + """Force-reload an existing conversation from disk. + + Returns ``None`` if no on-disk history is found. Otherwise replaces + any cached entry with a fresh one hydrated from disk, marks it + most-recently-used, and evicts overflow. + + Use this when the caller knows the on-disk state has changed (or + wants to read it fresh) and the cached copy must not be trusted. + """ + messages = load_conversation_history(conversation_id) + if messages is None: + return None + self._entries[conversation_id] = Conversation( + id=conversation_id, + history=ConversationHistory(messages, instance_id=conversation_id), + ) + self._entries.move_to_end(conversation_id) + await self._evict_others(exclude=conversation_id) + return self._entries[conversation_id] + + def pop(self, conversation_id: str) -> None: + """Drop the cached entry if present. + + Used after explicit resets (``/new`` in Telegram, picking a + different conversation) so the next ``get`` rehydrates fresh from + disk instead of returning a stale in-memory copy. + """ + self._entries.pop(conversation_id, None) + + def clear(self) -> None: + """Drop every entry. Used in tests for clean isolation.""" + self._entries.clear() + + def __contains__(self, conversation_id: str) -> bool: + return conversation_id in self._entries + + def __len__(self) -> int: + return len(self._entries) + + def __iter__(self): + return iter(self._entries) + + async def _evict_others(self, *, exclude: str) -> None: + """Drop the oldest non-active entries until we're at or below cap. + + Conversations whose turn is currently in flight are skipped — + popping them would leave the running turn writing to a referent + nobody else can find, and a subsequent chat for the same id would + rehydrate from disk, producing two parallel writers. + + ``exclude`` skips the conversation that triggered this eviction. + The caller hasn't yet entered turn_scope for it, so ``is_active`` + cannot recognize it as protected — without this guard the + just-inserted entry would be evicted by its own insert in the + rare case where every other cached entry is mid-turn. + """ + while len(self._entries) > self._max_size: + for cid in self._entries: + if cid == exclude: + continue + if not self._is_active(cid): + self._entries.pop(cid) + if self._on_evict is not None: + try: + await self._on_evict(cid) + except Exception: # pragma: no cover - defensive + logger.exception( + "ConversationCache on_evict failed for %s", cid, + ) + logger.info( + "Evicted LRU conversation %s from in-memory cache", cid, + ) + break + else: + # Every cached conversation is mid-turn (or is the just- + # inserted caller) — accept temporary overflow rather than + # evict an active one. The next insert will retry. + return diff --git a/sdk/__init__.py b/sdk/__init__.py index 63f064a4..f2996412 100644 --- a/sdk/__init__.py +++ b/sdk/__init__.py @@ -18,13 +18,11 @@ # Imported last so that sdk.context, sdk.hooks, sdk.turn (without _executor) # and sdk.providers are all fully loaded before the executor module — which -# pulls from all of them — is initialized. -from .turn._executor import ( - Conversation, - SystemPromptBuilder, - TurnExecutor, - TurnPersistence, -) +# pulls from all of them — is initialized. ``Conversation`` lives in its +# own minimal module so importers below the SDK layer can grab it without +# pulling in the full executor. +from .turn._conversation import Conversation +from .turn._executor import SystemPromptBuilder, TurnExecutor, TurnPersistence __all__ = [ "BudgetGuard", diff --git a/sdk/turn/_conversation.py b/sdk/turn/_conversation.py new file mode 100644 index 00000000..26de5835 --- /dev/null +++ b/sdk/turn/_conversation.py @@ -0,0 +1,29 @@ +"""Per-conversation state owned by callers (channels, handlers, tools). + +Kept in its own module so importers that only need the dataclass don't +pay the cost of pulling in ``TurnExecutor`` + its transitive deps — +that matters for modules below the SDK in the dependency graph (e.g. +the ``conversations`` package's cache) where importing the executor +would form a cycle. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +from sdk.context._history import ConversationHistory + +__all__ = ["Conversation"] + + +@dataclass +class Conversation: + """Per-conversation state owned by the caller. + + Attributes: + id: Unique conversation identifier. + history: The conversation history. + """ + + id: str + history: ConversationHistory diff --git a/sdk/turn/_executor.py b/sdk/turn/_executor.py index 6a76655e..02d73b9c 100644 --- a/sdk/turn/_executor.py +++ b/sdk/turn/_executor.py @@ -13,11 +13,9 @@ import logging from collections.abc import AsyncGenerator, Callable, Iterable, Sequence from contextlib import suppress -from dataclasses import dataclass from typing import Protocol from agents.types import Agent -from sdk.context._history import ConversationHistory from sdk.context._manager import ContextManager from sdk.context._strategy import LLMCompactionStrategy from sdk.events._context import agent_span, get_current_dispatcher @@ -27,6 +25,7 @@ from sdk.hooks._persistence import PersistenceHook from sdk.skills import AgentState, get_skill from sdk.tools._core import get_core_tools +from sdk.turn._conversation import Conversation from sdk.turn._execution import run_turn from sdk.turn._turn import StopRequestedError, turn_scope @@ -36,19 +35,6 @@ _background_tasks: set[asyncio.Task] = set() -@dataclass -class Conversation: - """Per-conversation state owned by the caller. - - Attributes: - id: Unique conversation identifier. - history: The conversation history. - """ - - id: str - history: ConversationHistory - - class TurnPersistence(Protocol): """Optional persistence hooks invoked by ``TurnExecutor``. diff --git a/server/message_handler.py b/server/message_handler.py index 39983bac..090ff4fb 100644 --- a/server/message_handler.py +++ b/server/message_handler.py @@ -1,7 +1,6 @@ """Message handler for user prompts.""" import logging -from collections import OrderedDict from collections.abc import AsyncGenerator, Sequence from rich.console import Console @@ -15,13 +14,12 @@ ) from agents.types import Data from conversations import ( + ConversationCache, DiskTurnPersistence, load_agent_events, - load_conversation_history, load_preview_state, ) -from sdk import Conversation, TurnExecutor -from sdk.context import ConversationHistory +from sdk import TurnExecutor from sdk.events import ( AgentEvent, ContentPayload, @@ -87,82 +85,21 @@ def _log_turn_start(profile: AgentProfile) -> None: ) -# In-memory conversation cache. LRU-bounded so a long-lived process -# doesn't hold every conversation a user has ever opened. The on-disk -# state is authoritative; an evicted entry is rehydrated from disk on -# next access. -_MAX_CACHED_CONVERSATIONS = 25 -_conversations: OrderedDict[str, Conversation] = OrderedDict() +# Shared in-memory conversation cache. The on-disk store is authoritative; +# the cache exists so consecutive turns reuse the same Conversation object +# (and the PersistenceHook's in-place history mutations survive across +# turns). Eviction releases per-conversation browser contexts so evicted +# rows don't leak a Playwright instance. +_cache = ConversationCache( + is_active=is_turn_active, + on_evict=lambda cid: release_agent_browser(f"conv:{cid}"), +) # Shared turn executor — stateless, safe to reuse across conversations. _turn_executor = TurnExecutor() _persistence = DiskTurnPersistence() -async def _get_conversation(conversation_id: str) -> tuple[Conversation, bool]: - """Return ``(conversation, is_new)`` for the given ID, creating it if needed. - - ``is_new`` is True only when the conversation has no in-memory entry - AND no on-disk history — a genuine first-time use. On any cache miss - we hydrate from disk so turns survive process restarts: the browser - preserves a conversation id across server bounces (e.g. ``just - restart-app``), and without hydration the next turn would build on an - empty history and the persistence hook would overwrite the saved file. - - Cache hits move the entry to the end of the LRU; cache misses insert - at the end and may evict the least-recently-used entry whose turn is - not currently active. - """ - if not conversation_id: - msg = "conversation_id is required" - raise ValueError(msg) - if conversation_id in _conversations: - _conversations.move_to_end(conversation_id) - return _conversations[conversation_id], False - persisted = load_conversation_history(conversation_id) - is_new = persisted is None - if is_new: - logger.info("Creating new conversation %s", conversation_id) - _conversations[conversation_id] = Conversation( - id=conversation_id, - history=ConversationHistory(persisted, instance_id=conversation_id), - ) - await _evict_lru_conversation(exclude=conversation_id) - return _conversations[conversation_id], is_new - - -async def _evict_lru_conversation(exclude: str | None = None) -> None: - """Drop the oldest non-active entries until we are at or below the cap. - - Conversations whose turn is currently in flight are skipped — popping - them from the dict would leave the running turn writing to a referent - nobody else can find, and a subsequent chat for the same id would - rehydrate from disk, producing two parallel writers. - - ``exclude`` skips the conversation that triggered this eviction. The - caller has not yet entered ``turn_scope`` for it, so ``is_turn_active`` - cannot recognize it as protected — without this guard the just-inserted - entry would be evicted by its own insert in the rare case where every - other cached entry is mid-turn. - """ - while len(_conversations) > _MAX_CACHED_CONVERSATIONS: - for cid in _conversations: - if cid == exclude: - continue - if not is_turn_active(cid): - _conversations.pop(cid) - await release_agent_browser(f"conv:{cid}") - logger.info( - "Evicted LRU conversation %s from in-memory cache", cid, - ) - break - else: - # Every cached conversation is mid-turn (or is the just-inserted - # caller) — accept temporary overflow rather than evict an - # active one. The next insert will retry. - return - - async def resume_conversation(conversation_id: str) -> dict | None: """Load a conversation's full-fidelity history, events, and preview state. @@ -176,18 +113,11 @@ async def resume_conversation(conversation_id: str) -> dict | None: None if the conversation isn't found. """ - messages = load_conversation_history(conversation_id) - if messages is None: + conversation = await _cache.resume(conversation_id) + if conversation is None: return None - - _conversations[conversation_id] = Conversation( - id=conversation_id, - history=ConversationHistory(messages, instance_id=conversation_id), - ) - _conversations.move_to_end(conversation_id) - await _evict_lru_conversation(exclude=conversation_id) return { - "messages": messages, + "messages": list(conversation.history.messages), "events": load_agent_events(conversation_id), "preview_state": load_preview_state(conversation_id), } @@ -230,7 +160,7 @@ async def handle_user_message( if not conversation_id: msg = "conversation_id is required" raise ValueError(msg) - conversation, is_new_conversation = await _get_conversation(conversation_id) + conversation, is_new_conversation = await _cache.get(conversation_id) user_content = message if data: diff --git a/tests/unit/conversations/__init__.py b/tests/unit/conversations/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/unit/conversations/test_cache.py b/tests/unit/conversations/test_cache.py new file mode 100644 index 00000000..08a7ea42 --- /dev/null +++ b/tests/unit/conversations/test_cache.py @@ -0,0 +1,232 @@ +"""Unit tests for ``conversations.ConversationCache``.""" + +from __future__ import annotations + +from collections.abc import AsyncIterator +from pathlib import Path + +import pytest + +from conversations import ConversationCache +from conversations._store import save_conversation_history +from sdk import Conversation +from sdk.context import ConversationHistory + + +@pytest.fixture +async def cache() -> AsyncIterator[ConversationCache]: + """A fresh cache, no active-turn awareness, no on-evict side effect.""" + c = ConversationCache() + yield c + c.clear() + + +# --------------------------------------------------------------------------- +# get() — hydration semantics +# --------------------------------------------------------------------------- + + +async def test_get_cold_no_disk_creates_empty_and_marks_new(cache: ConversationCache) -> None: + conv, is_new = await cache.get("brand-new-id") + assert len(conv.history) == 0 + assert conv.history.instance_id == "brand-new-id" + assert is_new is True + + +async def test_get_cold_with_disk_hydrates_and_marks_not_new(cache: ConversationCache) -> None: + save_conversation_history("existing", [ + {"role": "user", "content": "hello"}, + {"role": "assistant", "content": "hi"}, + ]) + conv, is_new = await cache.get("existing") + assert len(conv.history) == 2 + loaded = conv.history.messages + assert loaded[0]["content"] == "hello" + assert loaded[1]["content"] == "hi" + assert is_new is False + + +async def test_warm_cache_wins_over_disk(cache: ConversationCache) -> None: + """An in-memory entry wins over whatever is on disk and is_new=False.""" + cached = Conversation( + id="cid", + history=ConversationHistory( + [{"role": "user", "content": "from-memory"}], + instance_id="cid", + ), + ) + cache._entries["cid"] = cached + save_conversation_history("cid", [{"role": "user", "content": "from-disk"}]) + + conv, is_new = await cache.get("cid") + assert conv is cached + assert conv.history.messages[0]["content"] == "from-memory" + assert is_new is False + + +async def test_subsequent_call_returns_same_instance(cache: ConversationCache) -> None: + first, first_new = await cache.get("same-id") + second, second_new = await cache.get("same-id") + assert first is second + assert first_new is True + assert second_new is False + + +async def test_empty_id_raises(cache: ConversationCache) -> None: + with pytest.raises(ValueError, match="conversation_id is required"): + await cache.get("") + + +async def test_corrupted_history_falls_back_to_empty( + cache: ConversationCache, tmp_path: Path, +) -> None: + from conversations._store import _get_conversations_dir + + cid = "corrupted" + conv_dir = _get_conversations_dir() / cid + conv_dir.mkdir(parents=True) + (conv_dir / "history.json").write_text("{not valid json", encoding="utf-8") + + conv, is_new = await cache.get(cid) + assert len(conv.history) == 0 + assert is_new is True + + +# --------------------------------------------------------------------------- +# LRU + eviction +# --------------------------------------------------------------------------- + + +async def test_lru_evicts_oldest_when_cap_exceeded() -> None: + cache = ConversationCache(max_size=3) + for cid in ("a", "b", "c"): + await cache.get(cid) + assert list(cache) == ["a", "b", "c"] + + await cache.get("d") + assert "a" not in cache + assert list(cache) == ["b", "c", "d"] + + +async def test_lru_access_promotes_to_most_recently_used() -> None: + cache = ConversationCache(max_size=3) + for cid in ("a", "b", "c"): + await cache.get(cid) + + # Touch 'a' — should become most-recently-used. + await cache.get("a") + assert list(cache) == ["b", "c", "a"] + + # Inserting a fourth should now evict 'b', not 'a'. + await cache.get("d") + assert "b" not in cache + assert "a" in cache + + +async def test_lru_skips_active_turn() -> None: + active = {"a"} + cache = ConversationCache(max_size=2, is_active=lambda cid: cid in active) + + await cache.get("a") + await cache.get("b") + assert list(cache) == ["a", "b"] + + # Inserting 'c' would normally evict 'a' (oldest). Active-skip jumps + # over 'a' and evicts 'b' instead. + await cache.get("c") + assert "a" in cache + assert "b" not in cache + assert "c" in cache + + +async def test_lru_overflow_when_all_active() -> None: + cache = ConversationCache(max_size=2, is_active=lambda _cid: True) + await cache.get("a") + await cache.get("b") + await cache.get("c") + assert len(cache) == 3 + assert set(cache) == {"a", "b", "c"} + + +async def test_lru_does_not_evict_just_inserted_when_others_active() -> None: + """Just-inserted survives even when every existing entry is mid-turn.""" + pinned = {"a", "b"} + cache = ConversationCache(max_size=2, is_active=lambda cid: cid in pinned) + await cache.get("a") + await cache.get("b") + await cache.get("c") + assert "c" in cache + assert set(cache) == {"a", "b", "c"} + + +# --------------------------------------------------------------------------- +# on_evict callback +# --------------------------------------------------------------------------- + + +async def test_on_evict_fires_with_evicted_id() -> None: + evicted: list[str] = [] + + async def _on_evict(cid: str) -> None: + evicted.append(cid) + + cache = ConversationCache(max_size=2, on_evict=_on_evict) + await cache.get("a") + await cache.get("b") + await cache.get("c") + assert evicted == ["a"] + + +# --------------------------------------------------------------------------- +# resume() — force-reload +# --------------------------------------------------------------------------- + + +async def test_resume_returns_none_for_missing(cache: ConversationCache) -> None: + assert await cache.resume("nope") is None + + +async def test_resume_hydrates_and_marks_most_recently_used() -> None: + cache = ConversationCache(max_size=3) + await cache.get("a") + save_conversation_history( + "from-disk", [{"role": "user", "content": "hi"}], + ) + conv = await cache.resume("from-disk") + assert conv is not None + # 'from-disk' is now most-recently-used. + assert list(cache)[-1] == "from-disk" + + +async def test_resume_overwrites_cached_entry(cache: ConversationCache) -> None: + """resume reloads from disk even if a cached entry exists for the id.""" + save_conversation_history( + "twice", [{"role": "user", "content": "first version"}], + ) + first = await cache.resume("twice") + assert first is not None + assert first.history.messages[0]["content"] == "first version" + + save_conversation_history( + "twice", [{"role": "user", "content": "second version"}], + ) + second = await cache.resume("twice") + assert second is not None + assert second is not first + assert second.history.messages[0]["content"] == "second version" + + +# --------------------------------------------------------------------------- +# pop() +# --------------------------------------------------------------------------- + + +async def test_pop_drops_entry(cache: ConversationCache) -> None: + await cache.get("a") + assert "a" in cache + cache.pop("a") + assert "a" not in cache + + +def test_pop_unknown_id_is_a_no_op(cache: ConversationCache) -> None: + cache.pop("never-existed") # should not raise diff --git a/tests/unit/server/test_message_handler.py b/tests/unit/server/test_message_handler.py index 5dcc21c1..03711d04 100644 --- a/tests/unit/server/test_message_handler.py +++ b/tests/unit/server/test_message_handler.py @@ -1,25 +1,32 @@ -"""Unit tests for ``server.message_handler`` cache + persistence behavior.""" +"""Unit tests for ``server.message_handler`` — the SSE handler glue. + +Cache behavior is covered in ``tests/unit/conversations/test_cache.py``. +What's here is the higher-level wiring this module is actually responsible +for — composing ``resume_conversation``'s payload from history + events + +preview state. +""" from __future__ import annotations from collections.abc import AsyncIterator -from pathlib import Path from unittest.mock import AsyncMock, patch import pytest -from conversations._store import save_conversation_history -from sdk import Conversation -from sdk.context import ConversationHistory +from conversations._store import ( + save_agent_events, + save_conversation_history, + save_preview_state, +) from server import message_handler as mh @pytest.fixture(autouse=True) -async def _clear_in_memory_conversations() -> AsyncIterator[None]: - """Reset the module-global conversation cache between tests.""" - mh._conversations.clear() +async def _clear_cache() -> AsyncIterator[None]: + """Reset the module-global cache between tests.""" + mh._cache.clear() yield - mh._conversations.clear() + mh._cache.clear() @pytest.fixture(autouse=True) @@ -29,165 +36,32 @@ def _stub_browser_release(): yield -async def test_get_conversation_cold_cache_no_disk_creates_empty_and_marks_new() -> None: - """No in-memory entry, no on-disk history -> empty + is_new=True.""" - conv, is_new = await mh._get_conversation("brand-new-id") - assert len(conv.history) == 0 - assert conv.history.instance_id == "brand-new-id" - assert is_new is True - - -async def test_get_conversation_cold_cache_with_disk_hydrates_and_marks_not_new() -> None: - """No in-memory entry, on-disk history present -> hydrated + is_new=False.""" - save_conversation_history("existing", [ - {"role": "user", "content": "hello"}, - {"role": "assistant", "content": "hi"}, - ]) - - conv, is_new = await mh._get_conversation("existing") - - assert len(conv.history) == 2 - loaded = conv.history.messages - assert loaded[0]["content"] == "hello" - assert loaded[1]["content"] == "hi" - assert is_new is False - - -async def test_get_conversation_warm_cache_does_not_reread_disk() -> None: - """An in-memory entry wins over whatever is on disk and is_new=False.""" - cached = Conversation( - id="cid", - history=ConversationHistory( - [{"role": "user", "content": "from-memory"}], - instance_id="cid", - ), - ) - mh._conversations["cid"] = cached - save_conversation_history("cid", [{"role": "user", "content": "from-disk"}]) - - conv, is_new = await mh._get_conversation("cid") - - assert conv is cached - assert conv.history.messages[0]["content"] == "from-memory" - assert is_new is False - - -async def test_get_conversation_subsequent_call_returns_same_instance() -> None: - """Two calls for the same id return the same Conversation object.""" - first, first_new = await mh._get_conversation("same-id") - second, second_new = await mh._get_conversation("same-id") - assert first is second - assert first_new is True - assert second_new is False - - -async def test_get_conversation_empty_id_raises() -> None: - """Empty string is rejected.""" - with pytest.raises(ValueError, match="conversation_id is required"): - await mh._get_conversation("") - - -async def test_get_conversation_corrupted_history_falls_back_to_empty(tmp_path: Path) -> None: - """A malformed history.json is treated as no on-disk history.""" - from conversations._store import _get_conversations_dir - - cid = "corrupted" - conv_dir = _get_conversations_dir() / cid - conv_dir.mkdir(parents=True) - (conv_dir / "history.json").write_text("{not valid json", encoding="utf-8") - - conv, is_new = await mh._get_conversation(cid) - - assert len(conv.history) == 0 - assert is_new is True - - -async def test_lru_evicts_oldest_when_cap_exceeded(monkeypatch: pytest.MonkeyPatch) -> None: - """Inserting beyond the cap evicts the least-recently-used entry.""" - monkeypatch.setattr(mh, "_MAX_CACHED_CONVERSATIONS", 3) +async def test_resume_returns_none_for_missing() -> None: + assert await mh.resume_conversation("does-not-exist") is None - await mh._get_conversation("a") - await mh._get_conversation("b") - await mh._get_conversation("c") - assert list(mh._conversations) == ["a", "b", "c"] - await mh._get_conversation("d") +async def test_resume_returns_messages_events_and_preview_state() -> None: + """resume_conversation reads history, events, and preview state from disk.""" + cid = "resume-test" + save_conversation_history(cid, [{"role": "user", "content": "hi"}]) + save_agent_events(cid, [{"payload": {"type": "content", "content": "x"}}]) + save_preview_state(cid, {"open": ["a.txt"]}) - assert "a" not in mh._conversations - assert list(mh._conversations) == ["b", "c", "d"] - - -async def test_lru_access_promotes_to_most_recently_used( - monkeypatch: pytest.MonkeyPatch, -) -> None: - """A cache hit moves the entry to the end so it survives the next eviction.""" - monkeypatch.setattr(mh, "_MAX_CACHED_CONVERSATIONS", 3) - - await mh._get_conversation("a") - await mh._get_conversation("b") - await mh._get_conversation("c") - - # Touch 'a' — should become most-recently-used. - await mh._get_conversation("a") - assert list(mh._conversations) == ["b", "c", "a"] - - # Inserting a fourth should now evict 'b', not 'a'. - await mh._get_conversation("d") - assert "b" not in mh._conversations - assert "a" in mh._conversations - - -async def test_lru_skips_active_turn(monkeypatch: pytest.MonkeyPatch) -> None: - """Conversations whose turn is in flight are not evicted.""" - monkeypatch.setattr(mh, "_MAX_CACHED_CONVERSATIONS", 2) - monkeypatch.setattr(mh, "is_turn_active", lambda cid: cid == "a") - - await mh._get_conversation("a") - await mh._get_conversation("b") - assert list(mh._conversations) == ["a", "b"] - - # Inserting 'c' would normally evict 'a' (oldest). Pinning skips - # over 'a' and evicts 'b' instead. - await mh._get_conversation("c") - assert "a" in mh._conversations - assert "b" not in mh._conversations - assert "c" in mh._conversations - - -async def test_lru_overflow_when_all_active(monkeypatch: pytest.MonkeyPatch) -> None: - """When every cached conv is mid-turn, the cache temporarily overflows.""" - monkeypatch.setattr(mh, "_MAX_CACHED_CONVERSATIONS", 2) - monkeypatch.setattr(mh, "is_turn_active", lambda _cid: True) - - await mh._get_conversation("a") - await mh._get_conversation("b") - await mh._get_conversation("c") - - assert len(mh._conversations) == 3 - assert set(mh._conversations) == {"a", "b", "c"} - - -async def test_lru_does_not_evict_just_inserted_when_others_active( - monkeypatch: pytest.MonkeyPatch, -) -> None: - """Just-inserted conv survives even when every existing entry is mid-turn.""" - monkeypatch.setattr(mh, "_MAX_CACHED_CONVERSATIONS", 2) - monkeypatch.setattr(mh, "is_turn_active", lambda cid: cid in {"a", "b"}) - - await mh._get_conversation("a") - await mh._get_conversation("b") - await mh._get_conversation("c") - - assert "c" in mh._conversations - assert set(mh._conversations) == {"a", "b", "c"} + result = await mh.resume_conversation(cid) + assert result is not None + assert result["messages"] == [{"role": "user", "content": "hi"}] + # events list and preview_state dict shapes are determined by the store; + # asserting they're truthy is enough — the store has its own tests. + assert result["events"] + assert result["preview_state"] -async def test_resume_conversation_marks_most_recently_used() -> None: - """resume_conversation places the resumed entry at the LRU tail.""" - await mh._get_conversation("a") - save_conversation_history("from-disk", [{"role": "user", "content": "hi"}]) - result = await mh.resume_conversation("from-disk") +async def test_resume_installs_into_cache() -> None: + """After resume, the conversation is in the cache (so the next turn + reuses the same Conversation object).""" + cid = "resume-then-turn" + save_conversation_history(cid, [{"role": "user", "content": "hi"}]) - assert result is not None - assert list(mh._conversations)[-1] == "from-disk" + await mh.resume_conversation(cid) + assert cid in mh._cache From a93ab3b9edddf31fc29e47cfb593a2f9621eb8b9 Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Fri, 29 May 2026 20:37:14 -0500 Subject: [PATCH 10/12] address PR review: markdown rendering, channel rename, notifier settings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - channels/telegram/_formatter.py: replace dead `escape_markdown` with `to_markdownv2_chunks`; agent output runs through telegramify-markdown (convert → split_markdownv2) and is sent with parse_mode="MarkdownV2" so **bold**, code blocks, lists, etc. render natively instead of as literal asterisks. Broker `send_message` verb gains optional parse_mode. - channels/telegram/_runner.py → _channel.py rename, with import + aiohttp app-key updates (`telegram_bot_runner` → `telegram_channel`). - TELEGRAM_INTEGRATION_ID / TELEGRAM_CHAT_ID env vars become settings (`telegram_notifier_integration_id`, `telegram_notifier_chat_id`) editable in Settings → System → Notifications; migration 006 seeds empty defaults on existing installs. Co-Authored-By: Claude Opus 4.7 (1M context) --- .env.example | 9 - README.md | 8 +- channels/telegram/__init__.py | 2 +- channels/telegram/{_runner.py => _channel.py} | 16 +- channels/telegram/_formatter.py | 72 ++-- .../brokers/telegram_broker/_verbs.py | 9 +- migrations/_006_telegram_notifier_settings.py | 45 +++ migrations/_runner.py | 4 + pyproject.toml | 1 + server/_integrations_routes.py | 6 +- server/aiohttp_app.py | 14 +- server/ui/src/components/SystemSettings.jsx | 62 +++- settings.py | 6 + tasks/_notifier.py | 28 +- .../unit/channels/telegram/test_formatter.py | 188 +++++------ .../channels/telegram/test_list_helpers.py | 4 +- .../brokers/telegram_broker/test_verbs.py | 25 +- .../test_006_telegram_notifier_settings.py | 61 ++++ tests/unit/server/test_settings.py | 29 ++ tests/unit/tasks/test_notifier.py | 313 ++++++++++-------- uv.lock | 150 +++++++++ 21 files changed, 707 insertions(+), 345 deletions(-) rename channels/telegram/{_runner.py => _channel.py} (98%) create mode 100644 migrations/_006_telegram_notifier_settings.py create mode 100644 tests/unit/migrations/test_006_telegram_notifier_settings.py diff --git a/.env.example b/.env.example index c170ac6f..b2f3a16a 100644 --- a/.env.example +++ b/.env.example @@ -10,12 +10,3 @@ LLM_API_KEY= # HuggingFace token (for gated models like Flux.1-schnell) HF_TOKEN= - -# Telegram notifications for goal completion/failure. The bot token lives -# inside the telegram broker (configured per integration); the notifier only -# needs to know which integration to target and which chat to send to. -# To find your chat ID: send a message to your bot, then visit -# https://api.telegram.org/bot/getUpdates and look for chat.id. -TELEGRAM_INTEGRATION_ID= -TELEGRAM_CHAT_ID= - diff --git a/README.md b/README.md index 59b2d5c1..088dc158 100644 --- a/README.md +++ b/README.md @@ -369,16 +369,16 @@ Pass these with `-e` when running the container: | `ENABLE_MUSIC_GEN` | No | Set to `1` to enable music generation (requires GPU). | | `ENABLE_DESKTOP` | No | Set to `1` to enable the desktop agent (GUI automation via Xfce). | | `ENABLE_GROUNDING` | No | Set to `1` to enable visual grounding in browser/desktop (requires GPU). | -| `TELEGRAM_BOT_TOKEN` | No | Telegram bot token for goal run notifications. | -| `TELEGRAM_CHAT_ID` | No | Telegram chat ID to receive notifications. | + +Telegram bot setup (both bidirectional chat and goal-run push notifications) +is configured in-app: add a Telegram integration on the Integrations tab, +then wire push notifications to it from Settings → System → Notifications. To pass multiple env vars, add `-e` for each one: ```bash docker run -d --name computron --shm-size=256m --network=host \ -e HF_TOKEN=hf_your_token_here \ - -e TELEGRAM_BOT_TOKEN=your_bot_token \ - -e TELEGRAM_CHAT_ID=your_chat_id \ -v computron_home:/home/computron \ -v computron_state:/var/lib/computron \ ghcr.io/lefoulkrod/computron_9000:latest diff --git a/channels/telegram/__init__.py b/channels/telegram/__init__.py index d9b10c6e..59ba0092 100644 --- a/channels/telegram/__init__.py +++ b/channels/telegram/__init__.py @@ -3,7 +3,7 @@ """ from channels.telegram._formatter import TelegramFormatter -from channels.telegram._runner import TelegramChannel +from channels.telegram._channel import TelegramChannel from channels.telegram._state import ConversationMap __all__ = [ diff --git a/channels/telegram/_runner.py b/channels/telegram/_channel.py similarity index 98% rename from channels/telegram/_runner.py rename to channels/telegram/_channel.py index 3354142d..cf26051b 100644 --- a/channels/telegram/_runner.py +++ b/channels/telegram/_channel.py @@ -645,13 +645,23 @@ async def _keep_typing(self, chat_id: int) -> None: # -- outbound ------------------------------------------------------- async def _send_text(self, chat_id: int, text: str) -> None: - """Send *text* to *chat_id*, splitting if necessary.""" - for chunk in self._formatter.split(text): + """Send *text* to *chat_id*, splitting if necessary. + + Agent output is CommonMark-flavored markdown. We convert to + Telegram MarkdownV2 and tag the send with ``parse_mode`` so the + formatting renders natively instead of showing as literal + asterisks and backticks. + """ + for chunk in self._formatter.to_markdownv2_chunks(text): try: await broker_call( self._integration_id, "send_message", - {"chat_id": chat_id, "text": chunk}, + { + "chat_id": chat_id, + "text": chunk, + "parse_mode": "MarkdownV2", + }, app_sock_path=self._app_sock, ) except IntegrationError as exc: diff --git a/channels/telegram/_formatter.py b/channels/telegram/_formatter.py index bbd9f46b..e5f19871 100644 --- a/channels/telegram/_formatter.py +++ b/channels/telegram/_formatter.py @@ -2,63 +2,43 @@ from __future__ import annotations -import re from pathlib import Path +import telegramify_markdown + __all__ = ["TelegramFormatter"] -# Telegram message size limit. -_MSG_LIMIT = 4096 +# Telegram's per-message limit, in UTF-16 code units. +_TELEGRAM_MSG_LIMIT = 4096 class TelegramFormatter: - """Prepares agent text for Telegram: splitting, escaping, truncating.""" + """Prepares agent text for Telegram. - # -- public API ----------------------------------------------------- + The agent emits CommonMark-flavored markdown (``**bold**``, fenced + code, lists, links). Telegram's MarkdownV2 uses different syntax and + requires escaping of many punctuation characters. We delegate the + conversion and the chunk splitting to ``telegramify_markdown`` so the + result renders natively in the Telegram client instead of showing + literal asterisks and backticks. + """ @staticmethod - def split(text: str, *, limit: int = _MSG_LIMIT) -> list[str]: - """Split *text* into chunks that fit Telegram's message size limit. - - Prefers paragraph boundaries, then sentence boundaries, then - falls back to hard splitting at *limit*. - """ - if len(text) <= limit: - return [text] - - chunks: list[str] = [] - remaining = text - while remaining: - if len(remaining) <= limit: - chunks.append(remaining) - break + def to_markdownv2_chunks(text: str) -> list[str]: + """Convert ``text`` to MarkdownV2 and split to fit Telegram's limit. - # Try paragraph break - cut = remaining.rfind("\n\n", 0, limit) - if cut == -1: - # Try single newline - cut = remaining.rfind("\n", 0, limit) - if cut == -1: - # Try sentence boundary - cut = max( - remaining.rfind(". ", 0, limit), - remaining.rfind("! ", 0, limit), - remaining.rfind("? ", 0, limit), - ) - if cut == -1 or cut < limit // 4: - # Hard split - cut = limit + Each returned chunk must be sent with ``parse_mode="MarkdownV2"``. - chunks.append(remaining[:cut].rstrip()) - remaining = remaining[cut:].lstrip("\n") - - return chunks - - @staticmethod - def escape_markdown(text: str) -> str: - """Escape characters that are special in Telegram MarkdownV2.""" - special = r"_*[]()~`>#+-=|{}.!" - return re.sub(r"([%s])" % re.escape(special), r"\\\1", text) + ``convert`` produces a plain text + entities pair. + ``split_markdownv2`` budgets against the **rendered** MarkdownV2 + size so chunks that grow under escaping (``.`` → ``\\.``) still + fit Telegram's 4096-code-unit limit, and won't cut a formatting + span in half. + """ + plain_text, entities = telegramify_markdown.convert(text) + return telegramify_markdown.split_markdownv2( + plain_text, entities, _TELEGRAM_MSG_LIMIT, + ) @staticmethod def file_caption(path: Path, *, index: int = 0, total: int = 1) -> str: @@ -66,4 +46,4 @@ def file_caption(path: Path, *, index: int = 0, total: int = 1) -> str: name = path.name if total == 1: return f"📎 {name}" - return f"📎 {name} ({index + 1}/{total})" \ No newline at end of file + return f"📎 {name} ({index + 1}/{total})" diff --git a/integrations/brokers/telegram_broker/_verbs.py b/integrations/brokers/telegram_broker/_verbs.py index b0ceecd9..4a7e712c 100644 --- a/integrations/brokers/telegram_broker/_verbs.py +++ b/integrations/brokers/telegram_broker/_verbs.py @@ -126,17 +126,23 @@ async def _handle_next_updates(self, args: dict[str, Any]) -> dict[str, Any]: return {"updates": updates} async def _handle_send_message(self, args: dict[str, Any]) -> dict[str, Any]: - """``send_message {chat_id, text, reply_to_message_id?, buttons?}`` → ``{message_id}``. + """``send_message {chat_id, text, reply_to_message_id?, buttons?, parse_mode?}`` → ``{message_id}``. ``buttons`` is an optional 2-D array of ``{text, data}`` dicts that becomes an inline keyboard attached to the message. Each tap fires a callback_query update with ``data`` matching what was sent. + + ``parse_mode`` is an optional Telegram parse-mode string + (``"MarkdownV2"``, ``"HTML"``, ``"Markdown"``). Omit for plain text. """ chat_id = _require_int(args, "chat_id") text = _require_str(args, "text") reply_to = args.get("reply_to_message_id") if reply_to is not None and (isinstance(reply_to, bool) or not isinstance(reply_to, int)): raise RpcError("BAD_REQUEST", "'reply_to_message_id' must be an integer") + parse_mode = args.get("parse_mode") + if parse_mode is not None and not isinstance(parse_mode, str): + raise RpcError("BAD_REQUEST", "'parse_mode' must be a string") reply_markup = _coerce_inline_keyboard(args.get("buttons")) try: msg = await self._bot.send_message( @@ -144,6 +150,7 @@ async def _handle_send_message(self, args: dict[str, Any]) -> dict[str, Any]: text=text, reply_to_message_id=reply_to, reply_markup=reply_markup, + parse_mode=parse_mode, ) except TelegramAPIError as exc: raise RpcError("INTERNAL", str(exc)) from exc diff --git a/migrations/_006_telegram_notifier_settings.py b/migrations/_006_telegram_notifier_settings.py new file mode 100644 index 00000000..2d088cc9 --- /dev/null +++ b/migrations/_006_telegram_notifier_settings.py @@ -0,0 +1,45 @@ +"""Migration 006: Add Telegram notifier settings keys. + +The goal-run push notifier previously read ``TELEGRAM_INTEGRATION_ID`` and +``TELEGRAM_CHAT_ID`` from the environment. Both now live in +``settings.json`` so the user can edit them in the app's Settings → System +page. This migration seeds the keys with empty defaults on existing +installs; users who relied on the env vars will need to re-enter the +values in the UI. +""" + +from __future__ import annotations + +import json +import logging +from pathlib import Path + +logger = logging.getLogger(__name__) + +_SETTINGS_FILE = "settings.json" + + +def migrate(state_dir: Path) -> None: + """Add telegram_notifier_* keys to settings.json if absent.""" + path = state_dir / _SETTINGS_FILE + if not path.exists(): + # No settings file yet — defaults in settings.py will apply on first read. + return + + try: + data = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError: + logger.warning("Corrupt %s, skipping migration 006", path) + return + + changed = False + if "telegram_notifier_integration_id" not in data: + data["telegram_notifier_integration_id"] = "" + changed = True + if "telegram_notifier_chat_id" not in data: + data["telegram_notifier_chat_id"] = None + changed = True + + if changed: + path.write_text(json.dumps(data, indent=2), encoding="utf-8") + logger.info("Seeded telegram notifier settings in %s", path) diff --git a/migrations/_runner.py b/migrations/_runner.py index 574f6cab..6aeaf616 100644 --- a/migrations/_runner.py +++ b/migrations/_runner.py @@ -12,6 +12,9 @@ from migrations._003_vision_settings import migrate as _003_vision_settings from migrations._004_rename_num_ctx import migrate as _004_rename_num_ctx from migrations._005_multi_provider import migrate as _005_multi_provider +from migrations._006_telegram_notifier_settings import ( + migrate as _006_telegram_notifier_settings, +) logger = logging.getLogger(__name__) @@ -26,6 +29,7 @@ ("003_vision_settings", _003_vision_settings), ("004_rename_num_ctx", _004_rename_num_ctx), ("005_multi_provider", _005_multi_provider), + ("006_telegram_notifier_settings", _006_telegram_notifier_settings), ] diff --git a/pyproject.toml b/pyproject.toml index 311bfc82..97bd089a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -24,6 +24,7 @@ dependencies = [ "google-auth-oauthlib>=1.2", "google-api-python-client>=2.130", "aiogram>=3.7", + "telegramify-markdown>=1.1.5", ] [project.optional-dependencies] diff --git a/server/_integrations_routes.py b/server/_integrations_routes.py index 66b4345c..1b87e14f 100644 --- a/server/_integrations_routes.py +++ b/server/_integrations_routes.py @@ -168,9 +168,9 @@ async def handle_add_integration(request: web.Request) -> web.Response: # Wake channels that were waiting on a relevant integration to appear. # Each channel filters by slug so unrelated adds are no-ops. - telegram_runner = request.app.get("telegram_bot_runner") - if telegram_runner is not None: - telegram_runner.notify_integration_added(slug) + telegram_channel = request.app.get("telegram_channel") + if telegram_channel is not None: + telegram_channel.notify_integration_added(slug) return web.json_response(result, status=201) diff --git a/server/aiohttp_app.py b/server/aiohttp_app.py index eebc9255..6cc2ffc3 100644 --- a/server/aiohttp_app.py +++ b/server/aiohttp_app.py @@ -606,18 +606,18 @@ async def _init_telegram_bot(app: web.Application) -> None: integration in the wizard is the only enable/disable knob. """ config = load_config() - runner = TelegramChannel( + channel = TelegramChannel( app_sock_path=Path(config.integrations.app_sock_path), ) - await runner.start() - app["telegram_bot_runner"] = runner + await channel.start() + app["telegram_channel"] = channel async def _stop_telegram_bot(app: web.Application) -> None: - """Stop the Telegram bot runner if present.""" - runner: TelegramChannel | None = app.get("telegram_bot_runner") - if runner: - await runner.stop() + """Stop the Telegram channel if present.""" + channel: TelegramChannel | None = app.get("telegram_channel") + if channel: + await channel.stop() __all__ = ["create_app"] diff --git a/server/ui/src/components/SystemSettings.jsx b/server/ui/src/components/SystemSettings.jsx index 03cc345c..c066f8f5 100644 --- a/server/ui/src/components/SystemSettings.jsx +++ b/server/ui/src/components/SystemSettings.jsx @@ -4,6 +4,7 @@ import ModelPicker from './ModelPicker.jsx'; import PackageIcon from './icons/PackageIcon'; import EyeIcon from './icons/EyeIcon'; import CompactionIcon from './icons/CompactionIcon'; +import SendIcon from './icons/SendIcon'; import ToggleSwitch from './ToggleSwitch.jsx'; import ChevronRightIcon from './icons/ChevronRightIcon'; @@ -11,23 +12,28 @@ export default function SystemSettings() { const [providers, setProviders] = useState([]); const [profiles, setProfiles] = useState([]); const [settings, setSettings] = useState({ default_agent: 'computron' }); + const [telegramIntegrations, setTelegramIntegrations] = useState([]); const [loading, setLoading] = useState(true); const [visionAdvancedOpen, setVisionAdvancedOpen] = useState(false); useEffect(() => { async function init() { try { - const [providersRes, settingsRes, profilesRes] = await Promise.all([ + const [providersRes, settingsRes, profilesRes, integrationsRes] = await Promise.all([ fetch('/api/providers'), fetch('/api/settings'), fetch('/api/profiles'), + fetch('/api/integrations'), ]); const providersData = await providersRes.json(); const settingsData = await settingsRes.json(); const profilesData = await profilesRes.json(); + const integrationsData = integrationsRes.ok ? await integrationsRes.json() : { integrations: [] }; setProviders(providersData.providers || []); setSettings(settingsData); setProfiles(profilesData); + const all = integrationsData.integrations || []; + setTelegramIntegrations(all.filter((i) => i.slug === 'telegram')); } catch { // keep defaults on error } finally { @@ -241,6 +247,60 @@ export default function SystemSettings() { /> + + {/* Notifications */} +
Notifications
+ +
+
+
+ +
+
+ Telegram + Push goal completion / failure messages to a Telegram chat via a bot integration. Leave either field blank to disable. +
+
+
+
+ Bot integration + Add a Telegram integration in the Integrations tab first, then pick it here. +
+ +
+
+
+ Chat ID + + Message your bot once, then visit{' '} + https://api.telegram.org/bot<TOKEN>/getUpdates{' '} + and copy chat.id. Group chat IDs are negative. + +
+ { + const raw = e.target.value; + const num = raw === '' ? null : Number(raw); + updateSetting('telegram_notifier_chat_id', num); + }} + /> +
+
); } diff --git a/settings.py b/settings.py index b16ed952..a75700d6 100644 --- a/settings.py +++ b/settings.py @@ -52,6 +52,10 @@ }, "title_provider": "", "title_model": "", + # Goal-run push notifications. Empty integration_id or null chat_id + # leaves the notifier disabled. + "telegram_notifier_integration_id": "", + "telegram_notifier_chat_id": None, } # Metadata service IPs that must never be reachable via user-supplied URLs. @@ -89,6 +93,8 @@ class SettingsUpdate(BaseModel): compaction_options: dict[str, Any] | None = None title_provider: str | None = None title_model: str | None = None + telegram_notifier_integration_id: str | None = None + telegram_notifier_chat_id: int | None = None @field_validator("direct_providers") @classmethod diff --git a/tasks/_notifier.py b/tasks/_notifier.py index 838e91b8..f12e5192 100644 --- a/tasks/_notifier.py +++ b/tasks/_notifier.py @@ -7,11 +7,11 @@ from __future__ import annotations import logging -import os from pathlib import Path from typing import TYPE_CHECKING from integrations.broker_client import IntegrationError, call as broker_call +from settings import load_settings if TYPE_CHECKING: from config import NotificationsConfig @@ -26,10 +26,10 @@ class TelegramNotifier: """Sends notification messages to Telegram via the broker. - Reads ``TELEGRAM_INTEGRATION_ID`` and ``TELEGRAM_CHAT_ID`` from the - environment. If either is missing, the notifier disables itself with a - warning. All public methods are fire-and-forget — errors are logged, - never raised. + Configuration is read from app settings (``telegram_notifier_integration_id`` + and ``telegram_notifier_chat_id``) on construction. If either is blank, + the notifier disables itself with a warning. All public methods are + fire-and-forget — errors are logged, never raised. """ def __init__( @@ -40,21 +40,15 @@ def __init__( ) -> None: self._config = config self._app_sock = app_sock_path - self._integration_id = os.environ.get("TELEGRAM_INTEGRATION_ID", "") - chat_id_raw = os.environ.get("TELEGRAM_CHAT_ID", "") - try: - self._chat_id: int | None = int(chat_id_raw) if chat_id_raw else None - except ValueError: - logger.warning( - "TELEGRAM_CHAT_ID is not an integer (%r); Telegram notifications " - "disabled", chat_id_raw, - ) - self._chat_id = None + settings = load_settings() + self._integration_id = settings.get("telegram_notifier_integration_id") or "" + chat_id = settings.get("telegram_notifier_chat_id") + self._chat_id: int | None = chat_id if isinstance(chat_id, int) else None if not self._integration_id or self._chat_id is None: logger.warning( - "TELEGRAM_INTEGRATION_ID or TELEGRAM_CHAT_ID not set — " - "Telegram notifications disabled", + "Telegram notifier integration_id or chat_id not configured — " + "notifications disabled", ) self._disabled = True else: diff --git a/tests/unit/channels/telegram/test_formatter.py b/tests/unit/channels/telegram/test_formatter.py index ba272ad7..97eecdc5 100644 --- a/tests/unit/channels/telegram/test_formatter.py +++ b/tests/unit/channels/telegram/test_formatter.py @@ -7,122 +7,90 @@ from channels.telegram._formatter import TelegramFormatter +# --------------------------------------------------------------------------- +# to_markdownv2_chunks() — CommonMark → MarkdownV2 conversion + chunking +# --------------------------------------------------------------------------- + + +@pytest.mark.unit +def test_short_text_returns_single_chunk(): + chunks = TelegramFormatter.to_markdownv2_chunks("hello world") + assert len(chunks) == 1 + + +@pytest.mark.unit +def test_bold_double_asterisk_converted_to_single_asterisk(): + """Agent emits ``**bold**`` (CommonMark); MarkdownV2 uses ``*bold*``.""" + chunks = TelegramFormatter.to_markdownv2_chunks("hello **world**") + body = "".join(chunks) + assert "*world*" in body + assert "**world**" not in body + + +@pytest.mark.unit +def test_italic_underscore_in_output(): + """``*italic*`` in CommonMark becomes ``_italic_`` in MarkdownV2.""" + chunks = TelegramFormatter.to_markdownv2_chunks("an *italic* word") + body = "".join(chunks) + assert "_italic_" in body + + +@pytest.mark.unit +def test_inline_code_preserved(): + chunks = TelegramFormatter.to_markdownv2_chunks("call `do_thing()` now") + body = "".join(chunks) + assert "`do_thing" in body + + +@pytest.mark.unit +def test_fenced_code_block_preserved(): + text = "before\n\n```python\ndef hi(): pass\n```\n\nafter" + chunks = TelegramFormatter.to_markdownv2_chunks(text) + body = "".join(chunks) + assert "```" in body + assert "def hi" in body + + @pytest.mark.unit -class TestSplit: - """Tests for the split() chunker — paragraph/sentence/hard-split branches.""" - - def test_short_text_returned_unchanged(self): - chunks = TelegramFormatter.split("hello world", limit=4096) - assert chunks == ["hello world"] - - def test_empty_string_returns_single_empty(self): - # An empty string is below the limit, so it comes back as a single chunk. - assert TelegramFormatter.split("", limit=100) == [""] - - def test_exact_limit_not_split(self): - text = "x" * 100 - chunks = TelegramFormatter.split(text, limit=100) - assert chunks == [text] - - def test_paragraph_break_preferred(self): - """When a double-newline exists before the limit, the first split - happens at the paragraph boundary.""" - text = "para one is here.\n\npara two is short." - chunks = TelegramFormatter.split(text, limit=30) - assert len(chunks) == 2 - assert chunks[0] == "para one is here." - assert chunks[1].startswith("para two") - - def test_single_newline_when_no_paragraph_break(self): - """Falls through to a single-newline split when no double-newline fits.""" - text = "line one of text\nline two of text\nline three" - chunks = TelegramFormatter.split(text, limit=20) - # Should split on a newline before position 20. - assert len(chunks) >= 2 - assert all(len(c) <= 20 for c in chunks) - # No chunk should contain a leading newline (lstrip\n applied). - assert all(not c.startswith("\n") for c in chunks) - - def test_sentence_boundary_fallback(self): - """Falls through to a sentence end when no newline before the limit. - - The splitter's sentence cut sits *before* the punctuation, so the - next chunk starts with the punctuation. Assert on the partitioning - rather than the boundary character. - """ - text = "First sentence ends here. Second sentence is right after! Third?" - chunks = TelegramFormatter.split(text, limit=30) - assert len(chunks) >= 2 - # Both halves reflect the original content split somewhere mid-string. - joined = "".join(chunks) - # Joining loses the whitespace that was stripped at the boundaries; - # confirm the meaningful tokens survive. - for token in ("First", "Second", "Third"): - assert token in joined - - def test_hard_split_when_no_boundary(self): - """Long unbroken text gets a hard split at the limit.""" - text = "x" * 50 - chunks = TelegramFormatter.split(text, limit=10) - # 50 / 10 = 5 hard-split chunks. - assert len(chunks) == 5 - assert all(len(c) == 10 for c in chunks) - - def test_hard_split_when_only_late_boundary(self): - """Sentence break later than limit//4 wins; earlier than that is too short, hard-split instead.""" - # Sentence break is at position 5, limit is 20, limit//4 is 5 — boundary - # >= limit//4 wins. Make it boundary < limit//4 to force hard split. - text = "ab. " + "x" * 40 # period at index 2, limit//4 = 5 -> too early - chunks = TelegramFormatter.split(text, limit=20) - # First chunk should be ~20 chars, not "ab." - assert len(chunks[0]) > 5 - - def test_chunks_join_to_original_modulo_whitespace_trim(self): - """Chunks reassemble back to the original input ignoring per-chunk - leading-newline strip and trailing-whitespace trim.""" - text = ( - "Paragraph one with some content.\n\n" - "Paragraph two has a fair bit more text so it crosses the limit boundary.\n\n" - "Paragraph three." - ) - chunks = TelegramFormatter.split(text, limit=60) - joined = "\n\n".join(chunks) - # Whitespace normalization makes exact equality tricky; just confirm - # each paragraph survives somewhere in the output. - assert "Paragraph one" in joined - assert "Paragraph two" in joined - assert "Paragraph three" in joined +def test_punctuation_escaped_outside_code(): + """MarkdownV2 requires escaping bare ``.`` and ``(`` ``)`` in body text.""" + chunks = TelegramFormatter.to_markdownv2_chunks("hello (world).") + body = "".join(chunks) + assert "\\(" in body + assert "\\)" in body + assert "\\." in body @pytest.mark.unit -class TestEscapeMarkdown: - """Tests for MarkdownV2 escaping.""" +def test_long_text_is_split_under_4096_each(): + """Output chunks fit Telegram's 4096-byte message limit.""" + # Build something well past 4096 chars with plenty of structure so the + # splitter has natural boundaries to cut on. + text = ("paragraph with some words here.\n\n" * 300) + chunks = TelegramFormatter.to_markdownv2_chunks(text) + assert len(chunks) >= 2 + for chunk in chunks: + assert len(chunk.encode("utf-16-le")) // 2 <= 4096 + - def test_escapes_all_special_chars(self): - out = TelegramFormatter.escape_markdown("hello *world* (yes)") - assert "\\*" in out - assert "\\(" in out - assert "\\)" in out +# --------------------------------------------------------------------------- +# file_caption() +# --------------------------------------------------------------------------- - def test_plain_text_passthrough(self): - out = TelegramFormatter.escape_markdown("plain text") - assert out == "plain text" + +@pytest.mark.unit +def test_single_file_caption(): + caption = TelegramFormatter.file_caption(Path("report.pdf")) + assert caption == "📎 report.pdf" @pytest.mark.unit -class TestFileCaption: - """Tests for file_caption().""" - - def test_single_file_caption(self): - caption = TelegramFormatter.file_caption(Path("report.pdf")) - assert caption == "📎 report.pdf" - - def test_multi_file_caption_includes_index(self): - caption = TelegramFormatter.file_caption( - Path("/tmp/a.txt"), index=0, total=3, - ) - assert caption == "📎 a.txt (1/3)" - caption = TelegramFormatter.file_caption( - Path("/tmp/b.txt"), index=2, total=3, - ) - assert caption == "📎 b.txt (3/3)" +def test_multi_file_caption_includes_index(): + caption = TelegramFormatter.file_caption( + Path("/tmp/a.txt"), index=0, total=3, + ) + assert caption == "📎 a.txt (1/3)" + caption = TelegramFormatter.file_caption( + Path("/tmp/b.txt"), index=2, total=3, + ) + assert caption == "📎 b.txt (3/3)" diff --git a/tests/unit/channels/telegram/test_list_helpers.py b/tests/unit/channels/telegram/test_list_helpers.py index 8c2752d6..208d1a0e 100644 --- a/tests/unit/channels/telegram/test_list_helpers.py +++ b/tests/unit/channels/telegram/test_list_helpers.py @@ -1,10 +1,10 @@ -"""Tests for the /list helpers in channels.telegram._runner.""" +"""Tests for the /list helpers in channels.telegram._channel.""" from __future__ import annotations import pytest -from channels.telegram._runner import _matches_query, _row_label +from channels.telegram._channel import _matches_query, _row_label from conversations._models import ConversationSummary diff --git a/tests/unit/integrations/brokers/telegram_broker/test_verbs.py b/tests/unit/integrations/brokers/telegram_broker/test_verbs.py index d70b44a8..3cb30505 100644 --- a/tests/unit/integrations/brokers/telegram_broker/test_verbs.py +++ b/tests/unit/integrations/brokers/telegram_broker/test_verbs.py @@ -174,9 +174,32 @@ async def test_sends_and_returns_message_id(self): ) assert out == {"message_id": 777} bot.send_message.assert_awaited_once_with( - chat_id=42, text="hi", reply_to_message_id=None, reply_markup=None, + chat_id=42, + text="hi", + reply_to_message_id=None, + reply_markup=None, + parse_mode=None, ) + async def test_passes_parse_mode_when_provided(self): + bot = MagicMock() + bot.send_message = AsyncMock(return_value=_FakeSentMessage(message_id=1)) + dispatcher = _make_dispatcher(bot=bot) + await dispatcher.dispatch( + "send_message", + {"chat_id": 1, "text": "hi", "parse_mode": "MarkdownV2"}, + ) + assert bot.send_message.await_args.kwargs["parse_mode"] == "MarkdownV2" + + async def test_rejects_non_string_parse_mode(self): + dispatcher = _make_dispatcher() + with pytest.raises(RpcError) as exc: + await dispatcher.dispatch( + "send_message", + {"chat_id": 1, "text": "hi", "parse_mode": 42}, + ) + assert exc.value.code == "BAD_REQUEST" + async def test_passes_reply_to_message_id_when_provided(self): bot = MagicMock() bot.send_message = AsyncMock(return_value=_FakeSentMessage(message_id=1)) diff --git a/tests/unit/migrations/test_006_telegram_notifier_settings.py b/tests/unit/migrations/test_006_telegram_notifier_settings.py new file mode 100644 index 00000000..e7474f6a --- /dev/null +++ b/tests/unit/migrations/test_006_telegram_notifier_settings.py @@ -0,0 +1,61 @@ +"""Tests for migration 006: Telegram notifier keys seeded into settings.json.""" + +import json + +import pytest + +from migrations._006_telegram_notifier_settings import migrate + + +@pytest.fixture() +def state_dir(tmp_path): + """State directory root.""" + return tmp_path + + +@pytest.mark.unit +def test_no_settings_file_is_noop(state_dir): + """Install without a settings.json does nothing — defaults apply on read.""" + migrate(state_dir) + assert not (state_dir / "settings.json").exists() + + +@pytest.mark.unit +def test_seeds_missing_fields(state_dir): + """A pre-existing settings.json without notifier fields gets them filled in.""" + path = state_dir / "settings.json" + path.write_text(json.dumps({"setup_complete": True})) + + migrate(state_dir) + + data = json.loads(path.read_text()) + assert data["setup_complete"] is True + assert data["telegram_notifier_integration_id"] == "" + assert data["telegram_notifier_chat_id"] is None + + +@pytest.mark.unit +def test_preserves_existing_values(state_dir): + """User-set notifier values are not overwritten.""" + path = state_dir / "settings.json" + path.write_text(json.dumps({ + "telegram_notifier_integration_id": "telegram_personal", + "telegram_notifier_chat_id": 42, + })) + + migrate(state_dir) + + data = json.loads(path.read_text()) + assert data["telegram_notifier_integration_id"] == "telegram_personal" + assert data["telegram_notifier_chat_id"] == 42 + + +@pytest.mark.unit +def test_corrupt_settings_file_is_skipped(state_dir): + """A corrupt settings.json doesn't raise; it's left alone.""" + path = state_dir / "settings.json" + path.write_text("{not-json") + + migrate(state_dir) + + assert path.read_text() == "{not-json" diff --git a/tests/unit/server/test_settings.py b/tests/unit/server/test_settings.py index 613299f2..86077bea 100644 --- a/tests/unit/server/test_settings.py +++ b/tests/unit/server/test_settings.py @@ -156,3 +156,32 @@ def test_exclude_unset_omits_defaults(self): assert "vision_model" in dumped assert "setup_complete" not in dumped assert "direct_providers" not in dumped + + +@pytest.mark.unit +def test_telegram_notifier_settings_accepted(): + """The notifier integration_id and chat_id fields round-trip through the model.""" + u = SettingsUpdate( + telegram_notifier_integration_id="telegram_personal", + telegram_notifier_chat_id=42, + ) + assert u.telegram_notifier_integration_id == "telegram_personal" + assert u.telegram_notifier_chat_id == 42 + + +@pytest.mark.unit +def test_telegram_notifier_settings_nullable(): + """Both fields accept null (the explicit "disabled" value).""" + u = SettingsUpdate( + telegram_notifier_integration_id=None, + telegram_notifier_chat_id=None, + ) + assert u.telegram_notifier_integration_id is None + assert u.telegram_notifier_chat_id is None + + +@pytest.mark.unit +def test_telegram_notifier_chat_id_accepts_negative(): + """Group chats have negative IDs; the field must accept them.""" + u = SettingsUpdate(telegram_notifier_chat_id=-1001234567890) + assert u.telegram_notifier_chat_id == -1001234567890 diff --git a/tests/unit/tasks/test_notifier.py b/tests/unit/tasks/test_notifier.py index 83843c7a..9c49af22 100644 --- a/tests/unit/tasks/test_notifier.py +++ b/tests/unit/tasks/test_notifier.py @@ -1,6 +1,5 @@ """Tests for tasks._notifier — Telegram push notifications via the broker.""" -import os from pathlib import Path from unittest.mock import AsyncMock, patch @@ -21,152 +20,186 @@ def _make_config(**overrides): return NotificationsConfig(**overrides) -_ENABLED_ENV = { - "TELEGRAM_INTEGRATION_ID": "telegram_personal", - "TELEGRAM_CHAT_ID": "42", +_ENABLED_SETTINGS = { + "telegram_notifier_integration_id": "telegram_personal", + "telegram_notifier_chat_id": 42, } _APP_SOCK = Path("/tmp/test_app.sock") +def _patch_settings(settings: dict): + """Patch ``load_settings`` inside the notifier module.""" + return patch("tasks._notifier.load_settings", return_value=settings) + + @pytest.mark.unit -class TestTelegramNotifier: - """Test TelegramNotifier init and send behavior.""" - - def test_enables_when_env_vars_present(self): - """Notifier is enabled when both env vars are set.""" - with patch.dict(os.environ, _ENABLED_ENV): - notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) - assert notifier.enabled - - def test_disabled_when_env_vars_missing(self): - """Notifier disables itself with a warning when either env is unset.""" - with patch.dict(os.environ, {}, clear=True): - notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) - assert not notifier.enabled - - def test_disabled_when_chat_id_not_numeric(self): - """A non-integer chat ID disables the notifier.""" - with patch.dict( - os.environ, - {"TELEGRAM_INTEGRATION_ID": "telegram_personal", "TELEGRAM_CHAT_ID": "not-a-number"}, - ): - notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) - assert not notifier.enabled - - async def test_send_noop_when_disabled(self): - """Sending on a disabled notifier is a silent no-op (no broker call).""" - with patch.dict(os.environ, {}, clear=True): - notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) - with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: - await notifier.send("hello") - mock_call.assert_not_called() - - async def test_send_calls_broker(self): - """Sends a message via broker_client.call('send_message', ...).""" - with patch.dict(os.environ, _ENABLED_ENV): - notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) - with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: - mock_call.return_value = {"message_id": 1} - - await notifier.send("test message") - - mock_call.assert_awaited_once_with( - "telegram_personal", - "send_message", - {"chat_id": 42, "text": "test message"}, - app_sock_path=_APP_SOCK, - ) - - async def test_send_truncates_long_messages(self): - """Messages over the Telegram limit are truncated before sending.""" - with patch.dict(os.environ, _ENABLED_ENV): - notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) - with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: - mock_call.return_value = {"message_id": 1} - - await notifier.send("x" * 5000) - - args = mock_call.await_args.args - sent_text = args[2]["text"] - assert len(sent_text) <= 4096 - assert sent_text.endswith("… (truncated)") - - async def test_send_skips_attachments_for_now(self, tmp_path): - """Attachments are logged-and-skipped until broker send_document lands.""" - test_file = tmp_path / "report.pdf" - test_file.write_bytes(b"fake pdf content") - - with patch.dict(os.environ, _ENABLED_ENV): - notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) - with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: - mock_call.return_value = {"message_id": 1} - - await notifier.send("msg", attachments=[test_file]) - - # Only the text send_message call; no document call. - assert mock_call.await_count == 1 - assert mock_call.await_args.args[1] == "send_message" - - async def test_send_does_not_raise_on_broker_error(self): - """Errors from the broker are logged, never raised.""" - with patch.dict(os.environ, _ENABLED_ENV): - notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) - with patch( - "tasks._notifier.broker_call", - new_callable=AsyncMock, - side_effect=IntegrationError("broker offline"), - ): - # Should not raise - await notifier.send("test") +def test_enables_when_settings_present(): + """Notifier is enabled when both settings are populated.""" + with _patch_settings(_ENABLED_SETTINGS): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + assert notifier.enabled @pytest.mark.unit -class TestMessageFormatting: - """Test notification message formatting.""" - - def test_format_run_completed(self): - """Success message includes goal name, stats, output, and file count.""" - msg = format_run_completed( - goal_description="Find Pop-Tarts prices", - run_number=2, - duration="47s", - total_tasks=3, - completed_tasks=3, - final_output="Walmart: $3.48", - file_count=1, - ) - assert "Find Pop-Tarts prices" in msg - assert "Run #2" in msg - assert "3/3" in msg - assert "Walmart: $3.48" in msg - assert "1 file attached" in msg - - def test_format_run_completed_no_files(self): - """Success message omits file line when no files.""" - msg = format_run_completed( - goal_description="Test", - run_number=1, - duration="5s", - total_tasks=1, - completed_tasks=1, - final_output="done", - file_count=0, - ) - assert "file" not in msg - - def test_format_run_failed(self): - """Failure message includes error details.""" - msg = format_run_failed( - goal_description="Scrape data", - run_number=1, - duration="12s", - total_tasks=3, - completed_tasks=1, - failed_task_description="Fetch page", - error="ConnectionError: timeout", +def test_disabled_when_settings_blank(): + """Notifier disables itself when integration_id is blank or chat_id is None.""" + with _patch_settings({}): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + assert not notifier.enabled + + +@pytest.mark.unit +def test_disabled_when_only_integration_id_set(): + with _patch_settings( + {"telegram_notifier_integration_id": "telegram_personal", "telegram_notifier_chat_id": None}, + ): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + assert not notifier.enabled + + +@pytest.mark.unit +def test_disabled_when_only_chat_id_set(): + with _patch_settings( + {"telegram_notifier_integration_id": "", "telegram_notifier_chat_id": 42}, + ): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + assert not notifier.enabled + + +@pytest.mark.unit +def test_disabled_when_chat_id_not_int(): + """A non-integer chat_id (e.g. stray string) disables the notifier rather than crashing.""" + with _patch_settings( + {"telegram_notifier_integration_id": "telegram_personal", "telegram_notifier_chat_id": "42"}, + ): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + assert not notifier.enabled + + +@pytest.mark.unit +async def test_send_noop_when_disabled(): + """Sending on a disabled notifier is a silent no-op (no broker call).""" + with _patch_settings({}): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: + await notifier.send("hello") + mock_call.assert_not_called() + + +@pytest.mark.unit +async def test_send_calls_broker(): + """Sends a message via broker_client.call('send_message', ...).""" + with _patch_settings(_ENABLED_SETTINGS): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: + mock_call.return_value = {"message_id": 1} + + await notifier.send("test message") + + mock_call.assert_awaited_once_with( + "telegram_personal", + "send_message", + {"chat_id": 42, "text": "test message"}, + app_sock_path=_APP_SOCK, ) - assert "Scrape data" in msg - assert "1/3" in msg - assert "Fetch page" in msg - assert "ConnectionError" in msg + + +@pytest.mark.unit +async def test_send_truncates_long_messages(): + """Messages over the Telegram limit are truncated before sending.""" + with _patch_settings(_ENABLED_SETTINGS): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: + mock_call.return_value = {"message_id": 1} + + await notifier.send("x" * 5000) + + args = mock_call.await_args.args + sent_text = args[2]["text"] + assert len(sent_text) <= 4096 + assert sent_text.endswith("… (truncated)") + + +@pytest.mark.unit +async def test_send_skips_attachments_for_now(tmp_path): + """Attachments are logged-and-skipped until broker send_document lands.""" + test_file = tmp_path / "report.pdf" + test_file.write_bytes(b"fake pdf content") + + with _patch_settings(_ENABLED_SETTINGS): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + with patch("tasks._notifier.broker_call", new_callable=AsyncMock) as mock_call: + mock_call.return_value = {"message_id": 1} + + await notifier.send("msg", attachments=[test_file]) + + # Only the text send_message call; no document call. + assert mock_call.await_count == 1 + assert mock_call.await_args.args[1] == "send_message" + + +@pytest.mark.unit +async def test_send_does_not_raise_on_broker_error(): + """Errors from the broker are logged, never raised.""" + with _patch_settings(_ENABLED_SETTINGS): + notifier = TelegramNotifier(_make_config(), app_sock_path=_APP_SOCK) + with patch( + "tasks._notifier.broker_call", + new_callable=AsyncMock, + side_effect=IntegrationError("broker offline"), + ): + # Should not raise + await notifier.send("test") + + +@pytest.mark.unit +def test_format_run_completed(): + """Success message includes goal name, stats, output, and file count.""" + msg = format_run_completed( + goal_description="Find Pop-Tarts prices", + run_number=2, + duration="47s", + total_tasks=3, + completed_tasks=3, + final_output="Walmart: $3.48", + file_count=1, + ) + assert "Find Pop-Tarts prices" in msg + assert "Run #2" in msg + assert "3/3" in msg + assert "Walmart: $3.48" in msg + assert "1 file attached" in msg + + +@pytest.mark.unit +def test_format_run_completed_no_files(): + """Success message omits file line when no files.""" + msg = format_run_completed( + goal_description="Test", + run_number=1, + duration="5s", + total_tasks=1, + completed_tasks=1, + final_output="done", + file_count=0, + ) + assert "file" not in msg + + +@pytest.mark.unit +def test_format_run_failed(): + """Failure message includes error details.""" + msg = format_run_failed( + goal_description="Scrape data", + run_number=1, + duration="12s", + total_tasks=3, + completed_tasks=1, + failed_task_description="Fetch page", + error="ConnectionError: timeout", + ) + assert "Scrape data" in msg + assert "1/3" in msg + assert "Fetch page" in msg + assert "ConnectionError" in msg diff --git a/uv.lock b/uv.lock index 1f4af395..d24172d3 100644 --- a/uv.lock +++ b/uv.lock @@ -351,6 +351,7 @@ dependencies = [ { name = "python-dotenv" }, { name = "pyyaml" }, { name = "rich" }, + { name = "telegramify-markdown" }, { name = "trafilatura" }, ] @@ -397,6 +398,7 @@ requires-dist = [ { name = "pyyaml" }, { name = "rich", specifier = ">=14.3.3" }, { name = "ruff", marker = "extra == 'dev'" }, + { name = "telegramify-markdown", specifier = ">=1.1.5" }, { name = "trafilatura", specifier = ">=2.0" }, { name = "types-cachetools", marker = "extra == 'dev'" }, { name = "types-pyyaml", marker = "extra == 'dev'" }, @@ -1466,6 +1468,142 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/10/bd/c038d7cc38edc1aa5bf91ab8068b63d4308c66c4c8bb3cbba7dfbc049f9c/pyparsing-3.3.2-py3-none-any.whl", hash = "sha256:850ba148bd908d7e2411587e247a1e4f0327839c40e2e5e6d05a007ecc69911d", size = 122781, upload-time = "2026-01-21T03:57:55.912Z" }, ] +[[package]] +name = "pyromark" +version = "0.9.11" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/5c/a3/b46d6253b3a07d1116080864ecf00fc6d3456b1978d4b5b7df3c0022903e/pyromark-0.9.11.tar.gz", hash = "sha256:cef5de337efc14544a3bdc27fd67a1e84096209c8ba28153202bf1f471b0dcec", size = 9090, upload-time = "2026-05-24T22:19:28.064Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/34/77/ebe5e19c60b2e7e052fcc448cccea4bb5a021c1e8b9be23960f8786b9677/pyromark-0.9.11-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:3a5eadf6a793fc183922fe493dfa1f9d0e6e35811772c6e02aa4957aca9d63c3", size = 347123, upload-time = "2026-05-24T22:20:56.739Z" }, + { url = "https://files.pythonhosted.org/packages/54/cf/d273c92a469a492b3196f4dae02f79bc94a5df7fac8536d764d7d14a9b35/pyromark-0.9.11-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:afff4b52de0b67a94a1dd9b7a542bf40f850c3df8f3ec6b3a7f5d11c579635e5", size = 329835, upload-time = "2026-05-24T22:21:05.158Z" }, + { url = "https://files.pythonhosted.org/packages/a5/d2/607f710b88c160b7f7a2ce8b2b74de469fa6a83553705ed058715295866e/pyromark-0.9.11-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6ef868d75cb5e850454f97068e37079ea4c080407712da520ca5a72e266e9d19", size = 363344, upload-time = "2026-05-24T22:20:40.154Z" }, + { url = "https://files.pythonhosted.org/packages/ed/ac/7f1376063aeac39f67ef0c6f6f205ae4c1d848aaf24d6a4a5529b66e4833/pyromark-0.9.11-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:da8615e35307173f19f3ff50106b888fe4199dfd1a6ff32607c37e120dd09460", size = 361739, upload-time = "2026-05-24T22:19:31.3Z" }, + { url = "https://files.pythonhosted.org/packages/50/15/7e6bea7c074c1b132696ff4619f5c7e5cb5302b04f2ead136665eb1809da/pyromark-0.9.11-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4417e5a8a30900d0608cee8c5290f307aa495e46c64f9ce335241ed9fc52f26d", size = 387403, upload-time = "2026-05-24T22:18:59.412Z" }, + { url = "https://files.pythonhosted.org/packages/89/69/b20628ecc8c72a825d3b99db0446d7ba07b40d81982b52cc35a87f2a1325/pyromark-0.9.11-cp312-cp312-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:5208ec52e2daf927c2e6b6e35d8949a95d450acc2b9e5fcc3c0e277db3681ab7", size = 418091, upload-time = "2026-05-24T22:21:12.607Z" }, + { url = "https://files.pythonhosted.org/packages/31/10/c104d243e66a6bbb3565de6a0229e8c7177ce6103184747f72ecc497f9f6/pyromark-0.9.11-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0be355ed4884a29e7acb0c2518cfc7a18d23822eaca6d8622bfeb99bf365e4f8", size = 410132, upload-time = "2026-05-24T22:19:52.941Z" }, + { url = "https://files.pythonhosted.org/packages/31/c9/50c2be699fadcd601ac3c103ae61d7382af6ddd18e6922983762a0f02600/pyromark-0.9.11-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:72b7ea10a1a9a6eee49cd3de65b61fc35dc0a793c1c271854a4ffaa0480d7d50", size = 448261, upload-time = "2026-05-24T22:19:43.188Z" }, + { url = "https://files.pythonhosted.org/packages/f9/1c/663791a32f81865326f39189bc0bf7670cc6e2118b46974bcfbcd53400ea/pyromark-0.9.11-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d1af981c168121025fb7e63647a1b2aa323c60a401eff8ffc6e26977bf73a5d3", size = 371474, upload-time = "2026-05-24T22:18:50.184Z" }, + { url = "https://files.pythonhosted.org/packages/e1/17/454c9b7cba68f28df9223ee08b07272af2278f26ad921794f258249fe9bb/pyromark-0.9.11-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:fe5a0b48eaa195164a0cb8af9a49c49863414e8934c9066e8e75db08fef1cad6", size = 363938, upload-time = "2026-05-24T22:22:44.927Z" }, + { url = "https://files.pythonhosted.org/packages/f7/e0/17e8174e0adc819abbc9af0ceb1bb56087a90c3c576faec7d3880b729cc1/pyromark-0.9.11-cp312-cp312-manylinux_2_28_armv7l.whl", hash = "sha256:f6c03533786ae03e937f127512347a290e1f949c6377abe77678d6363776f327", size = 361669, upload-time = "2026-05-24T22:20:03.002Z" }, + { url = "https://files.pythonhosted.org/packages/58/7d/dc8cacc95710c0b277ba793b16d95920ab2357f14617e5d8db3de0618f2c/pyromark-0.9.11-cp312-cp312-manylinux_2_28_i686.whl", hash = "sha256:73b7227e98b4cb923fea4c84078f16b455e4b498d4a6d3e3771763e77d6e7cf3", size = 387706, upload-time = "2026-05-24T22:21:34.502Z" }, + { url = "https://files.pythonhosted.org/packages/ac/6b/af991f52f6ff1f645034986074a7664af4631d87500558d1b3acb6df8d40/pyromark-0.9.11-cp312-cp312-manylinux_2_28_ppc64le.whl", hash = "sha256:d784577d5ba257ea9ae035b23566aa816b9659593638c1dd971ef5711a57ab02", size = 410263, upload-time = "2026-05-24T22:19:54.33Z" }, + { url = "https://files.pythonhosted.org/packages/5b/16/cf13492e02da17301e500e1c9765ded899a52192f91e47c5f1029ad019d1/pyromark-0.9.11-cp312-cp312-manylinux_2_28_s390x.whl", hash = "sha256:c2085dcd93283c9a36b9151eafeeeab06c277b29f651f2c0630214437e2f1025", size = 448472, upload-time = "2026-05-24T22:22:35.296Z" }, + { url = "https://files.pythonhosted.org/packages/c2/78/f1d4c67f2f10bbc20ee1326bf3e8cb3aaf3bcf7a333bc9aaf2c9d8632d5c/pyromark-0.9.11-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:5707945f3fb9b86a9de942c151d5e1c60739487b7568d8095bba8a661fe855f8", size = 371901, upload-time = "2026-05-24T22:19:37.594Z" }, + { url = "https://files.pythonhosted.org/packages/08/bb/d454b0643c7c32c7a38c792e60f5b1809daaeb886ef08211c766f0a16e3b/pyromark-0.9.11-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:12ed041049380da29edfc6fa32952de367155782994be897040df62f5d4aefb8", size = 372228, upload-time = "2026-05-24T22:22:54.827Z" }, + { url = "https://files.pythonhosted.org/packages/00/02/cba0e603f82bd57c50281497a5e53bcc283d5d88542f0f46ca7bcb635de8/pyromark-0.9.11-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8a09cb7fd3fed58efa6796d9c4142f5fb4def4b8f3d5871ad7fcb986af43efa8", size = 540654, upload-time = "2026-05-24T22:22:50.136Z" }, + { url = "https://files.pythonhosted.org/packages/56/b1/a848e7d7d09d278b6025ae9d090e9e7467de3495a35263e895ae1100d5ca/pyromark-0.9.11-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:d167b897625b7efc0d432a992fca1a4a128064bf07f4eaa3986c79ab8f4ad230", size = 638331, upload-time = "2026-05-24T22:22:51.767Z" }, + { url = "https://files.pythonhosted.org/packages/16/f8/1145df9fa6d990c80dfb977aa94e23a6e6e4894d8e3439678867852e652a/pyromark-0.9.11-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:009fc49414e147c886881860f008125448b92fbc52f90aa44d6ac987801b6711", size = 605152, upload-time = "2026-05-24T22:19:46.706Z" }, + { url = "https://files.pythonhosted.org/packages/af/e7/bb57d1e5fac16b3e81588eb550dec206b5947de9903420feb1dfebd3960b/pyromark-0.9.11-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:12945db6c9bf6fc5f742646e05d683a93ba5056f2b1fe6c1b2e15e640af8fbb6", size = 540741, upload-time = "2026-05-24T22:21:55.035Z" }, + { url = "https://files.pythonhosted.org/packages/3c/56/ccbfe1cd8fefb8c7d5fdbda1651cc8c2e913f8de4cb337c5e6dc983a0608/pyromark-0.9.11-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:f8837be092c667ecb8b0fef94d4f279325b2ad9ab63d3b07c48c56687d70100c", size = 546256, upload-time = "2026-05-24T22:22:28.385Z" }, + { url = "https://files.pythonhosted.org/packages/16/76/0a73a3a9d8aabddb69fc09400249ee72b8a3c85c736dc00cf79e6682cb85/pyromark-0.9.11-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:ada5b2190e6eda27c8cd0fdb8c07ab5cad9408680b17b78c53fbc1a5f845d645", size = 590983, upload-time = "2026-05-24T22:19:06.791Z" }, + { url = "https://files.pythonhosted.org/packages/32/53/efc4baf1fb44575264263bf226863e09f0cf51b54a0da439576aa42f9959/pyromark-0.9.11-cp312-cp312-win32.whl", hash = "sha256:698f383677087972cf88622b54627857b48cb910f0ab508857676f56ed157782", size = 261116, upload-time = "2026-05-24T22:20:41.612Z" }, + { url = "https://files.pythonhosted.org/packages/57/9f/8378d1c7f5097fa876747d3c594a0f23f4d8a684b07305494dff7416f111/pyromark-0.9.11-cp312-cp312-win_amd64.whl", hash = "sha256:d309fe00ff29869fcc79c74d46ebe1f073bbdcedbeb116189d637673f201b08e", size = 273258, upload-time = "2026-05-24T22:19:36.515Z" }, + { url = "https://files.pythonhosted.org/packages/0c/48/c24eb9ea06fbb651afac1a98ab39e9b071774cb8530f0a545ad196d89841/pyromark-0.9.11-cp312-cp312-win_arm64.whl", hash = "sha256:47d61a74d3d7b914b9ed33f5bfc0b48a53d4e30e044e4f4ba00875d934c5ea06", size = 258979, upload-time = "2026-05-24T22:22:08.195Z" }, + { url = "https://files.pythonhosted.org/packages/b8/28/8e01ddcdd0ea0e7f1ffa4ee3d9972b64832a9c57e7c2c135f8469026e73a/pyromark-0.9.11-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:a18d62fe13bfe9fa0b2e9bc60ab4bc233f949b6af87488b57f205cd89ac79d75", size = 347157, upload-time = "2026-05-24T22:22:17.871Z" }, + { url = "https://files.pythonhosted.org/packages/20/86/fdda575c727947cb172dfb9997df4d40846daf931dc578b8c51dd985b754/pyromark-0.9.11-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:a4e47c0f70c3193f73833e4a95243f85f19d5c6ad72fba4a5e966353c245fcd2", size = 329873, upload-time = "2026-05-24T22:21:39.137Z" }, + { url = "https://files.pythonhosted.org/packages/a5/29/4910abd9406d24ec95529de2e45532ccacb7465288da63195d32549b316d/pyromark-0.9.11-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0105620485a0345787828e9692567fef3721d43120c5c36f5f618c5fdb333924", size = 363306, upload-time = "2026-05-24T22:23:04.508Z" }, + { url = "https://files.pythonhosted.org/packages/87/88/1da6a8696bf127afbe5977a68dcb9d390cd43d5aa7b76239d074daadfad7/pyromark-0.9.11-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:c8fa803f6a08e510eb1506a498a977ae52a6936e34f462422de6810f30b38ead", size = 361365, upload-time = "2026-05-24T22:20:13.596Z" }, + { url = "https://files.pythonhosted.org/packages/53/2d/fb4390089637523eac70068a64622688675496fa760b7473477426ca09f7/pyromark-0.9.11-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e7654ad0c23babae77f28af1de7efa9788bbbca2c29299921d8620549c2643ef", size = 387168, upload-time = "2026-05-24T22:22:09.882Z" }, + { url = "https://files.pythonhosted.org/packages/35/8a/d65013e444497a170c280d7119b63382cbf978bad96cfb01c9fb4d0faee1/pyromark-0.9.11-cp313-cp313-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:69fb8cf5fa37a47a4456db2a04b40ef9cb7ad244d2ca1892270e6ae28a882e9e", size = 418452, upload-time = "2026-05-24T22:20:09.664Z" }, + { url = "https://files.pythonhosted.org/packages/77/47/ddc019e2133c42507cbd9d3cc8bc884921f1c455bd8c5d5d75c4b162806e/pyromark-0.9.11-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e0479d32569ed1162012353bbd92eeed925b4da9c591cde3c3aa981895e8951d", size = 409822, upload-time = "2026-05-24T22:18:44.743Z" }, + { url = "https://files.pythonhosted.org/packages/9f/52/93b326323b002fad5c305983a239206490a9e4e415d0101ae8a3f76a4435/pyromark-0.9.11-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:eae992b292c128d470780accdbc79327cc2fab7664621ec038941ee256b6b35a", size = 448069, upload-time = "2026-05-24T22:20:37.046Z" }, + { url = "https://files.pythonhosted.org/packages/81/03/84e9c90e802f7000369fc16e9a2b4e3c987c55dcde5c46abe679dd574941/pyromark-0.9.11-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:38e8f90d6b1c36bc50c2de52dc9d717ec344474354b8c24812f3a32fcde35faf", size = 371499, upload-time = "2026-05-24T22:21:27.335Z" }, + { url = "https://files.pythonhosted.org/packages/eb/21/258c057a25f5f91bd3bd6edd9e9da1c809aa143842a07e1b286abe53a4d0/pyromark-0.9.11-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:cc065c1416d66672a6650e30e83e3aace55be981093f566a2bd12ec84d8f7279", size = 363946, upload-time = "2026-05-24T22:21:36.143Z" }, + { url = "https://files.pythonhosted.org/packages/ff/6f/c0fd6da5e483e408e3c7613f421bff98873187f2a942d7dcce212cae89cc/pyromark-0.9.11-cp313-cp313-manylinux_2_28_armv7l.whl", hash = "sha256:d1e44ab25a164b4cd49e907f8340ee8999f02c4f577d7fe21c962b0a41295916", size = 361279, upload-time = "2026-05-24T22:21:33.186Z" }, + { url = "https://files.pythonhosted.org/packages/39/45/6d17775ce47b514f82c70a71709871e69dc95fc786412ef0b344c966772f/pyromark-0.9.11-cp313-cp313-manylinux_2_28_i686.whl", hash = "sha256:e918c8cd291f295db0eead9f909c104016974eb53a5fa334d1c8f69794df6bc3", size = 387458, upload-time = "2026-05-24T22:22:11.295Z" }, + { url = "https://files.pythonhosted.org/packages/bb/08/c33729958990379b966106581e69a3e91c4607cd11fd3197156ff14a5096/pyromark-0.9.11-cp313-cp313-manylinux_2_28_ppc64le.whl", hash = "sha256:1bd71670031b0ee82200ea5bfed9b80d48dee6f21b2263031cd69c0136ebe6b7", size = 410032, upload-time = "2026-05-24T22:18:55.803Z" }, + { url = "https://files.pythonhosted.org/packages/7a/ce/b531fda481d4ee6555ac2469a81ee4ea9026acfc39c2eed066deaa403eeb/pyromark-0.9.11-cp313-cp313-manylinux_2_28_s390x.whl", hash = "sha256:25d4beeb4650d67518116d4c42719015ba523a303a2a8d93421154479af8e4e9", size = 448268, upload-time = "2026-05-24T22:22:16.394Z" }, + { url = "https://files.pythonhosted.org/packages/91/66/fbffc00186361a468e7b88e251e2fe1ae853059d5d3d95fe6e0479bcc607/pyromark-0.9.11-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:d135ce3e269be82a01866c69ff08a46a42d91a4db4f81057ba5828759f7c41b8", size = 371933, upload-time = "2026-05-24T22:20:55.517Z" }, + { url = "https://files.pythonhosted.org/packages/56/d8/0906e6a18e98e6afe3d529c6c7074818f65e67ee230507e2f025512ded38/pyromark-0.9.11-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:7c9eb49d9ad54b112c9e7402a158fcb66ac6d9d51eea3378ad2487841fb4b4cb", size = 372193, upload-time = "2026-05-24T22:22:41.918Z" }, + { url = "https://files.pythonhosted.org/packages/6f/ab/98271148db4bead714b55a8dc149d0b1c2d8e10945c8115a7d15ab139237/pyromark-0.9.11-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:6abe525ac95fcc6e5fa7430b171d81aa315f3201fbe635c06a2a2037642bc30f", size = 540649, upload-time = "2026-05-24T22:21:20.838Z" }, + { url = "https://files.pythonhosted.org/packages/9e/89/d29848725bd74bd6b86c1392e8b8b471fb4393e8b3946a2c6a4a2b364f5d/pyromark-0.9.11-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:6780144b731ddf5f4306e40d422c2e4c16d0223cbacce1d95b9f10bc2fe844c1", size = 638026, upload-time = "2026-05-24T22:20:18.993Z" }, + { url = "https://files.pythonhosted.org/packages/f3/20/77afe176a717a88b00b10a9a88b1d49872f0922631bf1aca55dc590c4481/pyromark-0.9.11-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:427cf7a4c79ea2bf00fcb1ff264ec57c1f2909c601ece4664a74e96b3c79dcf6", size = 604908, upload-time = "2026-05-24T22:22:04.804Z" }, + { url = "https://files.pythonhosted.org/packages/82/bd/13e1976e2604f49dd31b68c140eee8b5b537df4d65111e6347932f016b12/pyromark-0.9.11-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:46ae4c84fd5c7a9c662b4b66897924f11e28ff0c51b0248d025d5ea9cec3f3fa", size = 540573, upload-time = "2026-05-24T22:18:57.137Z" }, + { url = "https://files.pythonhosted.org/packages/41/19/7f9323efa7c69f4982f7132d3b76376457fb9b89099824e6b860fe1aefb6/pyromark-0.9.11-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:e9f3ca3d61a9af62b83012f3e0f04e8e768d71b61c1cff26c16b13e054d15778", size = 546207, upload-time = "2026-05-24T22:22:31.941Z" }, + { url = "https://files.pythonhosted.org/packages/87/d1/2286dc71cf3a12e8bbaa2311387042b5fc479c4afbf67ac7bdc50f7f3f91/pyromark-0.9.11-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:4d3c29e28bad3594acd1892c2919236060acda0e1dd9f77eeff9765d90b9b496", size = 591015, upload-time = "2026-05-24T22:21:10.837Z" }, + { url = "https://files.pythonhosted.org/packages/12/98/e35366fe410e289e56e133bfa7170d73af7a0a5ea77c2bebf8893fe14a73/pyromark-0.9.11-cp313-cp313-win32.whl", hash = "sha256:7ee0dc79569d35aa22d3d9492525aa47ab6fc0c283ffed449320f3f75f3fb5de", size = 260868, upload-time = "2026-05-24T22:20:20.588Z" }, + { url = "https://files.pythonhosted.org/packages/23/f3/bf3b72ab66f7b2445182b7c33a50a05f27624052d9febdb6e083343c857a/pyromark-0.9.11-cp313-cp313-win_amd64.whl", hash = "sha256:da3080c84b4688c8a755bb4e42a6ce15f04a9eb22b5b173121ae4fcd2cb7e0c8", size = 273423, upload-time = "2026-05-24T22:21:16.561Z" }, + { url = "https://files.pythonhosted.org/packages/df/cc/1744b5b51148846886a8939b60409f810490b4ac140e2f2dbd4043f56225/pyromark-0.9.11-cp313-cp313-win_arm64.whl", hash = "sha256:8267a2a233669709d399232e4bb6139779cba9000f8c50845bdee6f220f28be2", size = 258978, upload-time = "2026-05-24T22:19:32.78Z" }, + { url = "https://files.pythonhosted.org/packages/5e/05/022efc4937b3ba579c86d954a396ae11b28efcad2fa30209a35676af91d2/pyromark-0.9.11-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:87703294cf6d13a2e89fe9d88ee0ec4deae5ba5a0b25dacd6a5dafff32055325", size = 345771, upload-time = "2026-05-24T22:19:22.497Z" }, + { url = "https://files.pythonhosted.org/packages/0a/37/a2753f8ce59bf64f856ffd76e5bcfe7f0a214c2796703a7bd279796d9fe2/pyromark-0.9.11-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:b9edca0c67d67f8a9b6ca1a792088d53f7179d5005ae6977f7185ea6524f5640", size = 328661, upload-time = "2026-05-24T22:21:30.353Z" }, + { url = "https://files.pythonhosted.org/packages/7f/cd/9db61f94e09c5abdb376e781d4c81e01f8b14dd15d0852c2712aabb8aa8e/pyromark-0.9.11-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1a83b9633de7ee66512b9d1c986c1fa58373caa177cc49e9f821dc6d4d8fc5d3", size = 361530, upload-time = "2026-05-24T22:21:08.054Z" }, + { url = "https://files.pythonhosted.org/packages/17/17/d534f50f05ecee8a636e30ac21e7e1fb21fa44d2f286880b673539c7d2f5/pyromark-0.9.11-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:dd85aa34a004e550ba0aac32de202e6dff65130e4c84b0f87568f6634e7910db", size = 359161, upload-time = "2026-05-24T22:19:38.709Z" }, + { url = "https://files.pythonhosted.org/packages/43/ef/8418c98a9a86c830ef8bb37450bb6eb9690967570e2ac4707766ff82000f/pyromark-0.9.11-cp313-cp313t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f60124c8ed7ed48c7cc9fa7d1291f3af454d0c88c6a85e948710587c2f7a3723", size = 385356, upload-time = "2026-05-24T22:22:38.996Z" }, + { url = "https://files.pythonhosted.org/packages/a2/2d/a04415b1989a91c8164cd8c7d0ad2dc24db6b0954bd79ba31f2b62cddfab/pyromark-0.9.11-cp313-cp313t-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:1a45d6de18f952880043cba718865963e2679e4fec4ac360666a0d6ae9fe9107", size = 416452, upload-time = "2026-05-24T22:19:56.632Z" }, + { url = "https://files.pythonhosted.org/packages/29/43/a1cda96871a9d7f15a09c5de5e75233cc6f389d5bf7c639080717f9250f6/pyromark-0.9.11-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:372c89600ec913f332f716944c66c3b15fe37dbfb7323ac1ee8b561a40393a35", size = 407084, upload-time = "2026-05-24T22:22:53.315Z" }, + { url = "https://files.pythonhosted.org/packages/23/36/49e04c5bfa8d0a8f09b028aaf93f127598c0dd357bc0a022e33bf259bffc/pyromark-0.9.11-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d14851cfacbbce1ee49ecfa923ea76183dd7f2dd1310b3d2d622e0e335606d8c", size = 445900, upload-time = "2026-05-24T22:21:42.219Z" }, + { url = "https://files.pythonhosted.org/packages/75/d4/92f71c717e4b2a404f0cb1d74462c9d06faf4c2cab3e4cb7eb38a3468bf0/pyromark-0.9.11-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:28a7896b4a5a5eda6f180b19af1e8aa43f6b66337660c306d46ca70cb999a073", size = 369863, upload-time = "2026-05-24T22:19:03.927Z" }, + { url = "https://files.pythonhosted.org/packages/53/ad/17105282b2dc324df9b36d2af124dc2b8f9079fd00cd0b240aef3c6d0b18/pyromark-0.9.11-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:332e21e098edbddf2c82bc9e3e25acd315b95d5a894374539e97b18b548c4685", size = 362208, upload-time = "2026-05-24T22:20:43.158Z" }, + { url = "https://files.pythonhosted.org/packages/97/a8/f1d706f3887dde8413bae2751a1b1e18c49d0f75b6d7a17d577248c00afb/pyromark-0.9.11-cp313-cp313t-manylinux_2_28_armv7l.whl", hash = "sha256:189862413cd50ec06e0639150df070b50fcb7cf9bcac743561bdf9479895a88e", size = 359096, upload-time = "2026-05-24T22:20:24.962Z" }, + { url = "https://files.pythonhosted.org/packages/77/77/1db1054e42148a79de81a969b2001690a9e8f0fa3f40c724bc6138d6e09c/pyromark-0.9.11-cp313-cp313t-manylinux_2_28_i686.whl", hash = "sha256:d7c4d5fbb586cc73c6e63041ac74d42f49165d09c2c60e1f36e0ac3b1f8d76d8", size = 385698, upload-time = "2026-05-24T22:22:13.037Z" }, + { url = "https://files.pythonhosted.org/packages/7b/27/2821d6fb8d9577eb8d540132d28d96a1c3fea347388ea5b9090ebef40c52/pyromark-0.9.11-cp313-cp313t-manylinux_2_28_ppc64le.whl", hash = "sha256:ca4d75f7c63e7c0c686f1cdf0946eea83a6e2b11a22530fccc5eb50e9915717a", size = 407290, upload-time = "2026-05-24T22:21:19.506Z" }, + { url = "https://files.pythonhosted.org/packages/5d/69/8acdfcd48229b11919dbd8e6b4d8e5735a5c415e043e61af695c3a238b1f/pyromark-0.9.11-cp313-cp313t-manylinux_2_28_s390x.whl", hash = "sha256:affcddd172b8a636cc7d5049f2e179a4e338fff419bcb7167980bf1a1f44d46a", size = 446109, upload-time = "2026-05-24T22:19:02.891Z" }, + { url = "https://files.pythonhosted.org/packages/03/08/e87518ddc80759bf7d88d1ff0dc2d8a61994304eb3187d61302fa4746a81/pyromark-0.9.11-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:b8e5f2bdefa1ddff59886ff6f729811deb367e9305bc25abd033139984bebea6", size = 370151, upload-time = "2026-05-24T22:20:48.608Z" }, + { url = "https://files.pythonhosted.org/packages/2a/76/f1f6a9983e8f43cc0fb0ad1dee66dd0ea608df59d2e57ce063e9c46077f1/pyromark-0.9.11-cp313-cp313t-manylinux_2_31_riscv64.whl", hash = "sha256:f577f9311f43175f04e6bf0756ca67cfafb9215c70d572608c8fab1e5c1f0836", size = 370623, upload-time = "2026-05-24T22:22:56.409Z" }, + { url = "https://files.pythonhosted.org/packages/72/f5/8f78d68e70b042e07413bd06399eb85e169fd7226148f91a51083b05f1df/pyromark-0.9.11-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:c9d9cbf126f33a0eb53d3780ee5f5ad76015fe359e3b4500b801bda37bd0f919", size = 539216, upload-time = "2026-05-24T22:20:34.078Z" }, + { url = "https://files.pythonhosted.org/packages/19/82/7739f8c579f36556679b21766a533b29fe6e9c8f25d17ee6940335274714/pyromark-0.9.11-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:8989b7d49568fad39df65698170323ba4bf973ed51a251c0791bb1fde80283b8", size = 636005, upload-time = "2026-05-24T22:19:15.359Z" }, + { url = "https://files.pythonhosted.org/packages/12/30/b58a7671a886446fc2d626be753c3fb26fd2dabadee8f63378f37d62792f/pyromark-0.9.11-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:830028b3d644f7e18c84b8af8aafc7468dc9092585787cdc0b340fc7a40f1306", size = 603104, upload-time = "2026-05-24T22:21:40.475Z" }, + { url = "https://files.pythonhosted.org/packages/92/fc/1801bb5e0045db924b3b0587fe4319640e72319eaf8b803acc58c32d896a/pyromark-0.9.11-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:75dc519a7b5d6182a58d5c8b35e2592fda70c9cbbadcbfa1506a60903026b8c6", size = 538467, upload-time = "2026-05-24T22:20:47.078Z" }, + { url = "https://files.pythonhosted.org/packages/9d/b6/4ed750bfefe1c657bd97b7433b2c333703a55c9db3be9b033e0dc03bf50e/pyromark-0.9.11-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:72752e772de34db218aad1d03653c917292565c2788aa77d22e4ba53339bb372", size = 544209, upload-time = "2026-05-24T22:19:00.585Z" }, + { url = "https://files.pythonhosted.org/packages/22/6c/9d201a7657553049ddf17662f120367252b5bc05a20effdfc955e1f9ec82/pyromark-0.9.11-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8965493591b2c1c7100bfe7f3d9d76ac847c6e363121cb78b3aca90de5922769", size = 589394, upload-time = "2026-05-24T22:21:45.517Z" }, + { url = "https://files.pythonhosted.org/packages/a5/a1/a2509165d9f8439dad26b73f6b35b4041a6d45b033b62ab66e6579ad2e08/pyromark-0.9.11-cp313-cp313t-win32.whl", hash = "sha256:84504c69141ac64385e2ad74b3774dc46124cc3574ed838793afe764e8afa738", size = 260541, upload-time = "2026-05-24T22:18:43.304Z" }, + { url = "https://files.pythonhosted.org/packages/ec/35/32dc5ef5a7e77b9fbfd2991d01090b6b2b7d91c7a558a9550e3fd18ddf25/pyromark-0.9.11-cp313-cp313t-win_amd64.whl", hash = "sha256:a64a74e5e3d6d69dbc65ff38404cce961ccdfebdc59f195e70c809ad4e2f207b", size = 273244, upload-time = "2026-05-24T22:21:25.741Z" }, + { url = "https://files.pythonhosted.org/packages/8f/ae/c360c783f6717cc1397027c908dcee6346cc1612efe1d8cdffb8defdb535/pyromark-0.9.11-cp313-cp313t-win_arm64.whl", hash = "sha256:2badd596491959bf9db3677a2d2214554f8834cf4375e37025567c7d676b2f0a", size = 257893, upload-time = "2026-05-24T22:22:59.515Z" }, + { url = "https://files.pythonhosted.org/packages/fc/a4/e6e0995fa79d3fb05a71f1aa2c587e988247c04e58c3d905c309c05b6b0a/pyromark-0.9.11-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:4ab000c6507b3b7e5c44f86d6ac9087f9fe2037bf76c18ea94fe9b0fc00bfeb1", size = 347141, upload-time = "2026-05-24T22:22:58.002Z" }, + { url = "https://files.pythonhosted.org/packages/33/b1/35229efeb45136581b5d2e03220cfa969b87d32df520d2318ebad84a6c67/pyromark-0.9.11-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:edacbabfced967c2752e826af36b0f335163c212154f682de40c25dfb9634e10", size = 329689, upload-time = "2026-05-24T22:21:52.142Z" }, + { url = "https://files.pythonhosted.org/packages/9d/05/fe95ecf4a2e29654e93ce44ae48a63efee08ac667cd570094dad6a26de9b/pyromark-0.9.11-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2044ccd2ceff6b3bdd1b2049359e80f85d50e10bca74cbf6e0ec15702e88fbb0", size = 362984, upload-time = "2026-05-24T22:23:06.392Z" }, + { url = "https://files.pythonhosted.org/packages/2f/02/eea6a4a86468242990175e5c42e5870cd5b809e85f5ce79d5cb3f8db841b/pyromark-0.9.11-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:7b58b29fd60ff4e96f902cb77a3a1d23845770174f502c3287f76c7ee7f7d088", size = 361240, upload-time = "2026-05-24T22:19:30.21Z" }, + { url = "https://files.pythonhosted.org/packages/df/1b/13906bcdbd05f10746ac0e87a6c6391728b2a4c3bda8995c8d8a1b7b021a/pyromark-0.9.11-cp314-cp314-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:77212a23f21bd4ac2ba935fa7a81e237e90c3157f738bfb37884e939809d570a", size = 387018, upload-time = "2026-05-24T22:19:12.272Z" }, + { url = "https://files.pythonhosted.org/packages/91/07/c2c3ecc3a387c9e880b9a47db6cc68021fd4c2456c6eef4cc6132b49fdaf/pyromark-0.9.11-cp314-cp314-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:e04fba581cfc1304a65447e85990ed0bd3c27d3ee55de93397f6fc3d01f9ab57", size = 417669, upload-time = "2026-05-24T22:20:58.161Z" }, + { url = "https://files.pythonhosted.org/packages/72/45/5c6bc54688c77bc22f5e9918601a13007b5dbef45cc89b5e582657d25368/pyromark-0.9.11-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:6143a312a783144979c69a6ec180f4d148fc41d063a7d2cd938cc7edfb5aac0d", size = 409519, upload-time = "2026-05-24T22:19:58.001Z" }, + { url = "https://files.pythonhosted.org/packages/4b/b2/8b572ee152c0d6b461132e4c408cdc7833fb1ab6f18721622ccbecf2f357/pyromark-0.9.11-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5c751e1a269007ccefe177316b08dcfc09efc7debc549450d98a9563878c7b8c", size = 448060, upload-time = "2026-05-24T22:20:28.137Z" }, + { url = "https://files.pythonhosted.org/packages/77/33/ed6eaccf2c692da8987f5efa70c4d70b1750fba2102ab2a6b83e107ea80d/pyromark-0.9.11-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5d6b38603efb313ded143a3836082c0a2cdf3b92dc314795ed43d72c0d818304", size = 371338, upload-time = "2026-05-24T22:18:38.359Z" }, + { url = "https://files.pythonhosted.org/packages/86/2d/91dc78be20a481f727da6887f3477a4b5f7388a768966a2f3524870d5bc8/pyromark-0.9.11-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:faf026af5c90c73f18d05ab267b073aba8a145c954af5894fbb42d7287d742ff", size = 363598, upload-time = "2026-05-24T22:19:44.354Z" }, + { url = "https://files.pythonhosted.org/packages/e5/a9/963f92bd732c1fb889896730d6ae54ea02ed95bec9207f48741467330902/pyromark-0.9.11-cp314-cp314-manylinux_2_28_armv7l.whl", hash = "sha256:83564a870f4a081e90e13deb5ab483e2c48130d3ef7e34afda2d74db8111ce02", size = 361231, upload-time = "2026-05-24T22:19:25.219Z" }, + { url = "https://files.pythonhosted.org/packages/c3/cf/3a3ab3915314062762730baf13a6085fef69df55b8bef98c93b1af22bb5b/pyromark-0.9.11-cp314-cp314-manylinux_2_28_i686.whl", hash = "sha256:337f0e57aeeda6993ed4d2695437ca6c2821a60df794b0de89fa22950f7d5aa5", size = 387300, upload-time = "2026-05-24T22:22:26.876Z" }, + { url = "https://files.pythonhosted.org/packages/99/dc/4c497732a5c9f5e7abff266b3ac9b85d50513ec6c685ad3d6323b283ff93/pyromark-0.9.11-cp314-cp314-manylinux_2_28_ppc64le.whl", hash = "sha256:9739c4ac35926bec56abd112f208845e94cd440f757a3dacfb11dbd29ec9cf1e", size = 409636, upload-time = "2026-05-24T22:19:17.832Z" }, + { url = "https://files.pythonhosted.org/packages/4a/a6/05a9b913f3393082d90a5db1e928c6790a374b98c1c7fede32aa25f577ba/pyromark-0.9.11-cp314-cp314-manylinux_2_28_s390x.whl", hash = "sha256:ff024a8d19d792d1728c9bc1f03f697fb4dbe14ae02854b23d74909c42c134d1", size = 447742, upload-time = "2026-05-24T22:19:23.755Z" }, + { url = "https://files.pythonhosted.org/packages/1a/43/011b930f0c5d0288aba1b9de6a947f2e71e2bdfc90b56ae70d6fdd81c572/pyromark-0.9.11-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:2d88478728353f95582a594f734b6a2c85ba31c3dc6af7f9122055ed8d3e88b5", size = 371594, upload-time = "2026-05-24T22:21:37.504Z" }, + { url = "https://files.pythonhosted.org/packages/00/80/049d0a3c74cd1c436824ced55d136c339496337ae5a68614d31f1c177042/pyromark-0.9.11-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:26d52ebce02df169d2de1cf17b8db07eb4f2c5a543c651f6e15caf8ae6ecd783", size = 372069, upload-time = "2026-05-24T22:21:22.936Z" }, + { url = "https://files.pythonhosted.org/packages/21/f6/d7197ea8aaf9bee0d1ef88c9e4bd48bb59560fde767f0003b0adcfc5014c/pyromark-0.9.11-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:bb789d2d1f13d0efabf9ae2ef85d2fb483440653f54260198477a32a66a0fae0", size = 540529, upload-time = "2026-05-24T22:19:33.992Z" }, + { url = "https://files.pythonhosted.org/packages/b2/47/b403cf8f75aa5a854352ae1b8eb98923ac4089f8d1fcb77771921e72512a/pyromark-0.9.11-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:0837914b0c335ba2ea7285c4a44535506bc68b0dc08ebc54bfe1a9b016b7fbde", size = 637913, upload-time = "2026-05-24T22:19:41.757Z" }, + { url = "https://files.pythonhosted.org/packages/a8/37/6ba2aa1364c0b8c46999dfbec8648df22043dbb416500fa95ab3b410abdd/pyromark-0.9.11-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:3923bfa9cfdde43b78c3c7abdedd353b17c245cc4a7de9147e451cfe093c21c7", size = 604815, upload-time = "2026-05-24T22:20:32.717Z" }, + { url = "https://files.pythonhosted.org/packages/1c/b2/0b66636e267a5a469854fcb99b868c8d5f822893919d6bab5b0e9beb626e/pyromark-0.9.11-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:49ef7e711e0b62259878b38e7dd2eb6823cce1da237728e9c2117dfef667df93", size = 540311, upload-time = "2026-05-24T22:19:26.66Z" }, + { url = "https://files.pythonhosted.org/packages/cc/86/a04b45c16a009e3027be2c2d107ed1d09015637718860430d08633448630/pyromark-0.9.11-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:634cb7a2cd473d93e1d244d7527f3e215b2c1670fd664d1532e29b3d043f98b0", size = 546082, upload-time = "2026-05-24T22:18:36.206Z" }, + { url = "https://files.pythonhosted.org/packages/3c/6e/64a8ce53f9abd4f8c7ef1c892efe52dd32440b149b6d51c68761b3773cf4/pyromark-0.9.11-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:cad96a7b37a386bbcbac01d350cf6d55b78f69a4dbff812d895a6e584fe35e1f", size = 590821, upload-time = "2026-05-24T22:22:25.246Z" }, + { url = "https://files.pythonhosted.org/packages/79/4c/d8d2267317a8edfd29a25395ca11f0204ec59fb3ffbd75599c7fa7ef1507/pyromark-0.9.11-cp314-cp314-win32.whl", hash = "sha256:fcc29565e1d251d588d7f91dae83cb32770d758f81212a680ceea555aaef3a99", size = 260937, upload-time = "2026-05-24T22:19:16.802Z" }, + { url = "https://files.pythonhosted.org/packages/2a/97/0b5082e24276909df1d70299ad3c1b6353bbfcfae3c4887303ca0248fb20/pyromark-0.9.11-cp314-cp314-win_amd64.whl", hash = "sha256:0c13f278d03b0f4b321fd78fef25d3b72e5d19bed8e3464c71318265f0d2f04d", size = 273222, upload-time = "2026-05-24T22:18:48.08Z" }, + { url = "https://files.pythonhosted.org/packages/7b/1a/20f3178384dfdc0d126f9d91cc05b7970f83867f5eb78f8a77a06188d282/pyromark-0.9.11-cp314-cp314-win_arm64.whl", hash = "sha256:0b90ab42108f479d6de4508e65edab4b1c904217ed77919eacb2bbe0817fa3fb", size = 258753, upload-time = "2026-05-24T22:21:53.429Z" }, + { url = "https://files.pythonhosted.org/packages/d9/a1/aeff3c7ba70aeff34c6b28bf012575b33490c42bac99f2b69f5a4f9fde5b/pyromark-0.9.11-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:6d21c4c7cd823a39d79d6140b4f2b23e1d3fac53e202d729781e349bb7ec85cf", size = 345678, upload-time = "2026-05-24T22:19:35.354Z" }, + { url = "https://files.pythonhosted.org/packages/4e/b4/c570e14e75ad752f1575065657fd9c300b1c2ae0fd6c93d2a38ab153fa1a/pyromark-0.9.11-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:15e989f24275834873bc6968b77dd214497fcc4d6f4f562e2703469b453ed99f", size = 328498, upload-time = "2026-05-24T22:19:45.533Z" }, + { url = "https://files.pythonhosted.org/packages/0b/ea/5f07fc67f4f0af5a6c0ec2f2f2557bc426ef55f509c00a1cc0c2ff5ed241/pyromark-0.9.11-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:581dd4a29754a7fa007f3ea67142fbe59e7a2f86f0d4402ce8bc6af66f82cad0", size = 361320, upload-time = "2026-05-24T22:20:59.751Z" }, + { url = "https://files.pythonhosted.org/packages/0b/79/ba6de85c6e2f952e83624ca162a8a51740175750dc02eb17b0305285d271/pyromark-0.9.11-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e036669d6990258331a7bc303cf5dc5c001783f15d53b769a34ea7ba7ff5d1dc", size = 359148, upload-time = "2026-05-24T22:22:40.48Z" }, + { url = "https://files.pythonhosted.org/packages/3c/8c/39ad5a4fad0a46f4b0fbb76c33703ea38a903858b36c528c14c9d7512e46/pyromark-0.9.11-cp314-cp314t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:439181484e0eb660b68962fcd5868adc92db32ce1503948546ac9e96fc26b287", size = 385059, upload-time = "2026-05-24T22:19:40.149Z" }, + { url = "https://files.pythonhosted.org/packages/45/e2/65cc812aece6cc4880f9909ae71eb2e14f655eb116418d82b97c5d60ebd0/pyromark-0.9.11-cp314-cp314t-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:051f9af79107b024488a423c0ac1563eb55f946700dd10ffe72db1a6e2bc675b", size = 416203, upload-time = "2026-05-24T22:20:35.711Z" }, + { url = "https://files.pythonhosted.org/packages/1e/3b/cd470a8c9188c8159729c9ce7de1c47dabbf35ae5cbfcd77fc9fe2b27e1f/pyromark-0.9.11-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3ea9568d17b1ef94d1e951d1e8cb351614ea078dd8bb46bca5d5ab6541266583", size = 407225, upload-time = "2026-05-24T22:18:35.169Z" }, + { url = "https://files.pythonhosted.org/packages/5d/1f/35b911f1b66401da26ed46b8b4fce99b04ee1382532a7188f30b3b2ed44b/pyromark-0.9.11-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d9745842b06509c4f425fc31cbb97a80c1754135f396672cd142a84c216a1668", size = 445917, upload-time = "2026-05-24T22:20:44.409Z" }, + { url = "https://files.pythonhosted.org/packages/34/da/0e9d205b6fc4947c8280c89d9816dd68918a877096d583570682e1290676/pyromark-0.9.11-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6efca2fb6022011b501454b8a09ac280db94dbd840fe0f83bd4d5688dc044944", size = 369625, upload-time = "2026-05-24T22:18:49.027Z" }, + { url = "https://files.pythonhosted.org/packages/ac/d4/f4fd5c9db786345eb1d0a84798fac4601e34dbff21f4852bd522a192407a/pyromark-0.9.11-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:fe125b76e53994adc8bb14a59f582a912adee2d7f3614875729dd1e0fe23187b", size = 362002, upload-time = "2026-05-24T22:21:02.274Z" }, + { url = "https://files.pythonhosted.org/packages/28/11/cf271b215d98531a8c24ecce7baae9003d3b5a726ef828b72790921fbc89/pyromark-0.9.11-cp314-cp314t-manylinux_2_28_armv7l.whl", hash = "sha256:48adb71a94530f29cb0f0de2b9aa6ca039d81345bec77d6059066352609bcdb1", size = 359172, upload-time = "2026-05-24T22:21:18.123Z" }, + { url = "https://files.pythonhosted.org/packages/4a/57/a5705bdd1e3687df643aba383a4971d2aca240080d997d746eecaed9b382/pyromark-0.9.11-cp314-cp314t-manylinux_2_28_i686.whl", hash = "sha256:0ed5ca997d906eb954235c61aef3b95ec5193cf54916fcda16a46e76814b9aed", size = 385505, upload-time = "2026-05-24T22:20:12.488Z" }, + { url = "https://files.pythonhosted.org/packages/48/23/0aa20e8c9e974dd93057ec3d4283e945ed181fd917cbafe2f685c475d290/pyromark-0.9.11-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:174b20eab1bef372735c8eb6398f66eec93b5e7e0316a0952730785b83750011", size = 407332, upload-time = "2026-05-24T22:19:21.414Z" }, + { url = "https://files.pythonhosted.org/packages/b6/94/ed1dedaae23143173dc29ea1349409568b929a133c8474a9b1e195b5f5f8/pyromark-0.9.11-cp314-cp314t-manylinux_2_28_s390x.whl", hash = "sha256:9e7af952342e7394bee512cea64aebc8cd8e1e3ac989cddbc47128e3e411e8b5", size = 445708, upload-time = "2026-05-24T22:21:01.024Z" }, + { url = "https://files.pythonhosted.org/packages/5f/f6/2e146609876e90eac22e75d0391b23e30eb6ddc64f5463edf5422e1429eb/pyromark-0.9.11-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:63589190815986e5c6f5bf66aa336bcaa9ee5e90796dff66b480001ca2eda12b", size = 369988, upload-time = "2026-05-24T22:20:14.844Z" }, + { url = "https://files.pythonhosted.org/packages/34/3c/c53ebcd8d0bb3e605155ce90474a34b8384ca6abcb0d711ddf25e4852806/pyromark-0.9.11-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:b9ac6dc765604a47b7268e78910344980cc2fd0a137c02cf217e4e1072fc355d", size = 370582, upload-time = "2026-05-24T22:21:47.277Z" }, + { url = "https://files.pythonhosted.org/packages/6a/72/313075d5fc4a78a59ec5259ed75ba3274a7c11786ab3080d801b9ab8bad4/pyromark-0.9.11-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:f1459db160281d6e6cf1ac234207bcc781c46ea4de6c1dc2c467023944950d13", size = 539038, upload-time = "2026-05-24T22:19:50.302Z" }, + { url = "https://files.pythonhosted.org/packages/77/4b/0674c8bff3cc038eb0b33da13ae42129310da078410ec78fa6608ca0b9f6/pyromark-0.9.11-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:156cb4db0911fd6037768e7d31b3571f2309955c9fa29a193d24ce4f26b0e4ed", size = 635924, upload-time = "2026-05-24T22:21:13.907Z" }, + { url = "https://files.pythonhosted.org/packages/4f/1b/03271177d632ef73132110215261886869a8cfdc1028fd520a78029596aa/pyromark-0.9.11-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:599fd85d8be2ab6d602b36b3f52b92a959738344e6c1ce569457aa3dae96e8f5", size = 603053, upload-time = "2026-05-24T22:22:00.338Z" }, + { url = "https://files.pythonhosted.org/packages/f0/62/b10c4aa0903c49cc6b227df3bc18322dfeb4f6d6594e02986284e98a2069/pyromark-0.9.11-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:a9797732d5bb2555474f0fbfdb6e29eb5431159f2abbfd8a52b2ed5755ab7206", size = 538467, upload-time = "2026-05-24T22:21:50.464Z" }, + { url = "https://files.pythonhosted.org/packages/c5/7f/d04c700b7b0b24cc54c6692222f0fe29efed6fc92c655a7c026847923218/pyromark-0.9.11-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:18102dc1d634f2094f5b8723f51049edf7e6c335110640bdc745c7a2126b21ac", size = 544009, upload-time = "2026-05-24T22:20:04.458Z" }, + { url = "https://files.pythonhosted.org/packages/1a/8e/d0abe1f991286b86e1c4074207675913a7c36fb7bcf9d56fb60711ab92b1/pyromark-0.9.11-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5d17ca83164862fc478f26b0ae643628f4a30d6971dcd0663dc13a73fa5c7151", size = 589315, upload-time = "2026-05-24T22:20:10.994Z" }, + { url = "https://files.pythonhosted.org/packages/b7/58/1f5a33d9854a4614ac555de4d3c12294005f12e0d2621f240a8310e2e390/pyromark-0.9.11-cp314-cp314t-win32.whl", hash = "sha256:134de0516db2ea8bb570a608383c4be443f48ec2a185b58935f053369971d94d", size = 260520, upload-time = "2026-05-24T22:23:08.187Z" }, + { url = "https://files.pythonhosted.org/packages/e9/99/59f011062c40670a98136180e6def19cb3a3f4e283ac3936520e8c091ff7/pyromark-0.9.11-cp314-cp314t-win_amd64.whl", hash = "sha256:d710e0577de792280df4b5fc05a90cc615b2226980a2db3ba0f3b428d78c09c1", size = 273172, upload-time = "2026-05-24T22:21:09.624Z" }, + { url = "https://files.pythonhosted.org/packages/af/57/da0a369bfb2910bf905b5a3687c7f4d7c4bd263a26b1d1d1055e67a7ad2b/pyromark-0.9.11-cp314-cp314t-win_arm64.whl", hash = "sha256:3c93e10dc4dc751e363494c75bf98ff3fbfdf2c5753bf49c04cc2339e512258f", size = 257990, upload-time = "2026-05-24T22:22:30.145Z" }, +] + [[package]] name = "pytest" version = "7.4.4" @@ -1890,6 +2028,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e7/9c/0e6afc12c269578be5c0c1c9f4b49a8d32770a080260c333ac04cc1c832d/soupsieve-2.7-py3-none-any.whl", hash = "sha256:6e60cc5c1ffaf1cebcc12e8188320b72071e922c2e897f737cadce79ad5d30c4", size = 36677, upload-time = "2025-04-20T18:50:07.196Z" }, ] +[[package]] +name = "telegramify-markdown" +version = "1.1.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pyromark" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/89/db/198bda5ba14b83b714fe5b632992684fac8fa2c07bbc7b14508e8f67e9a9/telegramify_markdown-1.1.5.tar.gz", hash = "sha256:56da17648849f86a351eb712eef4461f8c11aecc7fd27a293f5d9a0a6333b22d", size = 61782, upload-time = "2026-05-10T08:49:40.136Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ec/3e/5a68e10227008a820bad5e5b75cc71cbde03db1c4093d6026279463ccb65/telegramify_markdown-1.1.5-py3-none-any.whl", hash = "sha256:9742d56993be3db2cf9c2a61cbb257f8b34b6013a9ff27af5a569509df6673c0", size = 43329, upload-time = "2026-05-10T08:49:38.847Z" }, +] + [[package]] name = "text-unidecode" version = "1.3" From d0364aecc0b464f8aaccff613fc0c6c9772e3a67 Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Sat, 30 May 2026 20:48:38 -0500 Subject: [PATCH 11/12] refactor TurnExecutor into a thin turn driver MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per PR review on sdk/turn/_executor.py: TurnExecutor was orchestrating setup that callers already had context for (skill loading, persistence wiring, first-turn hooks, system-prompt building). Push those out to the caller; Conversation grows the per-conversation state that actually needs to survive across turns. - Conversation: add agent_state field; TYPE_CHECKING import keeps it a dependency-free leaf. - AgentState.create(skill_names=…): async factory used by every callsite to assemble (core tools + named skills), log+skip on missing. - TurnExecutor.execute(): kwargs slim to conversation/agent/user_content + a few span-metadata params. Raises if conversation.agent_state is None. Stops loading skills, saving events, saving skills, firing on_new_conversation, and evaluating build_system_prompt. - TurnPersistence Protocol moves to sdk/turn/_persistence.py; SystemPromptBuilder dropped (caller composes the instruction directly). - Callers updated: - server/message_handler.py and channels/telegram/_channel.py hydrate agent_state on cache miss, prefix the agent instruction with memory_prompt_block(), then save events / save skills / fire on_new_conversation themselves after the turn. - tasks/_executor.py: _build_agent returns (Agent, AgentState); state attaches to the Conversation it constructs. - sdk/tools/_spawn_agent.py: attaches the already-built state to the spawned sub-agent's Conversation. Co-Authored-By: Claude Opus 4.7 (1M context) --- channels/telegram/_channel.py | 45 ++- sdk/__init__.py | 4 +- sdk/skills/agent_state.py | 25 +- sdk/tools/_spawn_agent.py | 2 +- sdk/turn/_conversation.py | 14 + sdk/turn/_executor.py | 184 ++---------- sdk/turn/_persistence.py | 46 +++ server/message_handler.py | 39 ++- tasks/_executor.py | 16 +- .../sdk/skills/test_agent_state_create.py | 84 ++++++ tests/unit/sdk/turn/test_executor.py | 275 +++--------------- tests/unit/tasks/test_executor.py | 5 +- 12 files changed, 337 insertions(+), 402 deletions(-) create mode 100644 sdk/turn/_persistence.py create mode 100644 tests/unit/sdk/skills/test_agent_state_create.py diff --git a/channels/telegram/_channel.py b/channels/telegram/_channel.py index cf26051b..0b77f09d 100644 --- a/channels/telegram/_channel.py +++ b/channels/telegram/_channel.py @@ -35,6 +35,8 @@ call as broker_call, ) from sdk import TurnExecutor +from sdk.events import AgentEvent +from sdk.skills import AgentState from sdk.turn import is_turn_active, request_stop from tools.memory import forget, memory_prompt_block, remember from tools.virtual_computer.run_bash_cmd import run_bash_cmd @@ -91,6 +93,9 @@ def __init__(self, *, app_sock_path: Path) -> None: self._default_profile_id: str = "computron" self._pull_task: asyncio.Task[None] | None = None self._turn_tasks: set[asyncio.Task[None]] = set() + # Fire-and-forget tasks (e.g. title generation on first turn). Held + # so the GC doesn't reap them; cleared via done_callback. + self._background_tasks: set[asyncio.Task[None]] = set() # Per-chat in-flight turn tracker. Owned by the dispatch loop so the # "one turn at a time" check is race-free — ContextVar-based # ``is_turn_active`` doesn't flip True until the task actually runs, @@ -530,7 +535,16 @@ async def _run_turn(self, chat_id: int, text: str) -> None: ) return + if conversation.agent_state is None: + conversation.agent_state = await AgentState.create( + skill_names=[ + *profile.skills, + *self._persistence.load_skills(conversation.id), + ], + ) + agent = build_agent(profile, tools=[run_bash_cmd, remember, forget]) + agent.instruction = memory_prompt_block() + agent.instruction logger.info( "telegram turn start chat_id=%s conversation_id=%s is_new=%s profile=%s", @@ -545,18 +559,16 @@ async def _run_turn(self, chat_id: int, text: str) -> None: collected_text = "" file_paths: list[str] = [] wrote_started = False + events: list[AgentEvent] = [] try: async for event in self._turn_executor.execute( conversation=conversation, agent=agent, user_content=text, - is_new_conversation=is_new, - preloaded_skills=profile.skills, - persistence=self._persistence, - build_system_prompt=lambda: memory_prompt_block() + agent.instruction, profile_name=profile.name, ): + events.append(event) payload = event.payload ptype = payload.type if ptype == "tool_call" and hasattr(payload, "name"): @@ -597,6 +609,31 @@ async def _run_turn(self, chat_id: int, text: str) -> None: for path in file_paths: await self._send_document(chat_id, path) + # Post-turn persistence. Errors logged, never raised — the user + # already got their reply. + try: + self._persistence.save_events(conversation.id, events) + except Exception: + logger.exception( + "Failed to save agent events for '%s'", conversation.id, + ) + if conversation.agent_state.loaded_skill_names: + try: + self._persistence.save_skills( + conversation.id, + conversation.agent_state.loaded_skill_names, + ) + except Exception: + logger.exception( + "Failed to save loaded skills for '%s'", conversation.id, + ) + if is_new: + task = asyncio.create_task( + self._persistence.on_new_conversation(conversation.id, text), + ) + self._background_tasks.add(task) + task.add_done_callback(self._background_tasks.discard) + logger.info( "telegram turn end chat_id=%s conversation_id=%s text_len=%d files=%d", chat_id, conversation_id, len(collected_text), len(file_paths), diff --git a/sdk/__init__.py b/sdk/__init__.py index f2996412..998393f7 100644 --- a/sdk/__init__.py +++ b/sdk/__init__.py @@ -22,7 +22,8 @@ # own minimal module so importers below the SDK layer can grab it without # pulling in the full executor. from .turn._conversation import Conversation -from .turn._executor import SystemPromptBuilder, TurnExecutor, TurnPersistence +from .turn._executor import TurnExecutor +from .turn._persistence import TurnPersistence __all__ = [ "BudgetGuard", @@ -35,7 +36,6 @@ "LoopDetector", "PersistenceHook", "StopHook", - "SystemPromptBuilder", "TurnExecutor", "TurnPersistence", "default_hooks", diff --git a/sdk/skills/agent_state.py b/sdk/skills/agent_state.py index c6c1d8fb..466546a7 100644 --- a/sdk/skills/agent_state.py +++ b/sdk/skills/agent_state.py @@ -1,11 +1,13 @@ """Tracks base tools and dynamically loaded skills for an agent scope.""" import logging -from collections.abc import Callable +from collections.abc import Callable, Iterable from contextvars import ContextVar from typing import Any -from ._registry import Skill +from sdk.tools._core import get_core_tools + +from ._registry import Skill, get_skill logger = logging.getLogger(__name__) @@ -26,6 +28,25 @@ def __init__(self, base_tools: list[Callable[..., Any]]) -> None: self._base_tools: list[Callable[..., Any]] = list(base_tools) self._skills: dict[str, Skill] = {} + @classmethod + async def create(cls, *, skill_names: Iterable[str] = ()) -> "AgentState": + """Build an ``AgentState`` with the core tool set and named skills loaded. + + Missing skills are logged and skipped — matches the historical + tolerance baked into the turn loop. Callers that want stricter + semantics (raise on missing, surface to the LLM, etc.) should + pre-validate names via ``get_skill`` and then use the regular + constructor + ``add``. + """ + state = cls(await get_core_tools()) + for name in skill_names: + skill = get_skill(name) + if skill is None: + logger.warning("Skill %r not registered, skipping", name) + continue + state.add(skill) + return state + def add(self, skill: Skill) -> None: """Attach a skill to this state. No-op if already attached.""" if skill.name in self._skills: diff --git a/sdk/tools/_spawn_agent.py b/sdk/tools/_spawn_agent.py index f6e4a232..e61cea0e 100644 --- a/sdk/tools/_spawn_agent.py +++ b/sdk/tools/_spawn_agent.py @@ -191,6 +191,7 @@ async def spawn_agent( conversation = Conversation( id=instance_id, history=ConversationHistory(instance_id=instance_id), + agent_state=state, ) accumulated: list[str] = [] @@ -199,7 +200,6 @@ async def spawn_agent( conversation=conversation, agent=agent, user_content=instructions, - preloaded_skills=agent_profile.skills, profile_name=agent_profile.name, sub_agent_name=agent_name, sub_agent_id=short_id, diff --git a/sdk/turn/_conversation.py b/sdk/turn/_conversation.py index 26de5835..e81a1e69 100644 --- a/sdk/turn/_conversation.py +++ b/sdk/turn/_conversation.py @@ -5,14 +5,22 @@ that matters for modules below the SDK in the dependency graph (e.g. the ``conversations`` package's cache) where importing the executor would form a cycle. + +``AgentState`` is referenced via ``TYPE_CHECKING`` for the same reason: +``sdk.skills._registry`` eagerly pulls in every registered skill's tool +modules, which a bare cache miss has no business loading. """ from __future__ import annotations from dataclasses import dataclass +from typing import TYPE_CHECKING from sdk.context._history import ConversationHistory +if TYPE_CHECKING: + from sdk.skills import AgentState + __all__ = ["Conversation"] @@ -23,7 +31,13 @@ class Conversation: Attributes: id: Unique conversation identifier. history: The conversation history. + agent_state: Loaded skills + base tools that survive across turns. + Set by the caller (channels hydrate on cache miss; one-shot + callers like spawn_agent construct fresh). ``None`` means + "not yet populated"; callers should fill it before the first + turn runs. """ id: str history: ConversationHistory + agent_state: AgentState | None = None diff --git a/sdk/turn/_executor.py b/sdk/turn/_executor.py index 02d73b9c..1e79e53d 100644 --- a/sdk/turn/_executor.py +++ b/sdk/turn/_executor.py @@ -1,86 +1,42 @@ -"""High-level turn executor: wraps the agent loop with context management, -skill state, hooks, and optional caller-supplied persistence and prompt -augmentation. - -The caller builds the ``Agent``, creates a ``Conversation``, optionally -provides a ``TurnPersistence`` and a ``SystemPromptBuilder``, then iterates -the yielded events. +"""Single-turn agent driver. + +Given a fully-wired ``Conversation`` (with ``agent_state`` populated) +and an ``Agent``, drives one turn: opens the turn/agent spans, appends +the user message, composes the system prompt with the skill block, +runs the agent loop, and yields events to the caller. + +Per-conversation lifecycle work — hydrating ``agent_state`` from disk, +saving events, flushing skills, firing first-turn hooks — is the +caller's responsibility, not the executor's. See the channels (web / +telegram) and one-shot callers (TaskExecutor, ``spawn_agent``) for the +two shapes that lifecycle takes. """ from __future__ import annotations import asyncio import logging -from collections.abc import AsyncGenerator, Callable, Iterable, Sequence +from collections.abc import AsyncGenerator from contextlib import suppress -from typing import Protocol from agents.types import Agent from sdk.context._manager import ContextManager from sdk.context._strategy import LLMCompactionStrategy -from sdk.events._context import agent_span, get_current_dispatcher +from sdk.events._context import agent_span from sdk.events._models import AgentEvent -from sdk.hooks._agent_event_buffer import AgentEventBufferHook from sdk.hooks._default import default_hooks from sdk.hooks._persistence import PersistenceHook -from sdk.skills import AgentState, get_skill -from sdk.tools._core import get_core_tools from sdk.turn._conversation import Conversation from sdk.turn._execution import run_turn from sdk.turn._turn import StopRequestedError, turn_scope logger = logging.getLogger(__name__) -# Background tasks held to prevent GC; cleared via done callback. -_background_tasks: set[asyncio.Task] = set() - - -class TurnPersistence(Protocol): - """Optional persistence hooks invoked by ``TurnExecutor``. - - Channels that don't want persistence pass ``None`` instead of an - implementation. Implementations are typically thin wrappers over the - application's on-disk store. - """ - - def load_skills(self, conversation_id: str) -> Iterable[str]: - """Return persisted skill names to restore for this conversation.""" - - def save_skills(self, conversation_id: str, skills: Iterable[str]) -> None: - """Persist the set of currently-loaded skill names.""" - - def save_events( - self, - conversation_id: str, - events: list[AgentEvent], - ) -> None: - """Persist agent lifecycle events captured during the turn.""" - - async def on_new_conversation( - self, - conversation_id: str, - first_message: str, - ) -> None: - """Hook fired once when a conversation runs its first turn. - - Typical use: generate and persist a conversation title. - """ - - -SystemPromptBuilder = Callable[[], str] -"""Caller-supplied function returning the base system prompt for this turn. - -Called fresh each turn so the caller can inject up-to-date state (e.g. a -memory block). If absent, the agent's ``instruction`` is used as-is. -""" - class TurnExecutor: - """Executes a single agent turn with setup, hooks, and persistence. + """Drives a single agent turn against a pre-wired ``Conversation``. - Callers build the ``Agent`` themselves and supply optional persistence - and prompt-building injection points. The executor is stateless and - safe to share across conversations. + Stateless and safe to share across conversations. """ async def execute( @@ -89,10 +45,6 @@ async def execute( conversation: Conversation, agent: Agent, user_content: str, - is_new_conversation: bool = False, - preloaded_skills: Sequence[str] = (), - persistence: TurnPersistence | None = None, - build_system_prompt: SystemPromptBuilder | None = None, profile_name: str | None = None, sub_agent_name: str | None = None, sub_agent_id: str | None = None, @@ -101,20 +53,11 @@ async def execute( """Run a single turn and yield events. Args: - conversation: Per-conversation state. - agent: The fully-constructed Agent to run. + conversation: Per-conversation state. ``conversation.agent_state`` + must be populated before calling — callers hydrate it from + disk on cache miss or build it inline for one-shot turns. + agent: The fully-constructed ``Agent`` to run. user_content: The user's message, already augmented if needed. - is_new_conversation: True if this is the conversation's first - turn. Triggers ``persistence.on_new_conversation`` when set; - a no-op without a persistence implementation. - preloaded_skills: Skill names to install before turn start - (e.g. profile-attached skills). - persistence: Optional persistence bundle. ``None`` skips all - persistence calls. - build_system_prompt: Optional callable returning the base system - prompt; re-evaluated each turn so callers can inject - up-to-date state (e.g. a memory block). Falls back to - ``agent.instruction`` when ``None``. profile_name: Optional metadata threaded through ``agent_span``. sub_agent_name: When this turn is a sub-agent invocation, the short uppercase agent name. Forwarded to ``PersistenceHook`` @@ -129,7 +72,15 @@ async def execute( Yields: AgentEvent: Events emitted by the agent during the turn. """ + if conversation.agent_state is None: + msg = ( + f"Conversation {conversation.id!r} has no agent_state; " + "caller must populate it before running a turn." + ) + raise ValueError(msg) + conv_id = conversation.id + agent_state = conversation.agent_state logger.info( "Turn started: conv=%s agent=%s message=%.80s", conv_id, @@ -137,34 +88,6 @@ async def execute( user_content, ) - # Fresh AgentState each turn; pre-load profile skills then restore - # any persisted skills from a previous turn. - agent_state = AgentState(await get_core_tools() + agent.tools) - for skill_name in preloaded_skills: - skill = get_skill(skill_name) - if skill is None: - logger.warning( - "Preloaded skill '%s' not registered; skipping", skill_name, - ) - continue - agent_state.add(skill) - logger.info("Preloaded skill '%s' for conv=%s", skill_name, conv_id) - - if persistence is not None: - for skill_name in persistence.load_skills(conv_id): - if skill_name in agent_state.loaded_skill_names: - continue - skill = get_skill(skill_name) - if skill is None: - logger.warning( - "Persisted skill '%s' for conv=%s not found in registry; skipping", - skill_name, - conv_id, - ) - continue - agent_state.add(skill) - logger.info("Restored skill '%s' for conv=%s", skill_name, conv_id) - # Fresh ContextManager per turn — it borrows the live agent_state so # the token estimate reflects the current tool set, and the strategy # threshold tracks the agent's compaction setting. @@ -195,11 +118,6 @@ async def _producer() -> None: handler=_queue_handler, conversation_id=conv_id, ): - event_buffer = AgentEventBufferHook() - dispatcher = get_current_dispatcher() - if dispatcher: - dispatcher.subscribe(event_buffer.handle_event) - async with agent_span( agent.name, instruction=user_content, @@ -211,16 +129,11 @@ async def _producer() -> None: {"role": "user", "content": user_content}, ) - base_prompt = ( - build_system_prompt() - if build_system_prompt is not None - else agent.instruction - ) skill_prompt = agent_state.build_skill_prompt() full_prompt = ( - f"{base_prompt}\n{skill_prompt}" + f"{agent.instruction}\n{skill_prompt}" if skill_prompt - else base_prompt + else agent.instruction ) conversation.history.set_system_message(full_prompt) @@ -244,43 +157,6 @@ async def _producer() -> None: agent=agent, hooks=hooks, ) - - if persistence is not None and agent_state.loaded_skill_names: - try: - persistence.save_skills( - conv_id, - agent_state.loaded_skill_names, - ) - except Exception: - logger.exception( - "Failed to save loaded skills for '%s'", conv_id, - ) - - # Yield once so synchronous handlers registered via - # call_soon get to run before we read the buffer. - await asyncio.sleep(0) - - if persistence is not None: - buffered_events = event_buffer.get_events() - if buffered_events: - try: - persistence.save_events(conv_id, buffered_events) - logger.info( - "Saved %d agent events for conv=%s", - len(buffered_events), - conv_id, - ) - except Exception: - logger.exception( - "Failed to save agent events for '%s'", conv_id, - ) - - if is_new_conversation and persistence is not None: - task = asyncio.create_task( - persistence.on_new_conversation(conv_id, user_content), - ) - _background_tasks.add(task) - task.add_done_callback(_background_tasks.discard) finally: await queue.put(None) diff --git a/sdk/turn/_persistence.py b/sdk/turn/_persistence.py new file mode 100644 index 00000000..59905fbd --- /dev/null +++ b/sdk/turn/_persistence.py @@ -0,0 +1,46 @@ +"""Protocol for persistence hooks fired by channels around a turn. + +This is the small interface the channel uses to: hydrate per-conversation +skill state on cache miss, flush it after each turn, persist the agent +event stream, and fire a first-turn hook (typically title generation). + +Channels that don't want persistence simply pass ``None`` instead of an +implementation. +""" + +from __future__ import annotations + +from collections.abc import Iterable +from typing import TYPE_CHECKING, Protocol + +if TYPE_CHECKING: + from sdk.events._models import AgentEvent + +__all__ = ["TurnPersistence"] + + +class TurnPersistence(Protocol): + """Optional persistence surface used by channels around a turn.""" + + def load_skills(self, conversation_id: str) -> Iterable[str]: + """Return persisted skill names to restore for this conversation.""" + + def save_skills(self, conversation_id: str, skills: Iterable[str]) -> None: + """Persist the set of currently-loaded skill names.""" + + def save_events( + self, + conversation_id: str, + events: list[AgentEvent], + ) -> None: + """Persist agent lifecycle events captured during the turn.""" + + async def on_new_conversation( + self, + conversation_id: str, + first_message: str, + ) -> None: + """Hook fired once when a conversation runs its first turn. + + Typical use: generate and persist a conversation title. + """ diff --git a/server/message_handler.py b/server/message_handler.py index 090ff4fb..5e5b250e 100644 --- a/server/message_handler.py +++ b/server/message_handler.py @@ -1,5 +1,6 @@ """Message handler for user prompts.""" +import asyncio import logging from collections.abc import AsyncGenerator, Sequence @@ -25,12 +26,16 @@ ContentPayload, TurnEndPayload, ) +from sdk.skills import AgentState from sdk.turn import is_turn_active from tools.browser.core import release_agent_browser from tools.memory import forget, memory_prompt_block, remember from tools.virtual_computer.receive_file import receive_attachment from tools.virtual_computer.run_bash_cmd import run_bash_cmd +# Background tasks held to prevent GC; cleared via done callback. +_background_tasks: set[asyncio.Task] = set() + logger = logging.getLogger(__name__) _console = Console(stderr=True) @@ -180,19 +185,23 @@ async def handle_user_message( _log_turn_start(profile) + if conversation.agent_state is None: + conversation.agent_state = await AgentState.create( + skill_names=[*profile.skills, *_persistence.load_skills(conversation.id)], + ) + agent = build_agent(profile, tools=[run_bash_cmd, remember, forget]) + agent.instruction = memory_prompt_block() + agent.instruction + events: list[AgentEvent] = [] try: async for event in _turn_executor.execute( conversation=conversation, agent=agent, user_content=user_content, - is_new_conversation=is_new_conversation, - preloaded_skills=profile.skills, - persistence=_persistence, - build_system_prompt=lambda: memory_prompt_block() + agent.instruction, profile_name=profile.name, ): + events.append(event) yield event except Exception: logger.exception("Error handling user message") @@ -203,3 +212,25 @@ async def handle_user_message( ) ) yield AgentEvent(payload=TurnEndPayload(type="turn_end")) + return + + # Post-turn persistence. Errors logged, never raised — the user already + # got their reply. + try: + _persistence.save_events(conversation.id, events) + except Exception: + logger.exception("Failed to save agent events for '%s'", conversation.id) + if conversation.agent_state.loaded_skill_names: + try: + _persistence.save_skills( + conversation.id, + conversation.agent_state.loaded_skill_names, + ) + except Exception: + logger.exception("Failed to save loaded skills for '%s'", conversation.id) + if is_new_conversation: + task = asyncio.create_task( + _persistence.on_new_conversation(conversation.id, user_content), + ) + _background_tasks.add(task) + task.add_done_callback(_background_tasks.discard) diff --git a/tasks/_executor.py b/tasks/_executor.py index f2a0046e..4a69f76c 100644 --- a/tasks/_executor.py +++ b/tasks/_executor.py @@ -42,11 +42,12 @@ async def run(self, task_result: TaskResult, task: Task) -> tuple[str, list[str] conversation_id = f"goals/{run.goal_id}/{run.id}/{task_result.id}" self._store.set_conversation_id(task_result.id, conversation_id) - agent = await self._build_agent(task) + agent, agent_state = await self._build_agent(task) conversation = Conversation( id=conversation_id, history=ConversationHistory(instance_id=conversation_id), + agent_state=agent_state, ) accumulated_text: list[str] = [] @@ -65,12 +66,12 @@ async def run(self, task_result: TaskResult, task: Task) -> tuple[str, list[str] return "".join(accumulated_text), file_paths - async def _build_agent(self, task: Task) -> Agent: - """Construct an Agent from the task's agent profile. + async def _build_agent(self, task: Task) -> tuple[Agent, AgentState]: + """Construct an ``Agent`` and matching ``AgentState`` for the task. - Pre-validates the profile's skills so a missing one trips the task - runner with a clear message before the turn starts. ``TurnExecutor`` - independently restores the same skills via ``preloaded_skills``. + Strict on missing skills: a profile that references an unregistered + skill fails the task synchronously with a clear message rather than + running with a silently degraded tool set. """ if not task.agent_profile: msg = f"Task {task.id} has no agent_profile set" @@ -88,7 +89,8 @@ async def _build_agent(self, task: Task) -> Agent: raise RuntimeError(msg) state.add(skill) - return build_agent(profile, tools=state.tools, name="TASK_AGENT") + agent = build_agent(profile, tools=state.tools, name="TASK_AGENT") + return agent, state def _build_instruction( self, task_result: TaskResult, task: Task, goal: Goal diff --git a/tests/unit/sdk/skills/test_agent_state_create.py b/tests/unit/sdk/skills/test_agent_state_create.py new file mode 100644 index 00000000..e87e6bb2 --- /dev/null +++ b/tests/unit/sdk/skills/test_agent_state_create.py @@ -0,0 +1,84 @@ +"""Tests for AgentState.create() — the async factory used by all callsites.""" + +from __future__ import annotations + +from unittest.mock import AsyncMock, patch + +import pytest + +from sdk.skills._registry import Skill, _SKILL_REGISTRY, register_skill +from sdk.skills.agent_state import AgentState + +_MOD = "sdk.skills.agent_state" + + +def _make_tool(name: str): + async def tool() -> str: + return name + tool.__name__ = name + return tool + + +def _make_skill(name: str, tool_names: list[str]) -> Skill: + return Skill( + name=name, + description=f"desc_{name}", + prompt=f"{name}-prompt", + tools=[_make_tool(t) for t in tool_names], + ) + + +@pytest.fixture(autouse=True) +def _clean_registry(): + saved = dict(_SKILL_REGISTRY) + yield + _SKILL_REGISTRY.clear() + _SKILL_REGISTRY.update(saved) + + +@pytest.mark.unit +async def test_create_pulls_core_tools_and_starts_empty_without_skills(): + core = [_make_tool("core_a"), _make_tool("core_b")] + with patch(f"{_MOD}.get_core_tools", new=AsyncMock(return_value=core)): + state = await AgentState.create() + + assert state.loaded_skill_names == frozenset() + names = [getattr(t, "__name__", None) for t in state.tools] + assert names == ["core_a", "core_b"] + + +@pytest.mark.unit +async def test_create_loads_named_skills_from_registry(): + register_skill(_make_skill("alpha", ["a_tool"])) + register_skill(_make_skill("beta", ["b_tool"])) + + with patch(f"{_MOD}.get_core_tools", new=AsyncMock(return_value=[])): + state = await AgentState.create(skill_names=["alpha", "beta"]) + + assert state.loaded_skill_names == frozenset({"alpha", "beta"}) + names = {getattr(t, "__name__", None) for t in state.tools} + assert names == {"a_tool", "b_tool"} + + +@pytest.mark.unit +async def test_create_logs_and_skips_unknown_skills(caplog): + register_skill(_make_skill("known", ["t"])) + + with ( + patch(f"{_MOD}.get_core_tools", new=AsyncMock(return_value=[])), + caplog.at_level("WARNING"), + ): + state = await AgentState.create(skill_names=["known", "ghost"]) + + assert state.loaded_skill_names == frozenset({"known"}) + assert any("ghost" in rec.getMessage() for rec in caplog.records) + + +@pytest.mark.unit +async def test_create_deduplicates_repeated_skill_names(): + register_skill(_make_skill("alpha", ["a_tool"])) + + with patch(f"{_MOD}.get_core_tools", new=AsyncMock(return_value=[])): + state = await AgentState.create(skill_names=["alpha", "alpha", "alpha"]) + + assert state.loaded_skill_names == frozenset({"alpha"}) diff --git a/tests/unit/sdk/turn/test_executor.py b/tests/unit/sdk/turn/test_executor.py index 45a1af66..ec9a55b5 100644 --- a/tests/unit/sdk/turn/test_executor.py +++ b/tests/unit/sdk/turn/test_executor.py @@ -1,15 +1,16 @@ -"""Tests for sdk.turn._executor.TurnExecutor — injection points + persistence.""" +"""Tests for sdk.turn._executor.TurnExecutor — pre-wired turn driver.""" from __future__ import annotations from typing import Any -from unittest.mock import AsyncMock, MagicMock, patch +from unittest.mock import AsyncMock, patch import pytest from agents.types import Agent from sdk import Conversation, TurnExecutor from sdk.context import ConversationHistory +from sdk.skills import AgentState from sdk.skills._registry import Skill _MOD = "sdk.turn._executor" @@ -33,35 +34,18 @@ def _make_agent(**overrides: Any) -> Agent: return Agent(**defaults) -def _make_conversation(conv_id: str = "c1") -> Conversation: +def _make_conversation( + conv_id: str = "c1", *, agent_state: AgentState | None = None, +) -> Conversation: return Conversation( id=conv_id, history=ConversationHistory(instance_id=conv_id), + agent_state=agent_state if agent_state is not None else AgentState([]), ) -class _FakePersistence: - """Implements the TurnPersistence protocol with recording stubs.""" - - def __init__(self, persisted_skills: list[str] | None = None) -> None: - self._persisted_skills = persisted_skills or [] - self.load_skills_calls: list[str] = [] - self.save_skills_calls: list[tuple[str, list[str]]] = [] - self.save_events_calls: list[tuple[str, list[Any]]] = [] - self.on_new_calls: list[tuple[str, str]] = [] - - def load_skills(self, conversation_id: str): - self.load_skills_calls.append(conversation_id) - return list(self._persisted_skills) - - def save_skills(self, conversation_id: str, skills) -> None: - self.save_skills_calls.append((conversation_id, sorted(skills))) - - def save_events(self, conversation_id: str, events) -> None: - self.save_events_calls.append((conversation_id, list(events))) - - async def on_new_conversation(self, conversation_id: str, first_message: str) -> None: - self.on_new_calls.append((conversation_id, first_message)) +def _fake_skill(name: str) -> Skill: + return Skill(name=name, description=f"{name} skill", prompt=f"{name}-prompt", tools=[]) async def _drain(executor_call): @@ -70,72 +54,27 @@ async def _drain(executor_call): # --------------------------------------------------------------------------- -# build_system_prompt +# agent_state contract — must be populated before execute() # --------------------------------------------------------------------------- @pytest.mark.unit -async def test_build_system_prompt_result_is_used_as_base(): - conv = _make_conversation() - agent = _make_agent(instruction="DEFAULT") - - with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): - await _drain(TurnExecutor().execute( - conversation=conv, - agent=agent, - user_content="hi", - build_system_prompt=lambda: "FRESH PROMPT", - )) - - # The system message ends up at index 0 of history. - system = conv.history.messages[0] - assert system["role"] == "system" - assert system["content"] == "FRESH PROMPT" - - -@pytest.mark.unit -async def test_no_builder_falls_back_to_agent_instruction(): - conv = _make_conversation() - agent = _make_agent(instruction="DEFAULT") - - with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): - await _drain(TurnExecutor().execute( - conversation=conv, - agent=agent, - user_content="hi", - )) - - assert conv.history.messages[0]["content"] == "DEFAULT" - - -@pytest.mark.unit -async def test_builder_is_called_fresh_each_turn(): - """The closure is re-evaluated per turn so live state (memory) lands fresh.""" - conv = _make_conversation() +async def test_missing_agent_state_raises(): + """The executor refuses to run when the caller forgot to populate it.""" + conv = Conversation( + id="c1", + history=ConversationHistory(instance_id="c1"), + ) agent = _make_agent() - call_count = 0 - def _builder() -> str: - nonlocal call_count - call_count += 1 - return f"prompt-{call_count}" - - with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): + with pytest.raises(ValueError, match="agent_state"): await _drain(TurnExecutor().execute( - conversation=conv, agent=agent, user_content="t1", - build_system_prompt=_builder, - )) - await _drain(TurnExecutor().execute( - conversation=conv, agent=agent, user_content="t2", - build_system_prompt=_builder, + conversation=conv, agent=agent, user_content="hi", )) - assert call_count == 2 - assert conv.history.messages[0]["content"] == "prompt-2" - # --------------------------------------------------------------------------- -# user_content append + system prompt placement +# History writes — user message append + system prompt placement # --------------------------------------------------------------------------- @@ -149,180 +88,62 @@ async def test_user_message_appended_to_history(): conversation=conv, agent=agent, user_content="hello world", )) - roles = [m["role"] for m in conv.history.messages] - assert "user" in roles - user_msg = next(m for m in conv.history.messages if m["role"] == "user") - assert user_msg["content"] == "hello world" - - -# --------------------------------------------------------------------------- -# Skill preloading + persistence restore -# --------------------------------------------------------------------------- - - -def _fake_skill(name: str) -> Skill: - return Skill(name=name, description=f"{name} skill", prompt=f"{name}-prompt", tools=[]) + user_msgs = [m for m in conv.history.messages if m["role"] == "user"] + assert len(user_msgs) == 1 + assert user_msgs[0]["content"] == "hello world" @pytest.mark.unit -async def test_preloaded_skills_added_via_registry(): +async def test_system_message_is_agent_instruction_when_no_skills_loaded(): conv = _make_conversation() - agent = _make_agent() - - skills_lookup = {"foo": _fake_skill("foo"), "bar": _fake_skill("bar")} + agent = _make_agent(instruction="MY INSTRUCTION") - with ( - patch(f"{_MOD}.run_turn", new_callable=AsyncMock), - patch(f"{_MOD}.get_skill", side_effect=lambda n: skills_lookup.get(n)), - ): - await _drain(TurnExecutor().execute( - conversation=conv, agent=agent, user_content="hi", - preloaded_skills=["foo", "bar"], - )) - - # Skill names land in the system prompt via build_skill_prompt. - system = conv.history.messages[0]["content"] - assert "foo-prompt" in system - assert "bar-prompt" in system - - -@pytest.mark.unit -async def test_preloaded_skill_not_in_registry_is_logged_and_skipped(caplog): - conv = _make_conversation() - agent = _make_agent() - - with ( - patch(f"{_MOD}.run_turn", new_callable=AsyncMock), - patch(f"{_MOD}.get_skill", return_value=None), - caplog.at_level("WARNING"), - ): + with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): await _drain(TurnExecutor().execute( conversation=conv, agent=agent, user_content="hi", - preloaded_skills=["ghost"], )) - # Logged a warning; no crash. - assert any("ghost" in rec.getMessage() for rec in caplog.records) + system = conv.history.messages[0] + assert system["role"] == "system" + assert system["content"] == "MY INSTRUCTION" @pytest.mark.unit -async def test_persistence_load_skills_restores_them(): - conv = _make_conversation() - agent = _make_agent() - persistence = _FakePersistence(persisted_skills=["restored_a", "restored_b"]) - skills = {n: _fake_skill(n) for n in ("restored_a", "restored_b")} +async def test_system_message_includes_skill_prompt_when_skills_loaded(): + """When agent_state has skills, their prompts append to the agent's instruction.""" + state = AgentState([]) + state.add(_fake_skill("alpha")) + state.add(_fake_skill("beta")) + conv = _make_conversation(agent_state=state) + agent = _make_agent(instruction="BASE") - with ( - patch(f"{_MOD}.run_turn", new_callable=AsyncMock), - patch(f"{_MOD}.get_skill", side_effect=lambda n: skills.get(n)), - ): + with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): await _drain(TurnExecutor().execute( conversation=conv, agent=agent, user_content="hi", - persistence=persistence, )) - assert persistence.load_skills_calls == [conv.id] system = conv.history.messages[0]["content"] - assert "restored_a-prompt" in system - assert "restored_b-prompt" in system - - -# --------------------------------------------------------------------------- -# Persistence write paths -# --------------------------------------------------------------------------- + assert system.startswith("BASE") + assert "alpha-prompt" in system + assert "beta-prompt" in system @pytest.mark.unit -async def test_persistence_save_skills_called_with_loaded_names(): - conv = _make_conversation() +async def test_executor_does_not_load_skills_or_call_persistence(): + """Verify the executor stays out of the persistence/skill-loading business + (those moved to callers in the refactor).""" + state = AgentState([]) + conv = _make_conversation(agent_state=state) agent = _make_agent() - persistence = _FakePersistence() - skill = _fake_skill("alpha") with ( patch(f"{_MOD}.run_turn", new_callable=AsyncMock), - patch(f"{_MOD}.get_skill", return_value=skill), + # If the executor reached for the registry we'd see it here. + patch("sdk.skills._registry.get_skill") as get_skill_mock, ): await _drain(TurnExecutor().execute( conversation=conv, agent=agent, user_content="hi", - preloaded_skills=["alpha"], - persistence=persistence, )) - assert persistence.save_skills_calls == [(conv.id, ["alpha"])] - - -@pytest.mark.unit -async def test_persistence_save_skills_skipped_when_no_skills_loaded(): - conv = _make_conversation() - agent = _make_agent() - persistence = _FakePersistence() - - with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): - await _drain(TurnExecutor().execute( - conversation=conv, agent=agent, user_content="hi", - persistence=persistence, - )) - - assert persistence.save_skills_calls == [] - - -@pytest.mark.unit -async def test_on_new_conversation_fires_only_when_flagged(): - conv = _make_conversation() - agent = _make_agent() - persistence = _FakePersistence() - - with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): - await _drain(TurnExecutor().execute( - conversation=conv, agent=agent, user_content="first-turn", - is_new_conversation=True, - persistence=persistence, - )) - - # on_new_conversation runs as a background task — give it a turn. - import asyncio - await asyncio.sleep(0) - assert persistence.on_new_calls == [(conv.id, "first-turn")] - - -@pytest.mark.unit -async def test_on_new_conversation_skipped_on_continuation(): - conv = _make_conversation() - agent = _make_agent() - persistence = _FakePersistence() - - with patch(f"{_MOD}.run_turn", new_callable=AsyncMock): - await _drain(TurnExecutor().execute( - conversation=conv, agent=agent, user_content="t", - is_new_conversation=False, - persistence=persistence, - )) - - import asyncio - await asyncio.sleep(0) - assert persistence.on_new_calls == [] - - -# --------------------------------------------------------------------------- -# persistence=None — every persistence call must be skipped -# --------------------------------------------------------------------------- - - -@pytest.mark.unit -async def test_no_persistence_is_safe(): - conv = _make_conversation() - agent = _make_agent() - skill = _fake_skill("alpha") - - with ( - patch(f"{_MOD}.run_turn", new_callable=AsyncMock), - patch(f"{_MOD}.get_skill", return_value=skill), - ): - # Nothing should raise, even with preloaded skills and is_new=True. - await _drain(TurnExecutor().execute( - conversation=conv, agent=agent, user_content="hi", - is_new_conversation=True, - preloaded_skills=["alpha"], - persistence=None, - )) + assert get_skill_mock.call_count == 0 + assert state.loaded_skill_names == frozenset() diff --git a/tests/unit/tasks/test_executor.py b/tests/unit/tasks/test_executor.py index 803e5479..65b77f9d 100644 --- a/tests/unit/tasks/test_executor.py +++ b/tests/unit/tasks/test_executor.py @@ -81,8 +81,11 @@ async def _stub_core_tools() -> list[Any]: ) executor = TaskExecutor(store=None) # type: ignore[arg-type] - agent = await executor._build_agent(task) + agent, state = await executor._build_agent(task) assert agent.name == "TASK_AGENT" + # state holds the core tools (none registered in this stub) — primarily + # asserting we got both pieces back, not their contents. + assert state is not None @pytest.mark.unit From 63d7f5b2bf1c2c472efff8b7a109643c5ddf1a21 Mon Sep 17 00:00:00 2001 From: larry foulkrod Date: Sun, 31 May 2026 13:57:44 -0500 Subject: [PATCH 12/12] docs: add resume-telegram plan capturing PR state and follow-ups A pick-up note for next session: where the PR lives (omnideck#4), what's landed on the branch, the known deferred items (no streaming reply, parallel sub-agent browser contention, live chat-ID discovery, markdown edge cases), and how to resume. Co-Authored-By: Claude Opus 4.7 (1M context) --- plans/resume-telegram.md | 87 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 87 insertions(+) create mode 100644 plans/resume-telegram.md diff --git a/plans/resume-telegram.md b/plans/resume-telegram.md new file mode 100644 index 00000000..837571d2 --- /dev/null +++ b/plans/resume-telegram.md @@ -0,0 +1,87 @@ +# Resume — Telegram integration PR + +Where to pick up next session. + +## PR + +- **omnideck-dev/omnideck#4** — `feature/telegram-integration-v2` → `main` +- Built originally on `lefoulkrod/computron_9000#55` (closed/abandoned once + the repo moved to the `omnideck` origin); branch history shows the full + trail of incremental commits. + +## Branch state + +- Synced with `origin/main` via merge `e15355f` (resolved 5 conflicts in + the integrations area where the new HTTP/`call_api` integration overlapped + with our Telegram additions — both kept, no behavior lost). +- 1429 unit tests pass. + +## What's landed on this branch + +1. **Telegram channel** (`channels/telegram/_channel.py`) — pulls updates + from the broker via `next_updates`, dispatches per-chat turns, renders + status messages (`🤔 Thinking…` → tool labels → `✍️ Writing…`), sends + replies as MarkdownV2 (via `telegramify-markdown`), handles inbound + photos + documents, `/list` (search and resume any conversation), + `/profile` (inline-keyboard picker). +2. **Credential-isolated broker** (`integrations/brokers/telegram_broker/`) + — bot token stays in the broker subprocess, never enters the main app. + Verbs: `get_me`, `next_updates`, `send_message`, `send_document`, + `send_chat_action`, `answer_callback_query`, `edit_message_text`, + `delete_message`. +3. **`ConversationCache`** (`conversations/_cache.py`) — bounded-LRU + hydrate/evict, shared between the web/SSE handler and the Telegram + channel; skips eviction of conversations with an in-flight turn. +4. **`TurnExecutor` refactor** (slimmed) — `Conversation` now holds + `agent_state`; the executor stops loading skills / saving events / + firing first-turn hooks. Channels do that wiring inline. Covers the + web handler, Telegram channel, `TaskExecutor`, and `spawn_agent`. + `TurnPersistence` Protocol moved to `sdk/turn/_persistence.py`. + `SystemPromptBuilder` dropped. +5. **Goal-run notifier settings** — env vars + `TELEGRAM_INTEGRATION_ID` / `TELEGRAM_CHAT_ID` retired. Now in + `settings.json` as `telegram_notifier_integration_id` / + `telegram_notifier_chat_id`, editable in **Settings → System → + Notifications**. Migration 006 seeds empty defaults on existing installs. +6. **Three earlier PR-review fixes** (formatter, channel rename, + env→settings) — see commits `a93ab3b`, `d0364ae` and the resolved + threads on the original computron PR for context. + +## Known / deferred follow-ups + +- **No streaming reply for Telegram.** We buffer the full agent reply + and send it in chunks at turn end (via `to_markdownv2_chunks`). + Real-time streaming would need either per-paragraph `send_message` + calls or `edit_message_text` to grow a single message — both have + Telegram rate-limit and chunk-boundary concerns. Not a blocker. +- **Parallel sub-agent browser contention** — when an agent spawns + sub-agents in parallel, browser tool calls still serialize. Needs + per-agent browser contexts. Tracked separately + ([[project_parallel_subagents]]). +- **Title generation on first turn is fire-and-forget** — if the title + model is slow or unhealthy, the user just sees the conversation ID + on `/list` until it lands. Acceptable. +- **No live chat-ID discovery** in the notifier setup — user has to + copy chat.id from `getUpdates` JSON. We discussed a broker-side + "recent chats" registry to power a dropdown; deferred until someone + trips over the manual flow. +- **`telegramify-markdown` corner cases** — if the agent emits + malformed markdown (unbalanced backticks across chunk boundaries, + weird nested formatting), the Telegram API may reject. Today we + log+warn on send failures and the user just sees nothing for that + chunk. Worth watching once real traffic hits. + +## How to pick up next session + +1. `cd ~/repos/computron_9000-telegram && git status` — confirm branch + `feature/telegram-integration-v2`, clean tree. +2. `gh pr view 4 --repo omnideck-dev/omnideck` — check for new review + comments and CI status. +3. If new inline review comments: `gh api repos/omnideck-dev/omnideck/pulls/4/comments` + to fetch, then walk them one at a time (same pattern as the three + from the computron PR). +4. For manual smoke testing: `just dev` brings up the container; add a + Telegram integration via the wizard; talk to the bot; verify + MarkdownV2 rendering + status indicator + `/list` resume work. +5. The dev container is `computron_virtual_computer` under **docker**, + not podman.