diff --git a/README.md b/README.md index 0d175e7f..31866279 100644 --- a/README.md +++ b/README.md @@ -2,27 +2,41 @@ English | [简体中文](README.zh-CN.md) -[OctoSense](https://github.com/OctoSense-org) is an agent shell on top of your operating system: a launcher and apps that look like the ones you know, with a system agent coordinating app agents. This repository holds all of OctoSense's own code in one place ([ADR 0001](docs/adr/0001-one-octosense-repository.md)): the shell, its services, the first-party system apps, and the three products built from them. +OctoSense is an agent shell that runs on top of an ordinary operating system. On screen it is a launcher and apps that look like the ones you know. Behind them, one AI kernel runs a **system agent** that works for the person and an **app agent** for each app that has one. Agents reach apps, and risky actions reach the person, only through the shell. + +This repository holds the shell, its services, the system apps, and the three products built from them. Most native apps come from other repositories: the OctoSense fork of Makepad, App Hub and Rinx ([What it depends on](#what-it-depends-on)). | Product | What it is | Where | | --- | --- | --- | -| **OctoSense desktop** | The shell as one Makepad window on macOS (Windows and Linux untested): launcher, dock, tiles, hosted apps | [`desktop/`](desktop/README.md) | +| **OctoSense desktop** | The shell as one Makepad window on macOS (Windows and Linux untested) | [`desktop/`](desktop/README.md) | | **OctoSense Home** | The phone shell, an ordinary Home app for any Android phone (also OpenHarmony and the iOS simulator) | [`phone/`](phone/README.md) | | **OctoSense ROM** | LineageOS 22.2 for the OnePlus 6 with Home, the privileged system bridge, Quickstep and SystemUI preinstalled | [`rom/`](rom/README.md) | -It was OctoSense-Desktop; OctoSense-ROM (retired; merged into this repository) and OctoSense-System-Apps were imported into it with their history on 2026-09-27. OctoSense-System-Apps is archived; the OctoSense-ROM repository no longer exists. - -> **Building an OctoSense app?** You do not need this repository to build, check or publish one. Start at the [OctoSense-org profile](https://github.com/OctoSense-org)'s reading list: [OctoScript-App-Design-Flow](https://github.com/OctoSense-org/OctoScript-App-Design-Flow) (`AGENTS.md`, then `docs/QUICKSTART.md`) and [OctoSense-App-Hub](https://github.com/OctoSense-org/OctoSense-App-Hub). The system apps in [`apps/`](apps/README.md) are complete examples of the same app shape (`apps//bundle/`). Build the desktop shell from here only to see your app in a shell before it is published ([PUBLISHING §4](https://github.com/OctoSense-org/OctoScript-App-Design-Flow/blob/main/docs/PUBLISHING.md#4-rehearse-the-store-path-locally)). +> **Building an app?** You don't need this repository to build, check or publish one. Start with [OctoScript-App-Design-Flow](https://github.com/OctoSense-org/OctoScript-App-Design-Flow) (`AGENTS.md`, then `docs/QUICKSTART.md`) and [OctoSense-App-Hub](https://github.com/OctoSense-org/OctoSense-App-Hub). The system apps in [`apps/`](apps/README.md) are complete examples. Build the desktop shell from here only to try your app in a shell before you publish it ([PUBLISHING §4](https://github.com/OctoSense-org/OctoScript-App-Design-Flow/blob/main/docs/PUBLISHING.md#4-rehearse-the-store-path-locally)). -## Start with the code walkthrough +## Key concepts -New to the codebase? Start with [From an app window to an agent turn](docs/architecture-walkthrough.md). It follows executable entry points, native and script hosting, app data access, human/system conversations, tool routing and the actual Tokio tasks. The [product walkthrough](desktop/docs/code-walkthrough.md) adds desktop, Home, ROM and system-app run recipes. +| Term | What it means here | +| --- | --- | +| **octos** | An open-source agent kernel written in Rust ([octos-org/octos](https://github.com/octos-org/octos)). It runs agent *sessions*: a conversation with a model, with its own tools, memory and workspace folder. A session can own other sessions, called *peers*. OctoSense runs one octos per device. | +| **OUP** | The octos UI protocol: JSON-RPC 2.0 messages (`octos-ui/v1alpha1`) between octos and its clients, normally over the kernel's stdin and stdout. | +| **Shell** | The one OctoSense process on a device. It draws everything, hosts the apps and holds the only full connection to octos. Desktop and phone build the same crate, `crates/shell`. | +| **System agent** | The person's assistant: one octos session that owns and supervises every app agent. | +| **App agent** | An octos peer for one app and one account, so Mail with two accounts has two agents. Each has its own memory, transcript, model and tool list, and usually a folder to work in. | +| **Lane** | One of an app agent's two conversations, which run side by side: the system agent's lane, and the person's lane, which also carries the turns the app starts itself. | +| **Native app** | A Rust app built with Makepad and listed in [`native-apps.json`](native-apps.json). It runs inside the shell as a module, or as its own sandboxed process (the Terminal on the desktop). | +| **Script app** | An OctoScript bundle: a manifest, a UI written in Splash (Makepad's UI script language) and, optionally, its agent's tools in `tools.json`. It reaches the shell only through `host.request`. The system apps (Mail, Calendar, News and the rest) and every store app are script apps. | +| **Card runner** | App Hub's runtime for script apps, inside the shell process: one isolated script VM per app instance, with its own file jail and quota. | +| **Host service** | Rust code in the shell that answers one family of `host.request` calls (`mail.*`, `calendar.*`) and runs that app's agent tools. | +| **Peer link** | How native apps other than Rinx talk to their own agent through the shell: Makepad's `OctosPeer` client, carried over the app's hub connection when it is a process, or over an in-memory channel when it is a module. | +| **Glance card** | A small card an app or its agent posts to the glance panel (desktop) or glance page (phone). It is either an L0 card (declarations only: the host fills in the data, and there are no expressions or calls) or a Splash script. | +| **Approval router** | The shell code that decides whether a tool call runs, needs the person, or is refused. | + +To read the code in order, start with [From an app window to an agent turn](docs/architecture-walkthrough.md). The [product walkthrough](desktop/docs/code-walkthrough.md) adds how to run each product. ## How it fits together -One shell process per device, one octos kernel per shell, and every agent is a session in that kernel. App-agent access goes through the shell broker. The shell holds the host connection and the host token, relays every agent tool call to the app that owns the tool, and routes approvals through developer mode, the person's standing rules or confirmation sheets. The opt-in AppCard prototype accesses the shared kernel service directly, bypassing the app-peer broker, as described below. The full picture, with the code paths and what is on `main` versus planned: [docs/architecture.md](docs/architecture.md); the decisions: [ADR 0004](docs/adr/0004-native-apps-hosting-and-peers.md). - -### Processes and connections +One shell process per device, one octos kernel per shell, and every agent is a session in that kernel. The shell is the kernel's only full client. It starts octos and holds its host token, starts every app agent's turns, relays every call to an app's tools, and owns every approval. Apps never talk to the kernel. ![OctoSense processes and connections](docs/images/agents-processes.png) @@ -34,7 +48,7 @@ flowchart LR ext["Talk to Octos client
web or terminal, opt-in"] subgraph shellp["OctoSense shell process"] ui["Window manager, launcher,
system chat, Ask app panel, sheets"] - mods["Native modules
App Hub, Rinx"] + mods["Native modules
App Hub, Rinx, Notes, Clock, …"] runner["Card runner
script apps, glance cards"] aihost["ai-host + app-peers broker
host connection"] relay["Host-tool relay"] @@ -50,7 +64,7 @@ flowchart LR ui --- mods ui --- runner ui -->|"person's turns:
Ask app, in-card chat"| aihost - mods -->|"OctosAppService"| aihost + mods -->|"OctosPeer, OctosAppService"| aihost runner -->|"host.request octos.*"| aihost term <-->|"hub: frames, AI bus"| ui term -.->|"peer link"| aihost @@ -65,51 +79,53 @@ flowchart LR -- **The shell** (`crates/shell`, one process) hosts the window manager, the native modules (App Hub, Rinx), App Hub's Card runner (every script app in its own isolate), the system chat, the approval router, the host-tool relay and [`crates/ai-host`](crates/ai-host/README.md), whose [app-peers broker](crates/app-peers/README.md) is the kernel's host connection. -- **The octos kernel** ([`crates/kernel`](crates/kernel/README.md)) starts on first use: a child process speaking OUP over stdio on the desktop (the packaged `octos-kernel` beside the shell, or `OCTOS_APP_CORE_BIN`) and Android (`liboctos.so`), an in-process task on OpenHarmony, none on iOS. It exits with the shell. -- **Process apps**: on the desktop the Terminal runs as its own process, attached over the shell's hub (frames and the AI bus), in an OS sandbox built from its `native-apps.json` entry (Seatbelt on macOS, Landlock and seccomp on Linux, not yet on Windows). A process app reaches its own agent over the **peer link**; the shell side is on `main`, but the Terminal is not granted an agent, so no process app uses it yet. -- **External clients**: Talk to Octos (opt-in) lets a web or terminal client use the system conversation with a limited external token: an allowlist of methods, no `peer/*` method, no app agent's session, no host-routed tools. - -### Every path into octos goes through the shell +- **The shell** hosts the window manager, the native apps, the Card runner with the script apps, the system chat, the approval router and the host-tool relay. Its AI side is [`crates/ai-host`](crates/ai-host/README.md), with the [app-peers broker](crates/app-peers/README.md) that drives each app agent. +- **The octos kernel** ([`crates/kernel`](crates/kernel/README.md)) starts on its first connection and exits with the shell. On the desktop and Android it is a child process that speaks OUP over stdio; on OpenHarmony it runs inside the shell; iOS has none. The person picks models and enters keys in the **AI providers** app, on host sheets. Keys stay with the shell (in the macOS keychain, or in an owner-only file elsewhere) and never reach an app. +- **Process apps** run outside the shell in an OS sandbox (Seatbelt on macOS, Landlock and seccomp on Linux, none yet on Windows). Today that is only the Terminal on the desktop. It sends its frames over the shell's local hub and reaches its agent over the peer link on the same connection. +- **External clients** (Talk to Octos, opt-in) can use the system conversation from a browser or a terminal with a limited token. They get no app agent, no `peer/*` method and no host tool. -| Who | Path | Status | -| --- | --- | --- | -| In-process native module | the same peer link as a process app, through Makepad's `OctosPeer` client (the module host claims the link for the instance that opened it) | on `main`; no module uses it yet | -| In-process native module (Rinx) | the injected `OctosAppService`: `open_conversation` (the app's conversation with its agent) and `open_context` (a per-client request context, such as a Rinx mini app) | on `main` | -| Script app, and its cards | `host.request("octos.session.open" / "octos.session.history" / "octos.turn.start" / "octos.turn.interrupt")` to the `octos` host service, for the names its manifest declares; a card's in-card chat (`sys.chat`) through the shell | on `main`, after first-use consent (the shipped gate; `OCTOSENSE_CONTAINED_APPS=1` skips first-use consent, `0` turns it off). Today's system apps with an agent declare no `octos.*` name: the shell drives their agents | -| Process app | the peer link on its hub connection (`octos.session.open`, `octos.turn.start`, …), identity stamped by the shell | shell side on `main`; no process app granted an agent yet | -| The system agent | the kernel's own session `_main:api:octosense#system`, reached from the shell's system chat | on `main` | -| Talk to Octos client | the system conversation only, with the external token | on `main` | +The opt-in AppCard prototype is the one exception to all this: it opens its own kernel connection instead of going through the broker. For the full picture with code paths, read [docs/architecture.md](docs/architecture.md); for the trust model and how to test the AI services locally, [docs/ai-services.md](docs/ai-services.md); for the decisions behind it, [ADR 0004](docs/adr/0004-native-apps-hosting-and-peers.md). ### The system agent and the app agents -**The system agent** is the kernel session `_main:api:octosense#system` on the `_main` profile (`crates/kernel/src/network.rs`). It owns every app agent and supervises them. +**The system agent** is the octos session `_main:api:octosense#system`. The person talks to it in the system chat: F8 or the dock's Assistant icon on the desktop, the Assistant tile on the phone. It has two sets of tools: -The person talks to it in the shell's **assistant pane**, the system chat (`crates/shell/src/system_chat/`: the dock's Assistant icon, the phone home's Assistant tile or F8; a medium pane at the left on a desktop that moves and resizes and renders Markdown, full screen on a phone), or from a paired Talk to Octos client. +- **Its own kernel tools**, the fixed list in [`SYSTEM_AGENT_TOOLS`](crates/kernel/src/system_tools.rs): the `peer_*` tools to supervise app agents, plus files in its own workspace, memory, questions to the person and web search. octos's own shell tools are never on it. +- **Host tools from the shell**: `agents.list` and `agents.ask` to find app agents and ask the person to allow one, `agents.provision` and `agents.status` to run Mail's new-mail automation, `terminal.run` while Setup's Command execution switch is on, and the read tools native apps share with it (the table's last column). -Its kernel tools are exactly `SYSTEM_AGENT_TOOLS` (`crates/kernel/src/system_tools.rs`): `peer_send_input`, `peer_gather`, `peer_list` and `peer_respond` to supervise app agents, its workspace's file tools, memory, `ask_user_question`, media viewing, `web_search`, `web_fetch` and `tool_search`. Every kernel start sets that list with octos's `session/tool_list/set`, so octos's own shell, the spawn family and `peer_close` are never offered. +The system agent can never approve a tool call; only the person can. -The system chat also registers host tools on the session: `agents.list` and `agents.ask` (`crates/shell/src/agents.rs`: which apps have an agent, and the first-use sheet and ready-peer wait for one), `terminal.run` while Setup › Assistant › Command execution is on, and the native apps' own read tools their `native-apps.json` entries name (`agent.system_tools`): Calculator's `calculator.eval`, Clock's `clock.now`, Notes' `notes.search` and `notes.read`, Reminders' `reminders.due` and `reminders.list`, Weather's `weather.current`. Such a call reaches the app's open instance (its AI bus service); a closed app answers "Open Notes first". +**An app agent** exists once the person allows it, on a sheet shown once per app. When it runs depends on the kind of app: -**An app agent** is one host-owned octos peer per (app, account) ([`crates/app-peers`](crates/app-peers/README.md)). These apps have one on `main` (`crates/shell/src/apps.rs`, `agent_apps`): +- A **script app's** agent is prepared at shell startup with the app's tools registered, so the system agent's `peer_list` sees it even while the app is closed. +- A **native app's** agent belongs to the app's open window and is live only while the app is open; its memory and transcript persist between openings. Its tools run in that window too: with Notes closed, a call answers "Open Notes first". -| App | Declared by | Its tools (run by) | What it puts on the glance screen | +Signing out keeps an agent. Removing the account or uninstalling the app erases its transcripts and memory. + +These apps have an agent: + +| App | Kind | Its agent's own tools | What the system agent may call | | --- | --- | --- | --- | -| Rinx (native) | `native-apps.json` `agent.octos` (the four `octos.*` services) and `agent.generic_tools` | octos's generic tools in its list; its own assistant UI | – | -| News (`os.news`) | `apps/news/bundle/tools.json`, the manifest's `agent` block | `news.list`, `news.read`, `news.notify` (the `news` host service and shell notice callback) | a notice card with a notification | -| Mail (`os.mail`) | `apps/mail/bundle/tools.json`, admitted instructions/skills, background event declaration and `glance` | Account-scoped `mail.accounts`, `mail.folders`, `mail.sync`, `mail.list`, `mail.peek`, `mail.notify`, `mail.publish_card`, `mail.skip_event` | a model-authored L0 card or notice, with an OctoSense notification | -| Calendar (`os.calendar`, desktop only) | `apps/calendar/bundle/tools.json`, the manifest's `agent` block and `glance` | `calendar.events`, `calendar.add_event`, `calendar.remove_event` (destructive: enters the approval router), `calendar.notify`, `calendar.agenda` (the `calendar` host service) | an event card or an agenda card, with a notification | -| Photos, Maps, YouTube; Camera on phone | Their bundle `tools.json`, `agent` block and `glance` | Each app's `.notify` (shell `NoticeService`) | a notice card with a notification | +| Rinx (Matrix chat) | native, in the shell | octos's file, memory and web tools | – | +| Terminal (desktop) | native, its own process | `terminal.read_screen`, `terminal.read_scrollback` | `terminal.run`, behind Setup's switch, approved per command | +| Calculator, Clock, Notes, Reminders, Weather | native, in the shell | each app's read tools | the same read tools | +| Mail | script app | `mail.*` tools scoped to the signed-in account, including `mail.publish_card` | – | +| Calendar (desktop) | script app | `calendar.events`, `calendar.add_event`, `calendar.remove_event` (asks first), `calendar.notify`, `calendar.agenda` | – | +| News | script app | `news.list`, `news.read`, `news.notify` | – | +| Photos, Maps, YouTube; Camera on phones | script apps | `.notify` | – | -AI providers has no app agent. +App Hub and AI providers have no agent. -A script app has an agent when its manifest declares `octos.*` names or an `agent` block (`"tools": ["ask_user_question"]` names the kernel tools it may use), or its bundle ships `tools.json` (each tool `.` with its schemas, `risk`, `confirm` and `shareable`). The broker identifies the app as `card.`; the kernel returns its peer slug during preparation. The person allows the agent once, on the first-use sheet (from the shell's "Ask " panel, the app's own `octos` call or the system agent's `agents.ask`). From then on the shell prepares the peer at startup, with the app's tools registered, so the system agent's `peer_list` shows it. +### How the system agent and an app agent talk -On Unix, a consented app agent with an available workspace also gets the host read tools `files.list`, `files.read` and `files.search` over its account folder; these do not expose every host-service database. On the phone, which builds `toolbox-peers` by default, it also gets the system toolbox's tools when its manifest asks for `research` or `crawl`, which no app does yet. +The system agent never runs an app's tools itself. It asks the app's agent to do the work: -Mail has an opt-in incoming-email path: after account sign-in and agent consent, the system agent uses `agents.provision` to configure instructions, skill text and polling. A host worker syncs Inbox, queues new-message events durably and starts the Mail agent without a chat prompt. The agent reads through scoped tools and decides whether to publish a card. Initial sync establishes a baseline; existing mail does not trigger a backlog. Admitted `AGENT.md` and skill text accompany each turn; this is not kernel-native skill installation. Other bundle triggers and schedules remain unimplemented. See the [Mail event walkthrough](docs/mail-agent-events.md) for boundaries and current verification limits. +1. The system agent calls `peer_send_input` with the request in plain words. +2. octos does not run that turn on its own. It passes it to the shell as a `peer/input` event. +3. The shell starts the turn on the app agent's session, with the app's tools, memory and approval rules. It refuses the input if the person has not allowed the agent or the account is signed out. +4. The result lands on the peers' shared *blackboard*, where the system agent reads it with `peer_gather`. -**From the system agent to a card on the glance screen**: +Here the person asks for a heads-up on the glance screen, and Mail's agent posts a card: ![From the system agent to a card on the glance screen](docs/images/agents-card-flow.png) @@ -140,20 +156,13 @@ sequenceDiagram -For the following notice/calendar tools, the model supplies text and the host fills fixed templates. Mail additionally offers `mail.publish_card`, which accepts model-authored L0 source checked by the host: - -- **Notice cards.** Mail, News, Photos and the other `.notify` tools use the shell’s [`notice.card`](crates/shell/resources/glance/notice.card). The shell supplies the app’s icon and name; the call supplies the title and body. Reusing a `card_id` replaces the app’s earlier notice. -- **Calendar cards.** `calendar.notify` and `calendar.agenda` use Calendar’s [event and agenda templates](apps/calendar/host-service/resources). - -Mail and News forward `notify` from their host services to the shell. For apps without their own service, the shell’s [`glance_notice.rs`](crates/shell/src/glance_notice.rs) handles the call directly. In either case, the shell publishes as the app and requires its `glance` grant (`glance::publish_for`). - -Pressing one of these cards opens the app. To discuss a Mail notice with Mail’s agent, open “Ask Mail” ([below](#talking-to-an-apps-agent-yourself)). [In-card chat](#in-card-chat) describes the separate chat feature and its current availability. +The `.notify` tools fill a fixed card template ([`notice.card`](crates/shell/resources/glance/notice.card), or Calendar's [event and agenda cards](apps/calendar/host-service/resources)), so the model writes only the text. Mail also has `mail.publish_card`, which takes a card the model wrote and checks it before publishing. Either way the shell publishes as the app and requires its `glance` permission. -`mail.notify` and `calendar.add_event` are `act` tools and normally run without a per-call sheet. Destructive and outward calls enter the approval path; the shell’s approval router decides which requests need a person ([below](#a-tool-call-with-an-approval)). +Mail's agent can also start on its own. Once the person has signed in, allowed Mail's agent and asked the system agent to turn on new-mail processing (`agents.provision`), a host worker syncs the inbox and starts the agent for each new message. The agent reads the message with its scoped tools and decides whether to post a card. No other app has events yet; the [Mail event walkthrough](docs/mail-agent-events.md) has the details. ### One app agent, two lanes -An app agent is one host-owned octos **peer** per (app, account), owned by the system agent, with its own workspace, memory namespace, model and tool list. The system agent and the person each talk to it in their own lane: +The system agent and the person talk to the same app agent, each in a lane of their own: ![One app agent, two lanes](docs/images/agents-two-lanes.png) @@ -183,29 +192,72 @@ flowchart TB -- **The system agent's lane** is the peer's own session, `…#peer-`. The system agent sends `peer_send_input`; octos delivers it to the shell's host connection as `peer/input`, and the shell starts the turn itself, so it runs with the app's tools, memory and approvals (or refuses it with `peer/input/reject` for a signed-out account or an app the person has not allowed). The peer's results go to the peers' blackboard, which the system agent reads. -- **The person's lane** is a request context, `…#peerctx-.`, opened with `share_history` by the shell's "Ask " panel (`agents::conversation`, client instance `shell-ask`), by a card's in-card chat, or by the app's own UI (a native module's `open_conversation`, a script app's `octos.session.open`, a process app's peer link), a new one for every handle ([octos#2636](https://github.com/octos-org/octos/pull/2636), UPCR-2026-034). The two lanes run in parallel, one turn at a time per session: a person's message never waits for the system agent's turn. Each turn sees the other lane's recent messages as a read-only block that is never written into its own transcript, and every turn is labelled by its speaker (`[from the person: ]`, `[from the system agent]`). The app follows both lanes, each event tagged with its `lane` and speaker; `octos.session.history` merges both transcripts by time. The person's turns also leave rounds on the blackboard (`origin: person`), so the system agent sees them with `peer_gather`. -- *Until 2026-09-29 both spoke in one shared conversation on the peer's session ([#166](https://github.com/OctoSense-org/OctoSense/pull/166), octos#2626): one queue per peer, one turn at a time.* -- **Rinx mini apps** keep their own request contexts (`open_context`), each with its own transcript and folder, not shared with either lane. +- **The system agent's lane** is the peer's own session, `…#peer-`. +- **The person's lane** is a request context, `…#peerctx-.`, opened with `share_history`. The person's turns run there, and so do the turns the app starts itself. + +The lanes run in parallel, so the person never waits behind a system agent task. Each turn sees the other lane's recent messages as read-only context, and every message is labelled with its speaker (`[from the person: Mail]`, `[from the system agent]`). The person's turns also leave their results on the blackboard, so the system agent knows what was done. Rinx mini apps get private contexts of their own (`open_context`), shared with neither lane. ### Talking to an app's agent yourself -The person is not limited to the system agent: they can talk to any app's own agent directly. Every turn the person starts is a person turn in the person's lane of that app's peer, beside the system agent's lane. It runs with the app's tools, memory and approvals, as a system agent's turn does. +The person can talk to any app's agent directly. These turns run in the person's lane with the app's tools, memory and approvals, just as the system agent's turns do. + +| Where | How | +| --- | --- | +| **"Ask <app>"** panel | A shell panel for every app with an agent, opened from the bar's "Ask <app>" button or with Shift+F8. On the desktop it opens beside the system chat. The phone draws it full screen but has no touch control for it yet. | +| **In-card chat** | Type in a glance card that declares a chat ([below](#in-card-chat)). | +| **The app's own UI** | An app can open the person's lane itself ([next section](#how-an-app-uses-its-agent)). Rinx draws its own assistant UI; for the other apps, the "Ask <app>" panel is the way in. | + +The panel asks for consent first and shows both lanes, each message with its speaker. Its Stop button ends only the person's own turn. [docs/architecture.md §2](docs/architecture.md#2-agents) covers the rest of its behavior. + +### How an app uses its agent + +An app reaches its agent only through the shell, never through the raw kernel protocol, and the shell stamps the app's identity on every call. -| Surface | Where it is | How it opens | +| Kind of app | API | Used by | | --- | --- | --- | -| **"Ask "** (`crates/shell/src/app_chat/`) | A shell panel for every app with an agent, whether or not the app draws a chat of its own: the system chat's pane drawn as the app's conversation (`app_panel: true`). On a desktop it stands right of the system chat, so the two lanes show side by side. | The bar's "Ask " button (shown while the focused window's app has an agent), Shift+F8, or the menu row "Ask this app's agent". With an app without an agent focused, the shell says "No app agent here". On the phone the pane is drawn as a full-screen sheet, but no touch control opens it on `main` yet. | -| **A card’s in-card chat** (`sys.chat`, `crates/shell/src/glance_chat.rs`) | A glance card that declares a chat | The person types in the card; the publishing app’s own agent answers, with its reply marked AI-written. See [in-card chat](#in-card-chat) for current availability and the demo. | -| **The app's own UI** | A native module's `open_conversation`, a script app's `octos.session.open`, a process app's peer link | Inside the app. Rinx draws its own assistant UI; none of the system apps draws a chat, so the “Ask ” panel is their current conversation entry. | +| Script app | `host.request("octos.session.open" / "octos.session.history" / "octos.turn.start" / "octos.turn.interrupt")`, limited to the names its manifest declares | store apps. The system apps declare none: the shell drives their agents. | +| Native app, in the shell or as a process | Makepad's `OctosPeer` client over the peer link: open the link, then `serve_tools` to answer the agent's calls to the app's own tools. The same code works in either hosting. | Calculator, Clock, Notes, Reminders, Weather, Terminal | +| Native app with the injected service | `OctosAppService`: `open_conversation` for the person's lane, `open_context` for a private context | Rinx | -How the "Ask " panel behaves: +For a script app, the smallest useful integration is two capabilities in the manifest: + +```json +"capabilities": ["octos.session.open", "octos.turn.start"] +``` + +and a few lines of Splash that open the person's lane and send one turn: + +```splash +fn ask(){ + ui.answer.set_text("Waiting for the assistant…") + host.request("octos.session.open", {}, fn(s){ + if !s.is_ok { + ui.answer.set_text("Assistant unavailable: " + s.error) + return + } + host.request("octos.turn.start", {text: ui.prompt.text()}, fn(r){ + if r.is_ok { ui.answer.set_text(r.data.text) } + else { ui.answer.set_text("Assistant unavailable: " + r.error) } + }) + }) +} +``` -- **Consent first.** An agent the person has not decided on shows the first-use sheet, and the panel waits ("'s assistant is not allowed yet: allow it on the sheet."). An agent turned off says so: "'s assistant is off. Turn it on in Setup › Assistant › Approvals." -- **Both lanes, labelled.** The panel follows both lanes and loads their merged history; every row shows its speaker (the person, the system agent, the app's agent). -- **Send** starts a person turn (`TurnTrigger::Person`). It never waits for the system agent's turn: Send stays available while only the system agent's lane runs. With a question from the app's agent open, the text answers it instead. -- **Stop** (in Send's place while the person's own turn runs) stops only that turn ("Stopped.", or "Nothing of yours was running."). A running system agent turn has its own row with **"Stop the system agent's task"**, which stops only that lane. The **"Stop 's agent"** button on the shell's approval and question sheets stops both lanes (`approvals::stop_agent`): the person owns the device. -- **Questions** from turns the person or the app started are shown and answered in the panel; the system agent's go to the system chat (F8). Approvals are the shell's sheets, as everywhere. -- **Close** hides the panel. Its context stays open with its follower, so a reopen shows the person's rows again. The context closes when the panel opens for another app, when the agent is turned off, or when the app's peer goes (a signed-out account, for Mail). +In a shell with a kernel, the first call asks the person to allow the app's agent. Treat "unavailable" as a normal state: the device may have no kernel (iOS) or no provider, or the person may have said no. This example and the rest of the API are in Design Flow's [AI-SERVICES guide](https://github.com/OctoSense-org/OctoScript-App-Design-Flow/blob/main/docs/AI-SERVICES.md#a-minimal-call-and-handling-unavailable). + +### What an app gives its agent + +An agent can only work with what its app hands it. A script app declares all of this in its bundle; a native app declares it in its `native-apps.json` entry. + +- **A declaration.** The manifest's `agent` block names the kernel tools the agent may use (the system apps ask only for `ask_user_question`), the model features it needs (`tool_calling`) and, optionally, an `AGENT.md` with instructions. A native app's entry also says which of its tools its own agent may call (`own_tools`) and which the system agent may call (`system_tools`). +- **Tools.** `tools.json` describes each tool, named `.`: its input schema, its `risk` (`read`, `act` or `destructive`), who confirms it (`confirm: host` for a shell sheet, `app` for the app's own sheet) and whether other apps' agents may use it (`shareable`). +- **Something to run the tools.** A declared tool needs an executor: the app's host service (Mail, Calendar, News), the shell's notice service (`.notify` for the other system apps) or a native app's open window. Store apps have no host service, and a tool marked `implemented_by: "app"` has no executor in the Card runner yet. So today a store app's agent can talk, ask questions and read its folder, but cannot act through tools of its own. +- **Data.** By default the agent works in its account's folder, `apps//accounts//` (a single `device` folder for an app without accounts), and reads it with the host's read-only `files.list`, `files.read` and `files.search` (on Unix). An app can declare `storage.agent_workspace: "none"` instead; Calculator, Clock, Notes, Reminders, Weather and the Terminal do, so their agents see only what their tools return. No agent sees another account's folder. +- **Memory.** Each agent has its own memory namespace, `app//acct-`, erased with the account. +- **A way to reach the person.** With the `glance` permission, its tools can publish cards. +- **Events** (only Mail, for now). A `triggers.events` entry, a skill and `AGENT.md` let Mail's agent react to new mail without being asked; the shell sends the admitted instructions and skill text with each of those turns. + +The steps for adding a tool (manifest, `tools.json`, grant, handler, approval path) are in [AGENTS.md](AGENTS.md#architecture-documentation-and-code-walkthroughs), and the design is [ADR 0002](docs/adr/0002-event-driven-app-agents.md). ### A tool call with an approval @@ -246,62 +298,51 @@ sequenceDiagram -- **Tool calls**: octos sends `peer/tool/call` to the shell's relay (`crates/shell/src/host_tools/`), which checks the grant by (owning app, tool) and caller, the arguments against the tool's schema and the caller's budget, and routes the call to the owning app's executor: an in-process module's, a script app's host service, a process app's peer link, or the Terminal's `run` on the AI bus. A tool the host confirms (`confirm: host`: the Terminal's `run`) stays off an app's peer link when the app also serves it on the AI bus; the host's sheet decides it there. An app's own agent may call the tools its `native-apps.json` entry allows (`agent.own_tools`; every one of its tools unless the entry narrows them): the Terminal's own agent reads its screen and scrollback only, and typing a command stays the system agent's `terminal.run`, behind Setup's switch. -- **Approvals** go to `crates/shell/src/approvals/`: external clients retain their prompts; developer mode approves calls for covered apps; `confirm: app` uses the owning app's registered sheet; mandatory live decisions bypass rules; then standing rules may decide, otherwise a shell sheet asks the person. The system agent cannot approve. [The walkthrough](docs/architecture-walkthrough.md#approval-order) gives the complete order, deadlines and audit behavior. -- **Deadlines and Stop** ([#167](https://github.com/OctoSense-org/OctoSense/pull/167)): an approval or question the shell holds for an app peer expires after 10 minutes (`OCTOSENSE_PROMPT_DEADLINE_SECS`): the router denies it, a question is declined, both stay visible as "Expired: no answer in 10 min". If the turn is still running 30 s later, the broker interrupts it so the next turn can start. The sheets' "Stop 's agent" ends the running turns of both lanes, the person's and the system agent's; the "Ask " panel's Stop ends only the person's own turn ([above](#talking-to-an-apps-agent-yourself)). -- **External clients' prompts** stay with the client: the shell does not answer or expire approvals of a Talk to Octos client's turns (octos#2624). +- **The relay** ([`crates/shell/src/host_tools/`](crates/shell/src/host_tools/)) receives every `peer/tool/call`. It checks that this caller may use this tool, validates the arguments against the tool's schema and charges the caller's budget (by default 32 tool calls a turn and 1000 a day). Only then does it run the tool in the app that owns it: a native app's open window, a script app's host service, or a process app over its peer link. It checks the result against the schema too. +- **The approval router** ([`crates/shell/src/approvals/`](crates/shell/src/approvals/)) decides in a fixed order. Developer mode, which only the person can turn on, approves everything for the apps it covers. A `confirm: app` tool goes to the app's own sheet. Calls that must always ask, such as a Terminal command, go straight to a sheet. Everything else may be decided by the person's standing rules, and otherwise a shell sheet shows the exact arguments. Every decision is written to an audit log. [The walkthrough](docs/architecture-walkthrough.md#approval-order) gives the full order. +- **Deadlines.** An approval or question nobody answers in 10 minutes is denied, never approved. If the turn is still running 30 seconds later, the shell interrupts it so the next turn can start. ### Cards and questions -#### Publishing and opening cards - -An app with the `glance` capability publishes as itself with `glance.publish`; it can also use `glance.withdraw` and `glance.list`. The shell takes the publisher from the caller, never from the arguments. A card is either an L0 `source` filled from `data` (presentation only, checked by Octoscript’s L0 checker) or a Splash `script`. - -The limits are 6 publishes per minute and 4 cards per app. The shell keeps 32 cards, ordered by priority, then recency: a phone’s glance page shows the first 6, and the desktop’s panel lists them all. See [`glance.rs`](crates/shell/src/glance.rs). - -| Surface | What happens | -| --- | --- | -| Desktop panel | A new card opens the glance panel unless a card window is already open. The bar’s bell or F9 also opens the panel. A press on a card outside its own controls opens it in the card window. The hovered card shows its open and dismiss actions, and a card that just came wears an accent mark for a few seconds. A card that does not fit whole peeks in at the end of the list. A card taller than its tile scrolls inside it with the wheel; a card with an in-card chat stays at its newest rows, so its field and the latest exchange stay in view. Opened with F9 (or a press in it), the panel has the keyboard: the arrows move a focus ring from card to card, Return opens the card, Delete dismisses it and Esc closes the panel. | -| Desktop notification | A card published with `notify` also posts a toast with the app’s icon and name, the card’s title and its `summary` (else the card’s own). Selecting the toast opens the card in its own window. At most three toasts show at once; a “+N more” chip under them shows the rest. While the panel is open, toasts stack to its left. | -| Desktop dismissal | The hovered card shows a dismiss button: `glance::dismiss` removes the card as if the app had withdrawn it. Clear all dismisses every card. A dismissal can be undone from the toast that reports it, or with ⌘Z while the panel has the keyboard. The panel has a separate close button. | -| Phone | `notify` posts a shade notification. Selecting it opens the glance page. | - -A card that names no theme takes the shell’s light or dark palette. Toasts and the panel slide in; `OCTOSENSE_REDUCE_MOTION=1` keeps them still. The desktop surfaces are implemented in [`glance_panel.rs`](crates/shell/src/glance_panel.rs), [`glance_sheet.rs`](crates/shell/src/glance_sheet.rs) and [`notifications.rs`](crates/shell/src/shell/notifications.rs) (`keep_clear_of`, [#273](https://github.com/OctoSense-org/OctoSense/pull/273)). - -#### Interactive cards - -A card runs under its app’s own policy, as the app’s UI does in the Card runner. A person’s action on a card passes through the app’s capability gate and host services. It is an app action, so it needs no extra shell approval for an agent tool call ([#153](https://github.com/OctoSense-org/OctoSense/pull/153)). +An app with the `glance` permission publishes cards as itself (`glance.publish`, `glance.withdraw`, `glance.list`); the shell takes the publisher from the caller, never from the arguments. A card runs under its app's own permissions, so a button pressed in a card is the app's own action, not an agent tool call. The desktop README describes [the glance panel](desktop/README.md#the-glance-panel) where cards appear. #### In-card chat -**Current availability:** none of the cards published by app agents on `main` declares a chat. The only shipped card with a chat is [`mail-request.card`](crates/shell/resources/glance/mail-request.card), enabled by `OCTOSENSE_GLANCE_DEMO=mail`; it gives canned replies. +A card can declare `sys.chat(app, thread, fields)`. The person types in the card, and only the publishing app's own agent answers, in the person's lane. The shell owns the transcript, kept in the app's account folder, and records as the person's only what the person typed. Model-written text is marked as AI-written and never runs as an action. No agent-published card declares a chat yet; the demo card [`mail-request.card`](crates/shell/resources/glance/mail-request.card) shows one with canned replies. See [`crates/l0-chat`](crates/l0-chat/README.md). -An L0 card can declare `sys.chat(app, thread, fields)` and draw `ChatEntry` rows. It can also display model-written text (`class: model-copy`), which is marked AI-written and never executed as an action ([#263](https://github.com/OctoSense-org/OctoSense/pull/263)). +#### Questions -The host owns the transcript. Only the publishing app’s own agent answers, in the person’s lane, and only text the person typed is recorded as theirs. Threads are stored in `apps//accounts//chat/.json`. See [`crates/l0-chat`](crates/l0-chat/README.md) and [`glance_chat.rs`](crates/shell/src/glance_chat.rs). +An agent's `ask_user_question` appears where its turn came from: in the app's conversation for the person's and the app's own turns, in the system chat for the system agent's. Only the person answers, on a shell surface. -#### Questions +## Why it stays light on memory and CPU + +A phone runs the shell, the kernel, up to a dozen app agents and their apps at the same time. These design choices keep that cheap: -Questions (`ask_user_question`) follow the turn’s trigger. A turn from the person’s lane or the app asks in the app’s conversation; a turn from the system agent’s lane asks in the system chat. Only the person answers, on a shell surface. +- **One kernel, and agents are sessions.** Every agent lives in the same octos kernel. There, an agent is stored state (its transcript, memory and blackboard entries) plus Tokio tasks while a turn runs. No agent has a process of its own. +- **One connection to the kernel.** The system chat and every app agent's broker share a single stream to the kernel. A small router ([`crates/kernel/src/router.rs`](crates/kernel/src/router.rs)) gives each request a unique id and sends each notification to the consumers of its session. +- **Threads per service, not per turn.** Turns are Tokio tasks. In the shell, the kernel service runs on one two-worker Tokio runtime, each live agent's broker on a one-thread runtime of its own (so a slow agent cannot stall another agent or the UI), and the system chat on one thread that wakes the UI through Makepad's `SignalToUI` only when something changed. The octos child process (desktop and Android) uses Tokio's default runtime, one worker per CPU core. +- **Started on demand, stopped when idle.** The kernel starts on its first connection: at startup if the person has already allowed a script app's agent, otherwise when an agent or the system chat first needs it. The system chat connects only while its pane is open or a turn is running. With Talk to Octos off, the kernel stops when its last connection closes. An app with no granted agent gets no broker and no peer. +- **Apps share the shell process.** Script apps are isolates in one Card runner and native apps are modules, so most apps cost no process of their own. A script error stays inside its isolate, and a native module's panic is caught at the module boundary. Only an app that needs an OS sandbox runs as a process. +- **Zero-copy frames for process apps.** The Terminal's frames reach the shell as shared GPU surfaces: IOSurface on macOS, D3D11 shared handles on Windows, DMA_BUF on Linux with Vulkan. Where only a CPU copy would work (Linux with OpenGL), every app runs inside the shell instead. ## Layout | Path | What it is | | --- | --- | | [`desktop/`](desktop/README.md) | Desktop packaging, package `octosense`: the entry point (`src/main.rs` only), catalogs (`config/apps.json`), the window-manager sync from upstream Makepad (`upstream/`, `scripts/upstream.py`), the desktop's system-app selection. | -| [`phone/`](phone/README.md) | The Home app, package `octosense-home` (APK id `dev.makepad.octosense`): the entry point that wraps the shell (`src/main.rs`), the built-in Settings app (`src/settings_*.rs`, `src/android_settings.rs`, `resources/settings/`), Android, OpenHarmony and iOS packaging, the phone side of the system bridge (`android/`), the phone's system-app selection. | -| [`rom/`](rom/README.md) | The OnePlus 6 ROM image only: `vendor/` (product, privileged permissions, overlays, Settings backends, the privileged agent), `patches/`, image, flash and OTA scripts, the Home APK build scripts, `web-installer/`, product tests. | -| `crates/shell/` | The one shell, package `octosense-shell`, linked by both packages: window manager (desk, styles, tiling, scene), hosting (processes, in-process modules, App Hub, the AI pane), the phone layer (home pages, shade, gestures, the Android launcher bridge), themes, wallpapers and icons (`resources/`). | -| [`crates/ai-host/`](crates/ai-host/README.md) | The shell's AI services behind one entry point, package `octosense-ai-host`: the octos kernel service, the `llm` host service with the platform's QR import, the `model` service (`model.complete`), the `octos` host service that gives each script app its agent (`card.`), and native apps' assistant access. | -| [`crates/kernel/`](crates/kernel/README.md) | The octos kernel service, package `octosense-kernel`: the [octos](https://github.com/octos-org/octos) agent kernel as a shell service, one per process, configured by AI providers and shared by its consumers; the system agent's exact tool list. | -| [`crates/app-peers/`](crates/app-peers/README.md) | The app-agent broker, package `octosense-app-peers`: one host-owned octos peer per (app, account), its two lanes, its tools, `peer/input`, deadlines, purge ([Rinx ADR 0007](https://github.com/hagency-org/Rinx/blob/main/docs/adr/0007-host-owned-octos-app-peers.md)). | -| [`crates/l0-chat/`](crates/l0-chat/README.md) | The host side of an L0 card's in-card chat (`sys.chat`), package `octosense-l0-chat`, shared by the shell's glance cards and AppCard. | -| [`crates/toolbox/`](crates/toolbox/README.md) | The system toolbox, package `octosense-toolbox`: workflow templates, their runner, forks and evaluation, and `mod.research`, offered to app agents as host tools behind the `toolbox-peers` feature. | -| [`apps/`](apps/README.md) | The system apps (News, Photos, Maps, Camera, Mail, Calendar, AI providers, YouTube) as contained script apps, their host services (`mail`, `calendar`, `news`, `llm`), `apps/reference`, and the opt-in AppCard assistant (`apps/appcard`). | -| `tools/` | `setup.py` (the pinned framework sources), the reviewed Makepad runtime patch (`runtime-patches/`), `kernel-artifact.py` (the octos kernel an Android APK bundles as `liboctos.so`), `check-shell-graph.sh` (the dependency-graph guards every shell build passes). | -| [`docs/adr/`](docs/adr/README.md) | Architecture decisions: this repository's, and the Home decisions 0001–0006 kept as history. | -| `Cargo.toml`, `Cargo.lock` | One workspace. Every external dependency is pinned once in `[workspace.dependencies]`. | -| `native-runtime.lock.json`, `runtime-patches.lock.json` | The OctoScript-Makepad release (and through it Makepad and OctoScript), and the reviewed patch on top of Makepad. | +| [`phone/`](phone/README.md) | The Home app, package `octosense-home` (APK id `dev.makepad.octosense`): the entry point, the built-in Settings app, Android, OpenHarmony and iOS packaging, the phone side of the system bridge (`android/`), the phone's system-app selection. | +| [`rom/`](rom/README.md) | The OnePlus 6 ROM image only: `vendor/`, `patches/`, image, flash and OTA scripts, the Home APK build scripts, `web-installer/`, product tests. | +| `crates/shell/` | The shell, package `octosense-shell`: window manager, app hosting, the system chat and "Ask <app>" panel, the host-tool relay and approval router, the phone layer, themes, wallpapers and icons. | +| [`crates/ai-host/`](crates/ai-host/README.md) | The shell's AI services behind one entry point, package `octosense-ai-host`: the kernel service, the `llm` and `model` services, and the `octos` host service that gives each script app its agent (`card.`). | +| [`crates/kernel/`](crates/kernel/README.md) | The octos kernel as a shell service, package `octosense-kernel`: one per process, configured by AI providers, shared by every consumer; the system agent's exact tool list. | +| [`crates/app-peers/`](crates/app-peers/README.md) | The app-agent broker, package `octosense-app-peers`: one peer per (app, account), its two lanes, its tools, `peer/input`, deadlines, purge. | +| [`crates/l0-chat/`](crates/l0-chat/README.md) | The host side of an L0 card's in-card chat (`sys.chat`), package `octosense-l0-chat`. | +| [`crates/toolbox/`](crates/toolbox/README.md) | The system toolbox, package `octosense-toolbox`: workflow templates and `mod.research`, offered to app agents behind the `toolbox-peers` feature. | +| [`apps/`](apps/README.md) | The system apps (News, Photos, Maps, Camera, Mail, Calendar, AI providers, YouTube) as script apps, their host services (`mail`, `calendar`, `news`, `llm`), `apps/reference`, and the opt-in AppCard assistant. | +| [`native-apps.json`](native-apps.json) | Every native app: where its code comes from, how it is hosted on each platform, and its agent. `tools/native_apps.py` generates the shell's code and Cargo entries from it. | +| `tools/` | `setup.py` (the pinned framework sources), the reviewed Makepad runtime patch (`runtime-patches/`), `kernel-artifact.py` (the octos kernel build), `check-shell-graph.sh` (dependency-graph guards). | +| [`docs/`](docs/architecture.md) | The architecture, the code walkthroughs and the [ADRs](docs/adr/README.md). | +| `Cargo.toml`, `Cargo.lock`, `native-runtime.lock.json`, `runtime-patches.lock.json` | One workspace. Every external dependency is pinned once; the runtime locks name the OctoScript-Makepad release and the reviewed patch on top of Makepad. | The shell exists once, in `crates/shell` ([ADR 0001](docs/adr/0001-one-octosense-repository.md)): desktop and phone differ by target and features, not by copies of the source. CI fails if a shell source file appears in two crates. @@ -311,22 +352,14 @@ Pinned exactly once, in the root `Cargo.toml` and the runtime locks: | Repository | Role | | --- | --- | -| [makepad (OctoSense fork)](https://github.com/OctoSense-org/makepad) | The UI framework and the `cargo-makepad` packager. Checked out in `.sources/makepad`, plus the reviewed runtime patch. | +| [makepad (OctoSense fork)](https://github.com/OctoSense-org/makepad) | The UI framework, the `cargo-makepad` packager and the native apps Calculator, Clock, Notes, Reminders, Weather, Terminal, Sheets and Task. Checked out in `.sources/makepad`, plus the reviewed runtime patch. | | [OctoScript-Makepad](https://github.com/OctoSense-org/OctoScript-Makepad), [OctoScript](https://github.com/OctoSense-org/OctoScript) | The runtime release that names the Makepad and OctoScript revisions (`native-runtime.lock.json`). | -| [OctoSense-App-Hub](https://github.com/OctoSense-org/OctoSense-App-Hub) | The signed catalog, the store, the Card runner that contains every app (`octosense-app-hub-app`). | -| [octos](https://github.com/octos-org/octos) | The agent kernel. On Android the APK bundles it as `liboctos.so`; on a desktop the kernel service runs the packaged `octos-kernel` beside the shell, checked against this revision (`tools/kernel-artifact.py --host --stage` builds it); `OCTOS_APP_CORE_BIN` overrides it. | +| [OctoSense-App-Hub](https://github.com/OctoSense-org/OctoSense-App-Hub) | The signed catalog, the store, and the Card runner that contains every script app. | +| [octos](https://github.com/octos-org/octos) | The agent kernel. Android bundles it in the APK as `liboctos.so`; the desktop runs the packaged `octos-kernel` beside the shell, checked against this revision (`tools/kernel-artifact.py --host --stage` builds it). | | [Rinx](https://github.com/hagency-org/Rinx) | Matrix chats and mini apps, hosted as a native module. | Related, not build inputs: [OctoScript-App-Design-Flow](https://github.com/OctoSense-org/OctoScript-App-Design-Flow) (how apps are built and published), [OctoScript-Android](https://github.com/OctoSense-org/OctoScript-Android) and [OctoScript-OH](https://github.com/OctoSense-org/OctoScript-OH) (other renderers), the [OctoSense website](https://github.com/OctoSense-org/octosense-org.github.io). -## AI services (octos) - -Each shell runs one [octos](https://github.com/octos-org/octos) agent kernel, started on first use: the APK's `liboctos.so` on Android, in process on OpenHarmony, on a desktop the packaged `octos-kernel` beside the shell (or the binary `OCTOS_APP_CORE_BIN` names), none on iOS. The person chooses its models and types keys in the **AI providers** system app, on host sheets; keys stay in the platform's secret store and never reach an app. [`crates/ai-host`](crates/ai-host/README.md) is the shells' one entry point, and [`crates/app-peers`](crates/app-peers/README.md) gives each granted native app its own octos peer (private contexts, workspace and memory `app//acct-`), owned by the shell's system agent. Peer tool approvals use the shell router: developer mode, eligible standing rules or a person on the owning app/Shell confirmation sheet. The system agent cannot answer them. - -What works today: native modules (Rinx) use their peer; AppCard (opt-in) uses the kernel directly. Contained script apps, system or store, reach it through the `octos` host service in a shell that hosts a kernel: each app gets its own host-owned peer (`card.`), and its tool approvals go to the shell's approval sheets like every other app agent's ([#155](https://github.com/OctoSense-org/OctoSense/pull/155)). The `llm` service manages providers for `os.*` apps only. An app's own agent (`tools.json`, `AGENT.md`, skills, triggers, glance cards) is [ADR 0002](docs/adr/0002-event-driven-app-agents.md); host-service-backed `tools.json` tools reach their agents through the shell relay since [#160](https://github.com/OctoSense-org/OctoSense/pull/160). Tools marked `implemented_by: "app"` still lack a Card-runner executor; declaring a tool does not implement it. - -The architecture, the trust model, what each kind of app can use, the plan with its status, and how to run and test it locally: [docs/ai-services.md](docs/ai-services.md). How it fits into the whole system: [docs/architecture.md](docs/architecture.md). For app developers: OctoScript-App-Design-Flow's [AI-SERVICES](https://github.com/OctoSense-org/OctoScript-App-Design-Flow/blob/main/docs/AI-SERVICES.md). - ## Set up Stable Rust (`cargo` in `~/.cargo/bin`), Git, Python 3.9+ (3.11 for `desktop/scripts/upstream.py`) and, on macOS, the Xcode Command Line Tools. Makepad and OctoScript resolve to checkouts in `.sources/` (git-ignored) that the setup script prepares at the pinned revisions: @@ -340,13 +373,15 @@ python3 tools/setup.py --check --cargo # verify: one Makepad, App Hub, octos an `--update` moves clean checkouts after the locks change; `--cache DIR` borrows Git objects from existing clones (`DIR/makepad`, `DIR/octoscript`, `DIR/octoscript-makepad`). Local changes in `.sources/` are preserved. -**Already have clones of these repositories?** Keep one clone of each on the machine and make every `.sources/` entry a `git worktree` of it, so there is one object store per repository and no stale copy. Name the directory that holds the clones (as `/makepad`, `/octoscript`, `/octoscript-makepad`) once, in `~/.config/octosense/sources.json`: +
Already have clones of these repositories? Use them as a hub. + +Keep one clone of each on the machine and make every `.sources/` entry a `git worktree` of it, so there is one object store per repository and no stale copy. Name the directory that holds the clones (as `/makepad`, `/octoscript`, `/octoscript-makepad`) once, in `~/.config/octosense/sources.json`: ```json { "hub": "/path/to/clones" } ``` -or per run with `--hub DIR` or `OCTOSENSE_SOURCES_HUB=DIR`; `OCTOSENSE_MAKEPAD_HUB=CLONE` (and `_OCTOSCRIPT_`, `_OCTOSCRIPT_MAKEPAD_`) names one clone, as does `"repositories": {"makepad": "CLONE"}` in the file. Setup then fetches each pinned revision into that clone and runs `git worktree add --detach .sources/ ` instead of cloning; `--update` moves the worktrees. Without a hub (CI, a fresh machine) it clones as before, and `--no-hub` forces that. A `.sources/` entry that is already a full clone is reported, not deleted; `--convert` replaces it with a worktree when it holds no local work. +or per run with `--hub DIR` or `OCTOSENSE_SOURCES_HUB=DIR`. Setup then fetches each pinned revision into that clone and runs `git worktree add --detach .sources/ ` instead of cloning; `--update` moves the worktrees. Without a hub (CI, a fresh machine) it clones as before, and `--no-hub` forces that. A `.sources/` entry that is already a full clone is reported, not deleted; `--convert` replaces it with a worktree when it holds no local work. Before deleting a checkout of this repository, remove its `.sources/` worktrees so the clones keep no stale entries: @@ -355,7 +390,7 @@ python3 tools/setup.py --remove-worktrees # git worktree remove + prune in eac git worktree remove # if it is itself a worktree ``` -By hand, the same is `git -C worktree remove --force .sources/` (the reviewed Makepad patch is staged, hence `--force`; check `git status` first) and `git -C worktree prune`. +
## Build @@ -367,7 +402,7 @@ cargo check --locked -p octosense --features mobile-apps cargo check --locked -p octosense -p octosense-appcard --features mobile-apps,app-appcard ``` -The assistant needs the octos kernel beside the shell: `python3 tools/kernel-artifact.py --host --stage target/release` builds the pinned revision and stages it, once per octos pin; the desktop refuses a staged kernel of another revision and says so ([Build and run](desktop/README.md#build-and-run)). Without one the desktop runs without an assistant. +The assistant needs the octos kernel beside the shell: `python3 tools/kernel-artifact.py --host --stage target/release` builds the pinned revision and stages it, once per octos pin. The desktop refuses a staged kernel of another revision and says so ([Build and run](desktop/README.md#build-and-run)). Without one, the desktop runs without an assistant. **Phone** (from `phone/`, which selects the phone's system apps; details in [phone/README.md](phone/README.md)): @@ -380,29 +415,27 @@ python3 ../rom/scripts/build-home.py --help # the Home and B **ROM image** (Linux build host, external LineageOS tree; not in CI): [rom/README.md](rom/README.md). -Hosted apps and UI tests run with hidden windows and a local control surface: `MAKEPAD_HIDE_WINDOWS=1 MAKEPAD_REMOTE=` (routes under `/help`). - ## CI Path-filtered workflows in `.github/workflows/`, so a change runs only the jobs its paths need: | Workflow | Runs for | Checks | | --- | --- | --- | -| `desktop.yml` | `desktop/`, `crates/`, `apps/`, the workspace files, `tools/` | compiles the desktop (default, `mobile-apps`, `mobile-apps,app-appcard`), the shell graph guards (`tools/check-shell-graph.sh`), one copy of every shell source, the `tools/` tests | -| `phone.yml` | `phone/`, `crates/`, `apps/`, the workspace files, `tools/` | compiles Home and its bundled modules, the shell graph guards, and runs the tests of the shell, Home, the AI services, App Hub admission and runtime policy on macOS; the longest job | -| `apps.yml` | `apps/`, `crates/`, the workspace files, `tools/setup.py` | the kernel service, app peers, AI providers config, the Mail and `llm` host services, the shell's AI services (`crates/ai-host`), AppCard | +| `desktop.yml` | `desktop/`, `crates/`, `apps/`, the workspace files, `tools/` | compiles the desktop (default, `mobile-apps`, `mobile-apps,app-appcard`), the shell graph guards, one copy of every shell source, the `tools/` tests | +| `phone.yml` | `phone/`, `crates/`, `apps/`, the workspace files, `tools/` | compiles Home and its bundled modules, the shell graph guards, and runs the tests of the shell, Home, the AI services, App Hub admission and runtime policy on macOS | +| `apps.yml` | `apps/`, `crates/`, the workspace files, `tools/setup.py` | the kernel service, app peers, AI providers config, the Mail and `llm` host services, the shell's AI services, AppCard | | `rom.yml` | `rom/`, `phone/android/`, the phone's Android resources and tests, `tools/kernel-artifact.py` | product tests, the generated Agent Binder client, the web installer | -| `release-desktop.yml` | a pushed `desktop-v*` tag, a manual run, or a pull request that changes the packaging (build and scan only) | unsigned desktop packages for macOS, Windows and Linux, the private-path scan; for a tag, signing in the `release` environment and a draft release ([desktop/README.md](desktop/README.md#release-builds)). Not run by `tools/ci-local.sh`. | +| `release-desktop.yml` | a `desktop-v*` tag, a manual run, or a pull request that changes the packaging | unsigned desktop packages for macOS, Windows and Linux and the private-path scan; for a tag, signing and a draft release ([desktop/README.md](desktop/README.md#release-builds)) | -Each workflow's graph check (`tools/setup.py --check --cargo`) asserts one Makepad, one App Hub, one octos and one Rinx in the locked graph. +Each workflow's graph check (`tools/setup.py --check --cargo`) asserts one Makepad, one App Hub, one octos and one Rinx in the locked graph. `tools/ci-local.sh` runs the same steps on your own machine ([docs/local-ci.md](docs/local-ci.md)). ## Releases -ADR 0001 tags each product on its own: `desktop-v*`, `home-v*` (APK), `rom-v*` (image), with build receipts that record the repository commit. A `desktop-v*` tag builds the desktop packages (`.dmg`, Windows installer, `.deb`, `.AppImage`) into a draft release ([Release builds](desktop/README.md#release-builds)). System apps ship only inside the shells, admitted by digest; they are not released separately. The ROM release published before the merge, `20260919-j`, is here as [`rom-v20260919-j`](https://github.com/OctoSense-org/OctoSense/releases/tag/rom-v20260919-j). Phones read `update.json` from the moving `rom-latest` release, not from `releases/latest` ([rom/docs/updates.md](rom/docs/updates.md)). Images `20260919-j` and earlier check the retired OctoSense-ROM repository instead, so a phone flashed with one must be reflashed once to receive updates over the air. +Each product is tagged on its own: `desktop-v*`, `home-v*` (the APK) and `rom-v*` (the image), with build receipts that record the repository commit. A `desktop-v*` tag builds the desktop packages (`.dmg`, Windows installer, `.deb`, `.AppImage`) into a draft release ([Release builds](desktop/README.md#release-builds)). System apps ship only inside the shells, admitted by digest. [rom/docs/updates.md](rom/docs/updates.md) explains how phones find ROM updates. ## Contributing -`main` is protected: every change goes through a pull request, and force pushes are blocked. One change is one pull request, across `desktop/`, `phone/`, `crates/` and `apps/` as needed; there are no internal pins to move. Rules for people and coding agents are in [AGENTS.md](AGENTS.md). +`main` is protected: every change goes through a pull request, and force pushes are blocked. One change is one pull request, across `desktop/`, `phone/`, `crates/` and `apps/` as needed. Rules for people and coding agents are in [AGENTS.md](AGENTS.md). ## License diff --git a/README.zh-CN.md b/README.zh-CN.md index fa0d92a5..f10b6329 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -2,27 +2,41 @@ [English](README.md) | 简体中文 -[OctoSense](https://github.com/OctoSense-org) 是运行在操作系统之上的 Agent 交互 Shell：启动器和应用看起来与你熟悉的一样，背后由系统 Agent 协调各应用 Agent。本仓库集中存放 OctoSense 自己的全部代码（[ADR 0001（英文）](docs/adr/0001-one-octosense-repository.md)）：Shell、Shell 服务、第一方系统应用，以及由它们构建的三个产品。 +OctoSense 是运行在普通操作系统之上的 Agent Shell。从屏幕上看，它就是你熟悉的启动器和应用；在它们背后，一个 AI 内核运行着为用户做事的**系统 Agent**，以及每个应用各自的**应用 Agent**。Agent 要动用应用、有风险的操作要找用户，都只能经过 Shell。 + +本仓库存放 Shell、Shell 服务、系统应用，以及由它们构建的三个产品。大多数原生应用来自其他仓库：OctoSense 的 Makepad fork、App Hub 和 Rinx（见[依赖](#依赖)）。 | 产品 | 是什么 | 位置 | | --- | --- | --- | -| **OctoSense 桌面端** | 在 macOS 上作为一个 Makepad 窗口运行的 Shell（Windows 和 Linux 未经测试）：启动器、dock、平铺窗口、托管应用 | [`desktop/`](desktop/README.zh-CN.md) | +| **OctoSense 桌面端** | 在 macOS 上作为一个 Makepad 窗口运行的 Shell（Windows 和 Linux 未经测试） | [`desktop/`](desktop/README.zh-CN.md) | | **OctoSense Home** | 手机 Shell，可作为普通 Home 应用安装在任意 Android 手机上（也支持 OpenHarmony 和 iOS 模拟器） | [`phone/`](phone/README.zh-CN.md) | | **OctoSense ROM** | 面向 OnePlus 6 的 LineageOS 22.2，预装 Home、具有系统权限的系统桥、Quickstep 和 SystemUI | [`rom/`](rom/README.zh-CN.md) | -本仓库原名 OctoSense-Desktop；OctoSense-ROM（已停用，并入本仓库）和 OctoSense-System-Apps 已于 2026-09-27 连同历史一起导入本仓库。OctoSense-System-Apps 已归档；OctoSense-ROM 仓库已不存在。 - -> **要开发 OctoSense 应用？** 开发、检查或发布应用都不需要本仓库。请从 [OctoSense-org 主页](https://github.com/OctoSense-org)的阅读列表开始：[OctoScript-App-Design-Flow](https://github.com/OctoSense-org/OctoScript-App-Design-Flow)（先读 `AGENTS.md`，再读 `docs/QUICKSTART.md`）和 [OctoSense-App-Hub](https://github.com/OctoSense-org/OctoSense-App-Hub)。[`apps/`](apps/README.zh-CN.md) 中的系统应用是同样应用结构的完整示例（`apps//bundle/`）。只有想在发布前先在 Shell 里看到自己的应用时，才需要从这里构建桌面端 Shell（[PUBLISHING §4](https://github.com/OctoSense-org/OctoScript-App-Design-Flow/blob/main/docs/PUBLISHING.md#4-rehearse-the-store-path-locally)）。 +> **要开发 OctoSense 应用？** 开发、检查或发布应用都不需要本仓库。请从 [OctoScript-App-Design-Flow](https://github.com/OctoSense-org/OctoScript-App-Design-Flow)（先读 `AGENTS.md`，再读 `docs/QUICKSTART.md`）和 [OctoSense-App-Hub](https://github.com/OctoSense-org/OctoSense-App-Hub) 开始。[`apps/`](apps/README.zh-CN.md) 中的系统应用就是完整的示例。只有想在发布前先在 Shell 里试用自己的应用时，才需要从这里构建桌面端 Shell（[PUBLISHING §4](https://github.com/OctoSense-org/OctoScript-App-Design-Flow/blob/main/docs/PUBLISHING.md#4-rehearse-the-store-path-locally)）。 -## 从代码导读开始 +## 关键概念 -初次阅读源码？从[从应用窗口到 Agent 回合](docs/architecture-walkthrough.zh-CN.md)开始，沿着入口、原生/脚本托管、数据访问、人类/系统对话、工具路由以及实际 Tokio 任务阅读源码。[产品导读](desktop/docs/code-walkthrough.zh-CN.md)补充桌面、Home、ROM 和系统应用的运行方式。 +| 术语 | 在这里的含义 | +| --- | --- | +| **octos** | 用 Rust 编写的开源 Agent 内核（[octos-org/octos](https://github.com/octos-org/octos)）。它运行 Agent **会话**：一段与模型的对话，带有自己的工具、记忆和工作区文件夹。一个会话可以拥有其他会话，称为 **peer**。OctoSense 每台设备运行一个 octos。 | +| **OUP** | octos UI 协议：octos 与其客户端之间的 JSON-RPC 2.0 消息（`octos-ui/v1alpha1`），通常经由内核的 stdin 和 stdout 传递。 | +| **Shell** | 设备上唯一的 OctoSense 进程。它绘制全部界面、托管应用，并持有与 octos 之间唯一的完整连接。桌面端和手机端构建的是同一个 crate：`crates/shell`。 | +| **系统 Agent** | 用户的助手：一个 octos 会话，拥有并监督所有应用 Agent。 | +| **应用 Agent** | 对应某个应用、某个账号的一个 octos peer，所以登录了两个账号的邮件应用有两个 Agent。每个 Agent 有自己的记忆、对话记录、模型和工具列表，通常还有一个工作用的文件夹。 | +| **通道** | 应用 Agent 并行进行的两段对话之一：系统 Agent 的通道，以及用户的通道；应用自己发起的回合也走用户的通道。 | +| **原生应用** | 用 Makepad 构建、登记在 [`native-apps.json`](native-apps.json) 中的 Rust 应用。它作为模块运行在 Shell 内，或作为独立的沙箱进程运行（桌面端的 Terminal）。 | +| **脚本应用** | 一个 OctoScript 应用包：manifest、用 Splash（Makepad 的界面脚本语言）编写的界面，以及可选的、写在 `tools.json` 里的 Agent 工具。它只能通过 `host.request` 访问 Shell。系统应用（邮件、日历、新闻等）和所有商店应用都是脚本应用。 | +| **Card runner** | App Hub 运行脚本应用的环境，位于 Shell 进程内：每个应用实例一个隔离的脚本虚拟机，有自己的文件 jail 和配额。 | +| **宿主服务** | Shell 中的 Rust 代码，负责应答某一族 `host.request` 调用（`mail.*`、`calendar.*`），并执行该应用的 Agent 工具。 | +| **peer link** | 除 Rinx 外，原生应用经 Shell 与自己的 Agent 通信的方式：使用 Makepad 的 `OctosPeer` 客户端；应用是进程时走它的 hub 连接，是模块时走一条内存中的通道。 | +| **glance 卡片** | 应用或它的 Agent 发到 glance 面板（桌面端）或 glance 页面（手机端）上的小卡片。它要么是 L0 卡片（只有声明：数据由宿主填入，没有表达式，也没有调用），要么是 Splash 脚本。 | +| **审批路由** | Shell 中决定一次工具调用是直接运行、需要用户确认还是被拒绝的代码。 | + +想按顺序读源码，请从[从应用窗口到 Agent 回合](docs/architecture-walkthrough.zh-CN.md)开始。[产品导读](desktop/docs/code-walkthrough.zh-CN.md)补充了各个产品的运行方式。 ## 整体如何运作 -每台设备一个 Shell 进程，每个 Shell 一个 octos 内核，每个 Agent 都是这个内核中的一个会话。应用 Agent 通过 Shell broker 访问内核。Shell 持有宿主连接和宿主 token，把每个 Agent 工具调用转交给拥有该工具的应用，并通过开发者模式、用户预设的常设规则或确认面板处理审批。下文的可选 AppCard 原型绕过应用 peer broker，直接访问共享内核服务。完整说明，包括代码路径以及哪些已在 `main` 上、哪些还在规划中：[docs/architecture.zh-CN.md](docs/architecture.zh-CN.md)；相关决策：[ADR 0004（英文）](docs/adr/0004-native-apps-hosting-and-peers.md)。 - -### 进程与连接 +每台设备一个 Shell 进程，每个 Shell 一个 octos 内核，每个 Agent 都是这个内核中的一个会话。Shell 是内核唯一的完整客户端：它启动 octos 并持有宿主 token，启动每个应用 Agent 的回合，转交每一次对应用工具的调用，并掌管所有审批。应用从不直接与内核通信。 ![OctoSense processes and connections](docs/images/agents-processes.png) @@ -34,7 +48,7 @@ flowchart LR ext["Talk to Octos 客户端
网页或终端，需手动开启"] subgraph shellp["OctoSense Shell 进程"] ui["窗口管理器、启动器、
系统对话、Ask app 面板、面板"] - mods["原生模块
App Hub、Rinx"] + mods["原生模块
App Hub、Rinx、Notes、Clock 等"] runner["Card runner
脚本应用、glance 卡片"] aihost["ai-host + app-peers 代理
宿主连接"] relay["宿主工具中转"] @@ -50,7 +64,7 @@ flowchart LR ui --- mods ui --- runner ui -->|"用户的回合：
Ask app、卡内对话"| aihost - mods -->|"OctosAppService"| aihost + mods -->|"OctosPeer、OctosAppService"| aihost runner -->|"host.request octos.*"| aihost term <-->|"hub：画面、AI bus"| ui term -.->|"peer link"| aihost @@ -65,51 +79,53 @@ flowchart LR -- **Shell**（`crates/shell`，一个进程）承载窗口管理器、原生模块（App Hub、Rinx）、App Hub 的 Card runner（每个脚本应用都在自己的隔离环境中）、系统对话、审批路由、宿主工具中转，以及 [`crates/ai-host`](crates/ai-host/README.md)；其中的 [app-peers 代理](crates/app-peers/README.md)就是内核的宿主连接。 -- **octos 内核**（[`crates/kernel`](crates/kernel/README.zh-CN.md)）首次使用时启动：桌面端（Shell 旁随附的 `octos-kernel`，或 `OCTOS_APP_CORE_BIN`）和 Android（`liboctos.so`）上是通过 stdio 讲 OUP 的子进程，OpenHarmony 上是进程内的任务，iOS 上没有。它随 Shell 一起退出。 -- **进程应用**：桌面端的 Terminal 作为独立进程运行，通过 Shell 的 hub 连接（画面和 AI bus），运行在按其 `native-apps.json` 条目构建的系统沙箱中（macOS 上是 Seatbelt，Linux 上是 Landlock 和 seccomp，Windows 上尚未实现）。进程应用通过 **peer link** 使用自己的 Agent；Shell 一侧已在 `main` 上，但 Terminal 没有被授予 Agent，所以目前还没有进程应用使用它。 -- **外部客户端**：Talk to Octos（需手动开启）让网页或终端客户端以受限的外部 token 使用系统对话：只有一份方法白名单，不能调用任何 `peer/*` 方法，不能进入任何应用 Agent 的会话，也拿不到宿主路由的工具。 - -### 进入 octos 的每条路径都经过 Shell +- **Shell** 承载窗口管理器、原生应用、运行脚本应用的 Card runner、系统对话、审批路由和宿主工具中转。它的 AI 部分是 [`crates/ai-host`](crates/ai-host/README.md)，其中的 [app-peers 代理](crates/app-peers/README.md)负责驱动每个应用 Agent。 +- **octos 内核**（[`crates/kernel`](crates/kernel/README.zh-CN.md)）在第一个连接到来时启动，随 Shell 一起退出。桌面端和 Android 上，它是通过 stdio 讲 OUP 的子进程；OpenHarmony 上，它在 Shell 进程内运行；iOS 上没有内核。用户在 **AI providers** 应用中、在宿主面板上选择模型并输入密钥。密钥留在 Shell 一侧（macOS 上存入钥匙串，其他平台存入仅所有者可读的文件），永远不会到达应用。 +- **进程应用**在 Shell 之外运行，处在系统沙箱中（macOS 上是 Seatbelt，Linux 上是 Landlock 和 seccomp，Windows 上尚未实现）。目前只有桌面端的 Terminal 是进程应用。它通过 Shell 的本地 hub 发送画面，并通过同一连接上的 peer link 使用自己的 Agent。 +- **外部客户端**（Talk to Octos，需手动开启）可以凭受限的 token 从浏览器或终端使用系统对话，但拿不到任何应用 Agent、`peer/*` 方法或宿主工具。 -| 谁 | 路径 | 状态 | -| --- | --- | --- | -| 进程内原生模块 | 与进程应用相同的 peer link，经 Makepad 的 `OctosPeer` 客户端（模块宿主把链接归给打开它的实例） | 已在 `main` 上；尚无模块使用 | -| 进程内原生模块（Rinx） | 注入的 `OctosAppService`：`open_conversation`（应用与自己 Agent 的对话）和 `open_context`（每个客户端一个请求上下文，例如 Rinx 小程序） | 已在 `main` 上 | -| 脚本应用及其卡片 | 向 `octos` 宿主服务发送 `host.request("octos.session.open" / "octos.session.history" / "octos.turn.start" / "octos.turn.interrupt")`，限于其 manifest 声明的名称；卡片的卡内对话（`sys.chat`）经过 Shell | 已在 `main` 上，需用户首次使用时同意（发布时的默认闸门；`OCTOSENSE_CONTAINED_APPS=1` 跳过首次使用同意，`0` 关闭）。如今带 Agent 的系统应用都没有声明 `octos.*`：它们的 Agent 由 Shell 驱动 | -| 进程应用 | 其 hub 连接上的 peer link（`octos.session.open`、`octos.turn.start` 等），身份由 Shell 标注 | Shell 一侧已在 `main` 上；尚无进程应用被授予 Agent | -| 系统 Agent | 内核自己的会话 `_main:api:octosense#system`，从 Shell 的系统对话进入 | 已在 `main` 上 | -| Talk to Octos 客户端 | 只能用系统对话，使用外部 token | 已在 `main` 上 | +可选的 AppCard 原型是唯一的例外：它不经过代理，而是自己打开内核连接。想看带代码路径的完整说明，请读 [docs/architecture.zh-CN.md](docs/architecture.zh-CN.md)；信任模型以及如何在本地测试 AI 服务，见 [docs/ai-services.zh-CN.md](docs/ai-services.zh-CN.md)；背后的决策见 [ADR 0004（英文）](docs/adr/0004-native-apps-hosting-and-peers.md)。 ### 系统 Agent 与应用 Agent -**系统 Agent** 是 `_main` profile 上的内核会话 `_main:api:octosense#system`（`crates/kernel/src/network.rs`）。它拥有并监督所有应用 Agent。 +**系统 Agent** 是 octos 会话 `_main:api:octosense#system`。用户在系统对话中与它交谈：桌面端按 F8 或点 Dock 上的 Assistant 图标，手机上点 Assistant 磁贴。它有两组工具： -用户在 Shell 的**助手面板**（即系统对话，`crates/shell/src/system_chat/`：Dock 上的 Assistant 图标、手机主屏的 Assistant 磁贴或 F8；桌面端是左侧一个中等大小、可移动、可调整大小并渲染 Markdown 的面板，手机上是全屏）或已配对的 Talk to Octos 客户端中与它对话。 +- **它自己的内核工具**，即 [`SYSTEM_AGENT_TOOLS`](crates/kernel/src/system_tools.rs) 中固定的列表：用于监督应用 Agent 的 `peer_*` 工具，以及自己工作区里的文件、记忆、向用户提问和网页搜索。octos 自带的 shell 工具永远不在其中。 +- **Shell 为它注册的宿主工具**：`agents.list` 和 `agents.ask` 用来查找应用 Agent、请用户允许某个 Agent；`agents.provision` 和 `agents.status` 用来运行邮件的新邮件自动处理；Setup 中的 Command execution 开关打开时还有 `terminal.run`；以及原生应用共享给它的只读工具（见下表最后一列）。 -它的内核工具恰好是 `SYSTEM_AGENT_TOOLS`（`crates/kernel/src/system_tools.rs`）：用于监督应用 Agent 的 `peer_send_input`、`peer_gather`、`peer_list` 和 `peer_respond`，其工作区的文件工具，记忆，`ask_user_question`，媒体查看，`web_search`、`web_fetch` 和 `tool_search`。每次内核启动都用 octos 的 `session/tool_list/set` 设置这份列表，所以 octos 自带的 shell、spawn 一族和 `peer_close` 永远不会提供给它。 +系统 Agent 永远不能批准工具调用，只有用户可以。 -系统对话还会在这个会话上注册宿主工具：`agents.list` 和 `agents.ask`（`crates/shell/src/agents.rs`：哪些应用有 Agent，以及某个 Agent 的首次使用面板与就绪等待），在 Setup › Assistant › Command execution 打开期间的 `terminal.run`，以及原生应用在 `native-apps.json` 条目中列出的只读工具（`agent.system_tools`）：Calculator 的 `calculator.eval`、Clock 的 `clock.now`、Notes 的 `notes.search` 和 `notes.read`、Reminders 的 `reminders.due` 和 `reminders.list`、Weather 的 `weather.current`。这样的调用会到达该应用已打开的实例（它的 AI bus 服务）；应用未打开时会回答“Open Notes first”。 +**应用 Agent** 要等用户允许后才会存在，每个应用只问一次。它何时运行取决于应用的类型： -**应用 Agent** 是每个（应用，账号）一个由宿主拥有的 octos peer（[`crates/app-peers`](crates/app-peers/README.md)）。`main` 上以下应用有 Agent（`crates/shell/src/apps.rs`，`agent_apps`）： +- **脚本应用**的 Agent 在 Shell 启动时就准备好，并注册了应用的工具，所以即使应用没有打开，系统 Agent 的 `peer_list` 也能看到它。 +- **原生应用**的 Agent 属于应用已打开的窗口，只在应用打开期间运行；它的记忆和对话记录在两次打开之间会保留。它的工具也在那个窗口里运行：Notes 没打开时，调用会回答 “Open Notes first”。 -| 应用 | 声明方式 | 它的工具（由谁执行） | 它放到 glance 屏幕上的内容 | +退出登录会保留 Agent；删除账号或卸载应用会清除它的对话记录和记忆。 + +以下应用有 Agent： + +| 应用 | 类型 | 它的 Agent 自己的工具 | 系统 Agent 可以调用 | | --- | --- | --- | --- | -| Rinx（原生） | `native-apps.json` 的 `agent.octos`（四个 `octos.*` 服务）和 `agent.generic_tools` | 其列表中的 octos 通用工具；它自己的助手界面 | – | -| 新闻（`os.news`） | `apps/news/bundle/tools.json`、manifest 中的 `agent` 块 | `news.list`、`news.read`、`news.notify`（`news` 宿主服务与 Shell 通知回调） | 通知卡片与通知 | -| 邮件（`os.mail`） | `apps/mail/bundle/tools.json`、已接纳的指令/技能、后台事件声明和 `glance` | 绑定账户的 `mail.accounts`、`mail.folders`、`mail.sync`、`mail.list`、`mail.peek`、`mail.notify`、`mail.publish_card`、`mail.skip_event` | 模型编写的 L0 卡片或通知卡片，并发出 OctoSense 通知 | -| 日历（`os.calendar`，仅桌面端） | `apps/calendar/bundle/tools.json`、manifest 中的 `agent` 块和 `glance` | `calendar.events`、`calendar.add_event`、`calendar.remove_event`（破坏性操作：进入审批路由）、`calendar.notify`、`calendar.agenda`（`calendar` 宿主服务） | 一张日程卡片或议程卡片，并发出一条通知 | -| Photos、Maps、YouTube；手机上的 Camera | 各自包内的 `tools.json`、`agent` 块与 `glance` | 各应用的 `.notify`（Shell `NoticeService`） | 通知卡片与通知 | +| Rinx（Matrix 聊天） | 原生，在 Shell 内 | octos 的文件、记忆和网页工具 | – | +| Terminal（桌面端） | 原生，独立进程 | `terminal.read_screen`、`terminal.read_scrollback` | `terminal.run`，受 Setup 开关控制，每条命令都要批准 | +| Calculator、Clock、Notes、Reminders、Weather | 原生，在 Shell 内 | 各自的只读工具 | 同样的只读工具 | +| 邮件 | 脚本应用 | 绑定当前登录账号的 `mail.*` 工具，包括 `mail.publish_card` | – | +| 日历（桌面端） | 脚本应用 | `calendar.events`、`calendar.add_event`、`calendar.remove_event`（先问用户）、`calendar.notify`、`calendar.agenda` | – | +| 新闻 | 脚本应用 | `news.list`、`news.read`、`news.notify` | – | +| 相册、地图、YouTube；手机上的相机 | 脚本应用 | `.notify` | – | -AI providers 没有应用 Agent。 +App Hub 和 AI providers 没有 Agent。 -脚本应用在以下情况下拥有 Agent：manifest 声明了 `octos.*` 名称或 `agent` 块（`"tools": ["ask_user_question"]` 列出它可以使用的内核工具），或者应用包附带 `tools.json`（每个工具名为 `.`，带 schema、`risk`、`confirm` 和 `shareable`）。Broker 用 `card.<应用 id>` 标识应用；内核在准备 peer 时返回 peer slug。用户只需在首次使用面板上允许一次（从 Shell 的 “Ask ” 面板、应用自己的 `octos` 调用，或系统 Agent 的 `agents.ask` 打开）。此后 Shell 在启动时就准备好这个 peer 并注册应用的工具，所以系统 Agent 的 `peer_list` 能看到它。 +### 系统 Agent 如何与应用 Agent 通信 -在 Unix 上，已获同意且有可用 workspace 的应用 Agent 还能用宿主只读工具 `files.list`、`files.read` 和 `files.search` 读取账号文件夹；这些工具不会暴露所有宿主服务数据库。手机默认构建 `toolbox-peers`，因此在手机上，manifest 申请了 `research` 或 `crawl` 的应用 Agent 还会得到系统工具箱的工具；目前还没有应用申请。 +系统 Agent 从不自己执行应用的工具，而是请应用的 Agent 去做： -Mail 已有可选的新邮件处理路径：登录并允许 Agent 后，系统 Agent 用 `agents.provision` 配置指令、技能文本和轮询。宿主同步 Inbox、持久保存新邮件事件，自动启动 Mail Agent；Agent 用账户绑定工具读取邮件，自行判断是否发布卡片。首次同步只建立基线，不处理历史邮件。已接纳的 `AGENT.md` 和技能文本随每轮提供，这不是内核原生技能安装。其他应用触发器和 cron 仍未实现。见[邮件事件导读](docs/mail-agent-events.zh-CN.md)。 +1. 系统 Agent 调用 `peer_send_input`，用普通的话写下请求。 +2. octos 不自己运行这一轮，而是把它作为 `peer/input` 事件交给 Shell。 +3. Shell 在应用 Agent 的会话上启动这一轮，带着应用的工具、记忆和审批规则。如果用户没有允许这个 Agent，或者账号已退出登录，Shell 会拒绝这次输入。 +4. 结果写到 peer 共享的**黑板**上，系统 Agent 用 `peer_gather` 读取。 -**从系统 Agent 到 glance 屏幕上的一张卡片**： +下图中，用户请系统 Agent 在 glance 屏幕上提醒自己，邮件的 Agent 随后发出一张卡片： ![From the system agent to a card on the glance screen](docs/images/agents-card-flow.png) @@ -140,20 +156,13 @@ sequenceDiagram -以下通知/日历工具由模型提供文字、宿主填充固定模板。Mail 另有 `mail.publish_card`，接收由模型编写并经宿主校验的 L0 源码： - -- **通知卡片。** 邮件、新闻、照片及其他 `.notify` 工具使用 Shell 的 [`notice.card`](crates/shell/resources/glance/notice.card)。Shell 提供应用图标和名称；调用提供标题和正文。重复使用同一个 `card_id` 会替换该应用之前的通知。 -- **日历卡片。** `calendar.notify` 和 `calendar.agenda` 使用日历自己的[日程与议程模板](apps/calendar/host-service/resources)。 - -邮件和新闻的宿主服务把 `notify` 转交给 Shell。没有独立宿主服务的应用则由 Shell 的 [`glance_notice.rs`](crates/shell/src/glance_notice.rs) 直接处理。无论哪种方式，Shell 都以应用的身份发布，并要求应用获准使用 `glance`（`glance::publish_for`）。 - -点击这些卡片会打开应用。要就邮件通知与邮件的 Agent 对话，请打开 “Ask Mail”（[见下文](#直接与应用的-agent-对话)）。独立的[卡内对话](#卡内对话)功能及其当前可用情况见后文。 +`.notify` 工具填充固定的卡片模板（[`notice.card`](crates/shell/resources/glance/notice.card)，或日历的[日程与议程卡片](apps/calendar/host-service/resources)），模型只负责文字。邮件另有 `mail.publish_card`，它接收模型编写的卡片，检查后再发布。无论哪种方式，Shell 都以应用的身份发布，并要求应用拥有 `glance` 权限。 -`mail.notify` 和 `calendar.add_event` 是 `act` 工具，通常不需要逐次调用的审批面板。破坏性和对外调用进入审批流程；Shell 的审批路由决定哪些请求需要用户处理（[见下文](#一次带审批的工具调用)）。 +邮件的 Agent 也可以自己启动。用户登录、允许邮件的 Agent，并请系统 Agent 打开新邮件处理（`agents.provision`）之后，宿主会在后台同步收件箱，每来一封新邮件就启动一次 Agent。Agent 用绑定账号的工具读取邮件，自己判断要不要发卡片。其他应用还没有事件；详见[邮件事件导读](docs/mail-agent-events.zh-CN.md)。 ### 一个应用 Agent，两条通道 -一个应用 Agent 就是每个（应用，账号）一个由宿主拥有的 octos **peer**，归系统 Agent 所有，有自己的工作区、记忆命名空间、模型和工具列表。系统 Agent 和用户各自在自己的通道里与它对话： +系统 Agent 和用户与同一个应用 Agent 对话，但各走各的通道： ![One app agent, two lanes](docs/images/agents-two-lanes.png) @@ -183,29 +192,72 @@ flowchart TB -- **系统 Agent 的通道**是 peer 自己的会话 `…#peer-`。系统 Agent 发送 `peer_send_input`；octos 把它作为 `peer/input` 交给 Shell 的宿主连接，由 Shell 自己启动这一轮，所以这一轮带着应用的工具、记忆和审批运行（对已退出登录的账号或用户未允许的应用，Shell 以 `peer/input/reject` 拒绝）。peer 的结果写到 peer 黑板上，由系统 Agent 读取。 -- **用户的通道**是一个请求上下文 `…#peerctx-.`，以 `share_history` 打开，打开方可以是 Shell 的 “Ask ” 面板（`agents::conversation`，客户端实例 `shell-ask`）、卡片的卡内对话，或应用自己的界面（原生模块的 `open_conversation`、脚本应用的 `octos.session.open`、进程应用的 peer link），每个句柄一个新的上下文（[octos#2636](https://github.com/octos-org/octos/pull/2636)，UPCR-2026-034）。两条通道并行运行，每个会话同一时间只有一轮：用户的消息不必等系统 Agent 的回合。每一轮都会以只读块的形式看到另一条通道的最近消息，这个块不会写入自己的对话记录；每一轮都标明说话者（`[from the person: ]`、`[from the system agent]`）。应用跟随两条通道，每个事件带有 `lane` 和说话者；`octos.session.history` 按时间合并两份对话记录。用户的回合也会在黑板上留下结果（`origin: person`），系统 Agent 用 `peer_gather` 就能看到。 -- *2026-09-29 之前两者在 peer 会话上的同一个共享对话中说话（[#166](https://github.com/OctoSense-org/OctoSense/pull/166)，octos#2626）：每个 peer 一个队列，一次一轮。* -- **Rinx 小程序**保留各自的请求上下文（`open_context`），各有自己的对话记录和文件夹，不与任一通道共享。 +- **系统 Agent 的通道**是 peer 自己的会话 `…#peer-`。 +- **用户的通道**是一个以 `share_history` 打开的请求上下文 `…#peerctx-.`。用户的回合在这里运行，应用自己发起的回合也在这里运行。 + +两条通道并行运行，用户永远不必排在系统 Agent 的任务后面。每一轮都会以只读上下文的形式看到另一条通道最近的消息，每条消息都标明说话者（`[from the person: Mail]`、`[from the system agent]`）。用户的回合也会把结果留在黑板上，系统 Agent 由此知道用户做了什么。Rinx 小程序有各自私有的上下文（`open_context`），不与任一通道共享。 ### 直接与应用的 Agent 对话 -用户不只能和系统 Agent 对话：任何应用自己的 Agent，用户都可以直接与它对话。用户发起的每一轮都是该应用 peer 上用户通道里的一轮，与系统 Agent 的通道并列。它和系统 Agent 发起的一轮一样，带着应用的工具、记忆和审批运行。 +用户可以直接与任何应用的 Agent 对话。这些回合在用户的通道里运行，和系统 Agent 的回合一样，带着应用的工具、记忆和审批。 + +| 入口 | 怎么用 | +| --- | --- | +| **“Ask <app>”** 面板 | Shell 为每个有 Agent 的应用提供的面板，从顶栏的 “Ask <app>” 按钮或按 Shift+F8 打开。桌面端上它出现在系统对话旁边。手机上它全屏显示，但还没有可以打开它的触控入口。 | +| **卡内对话** | 在声明了对话的 glance 卡片里输入（[见下文](#卡内对话)）。 | +| **应用自己的界面** | 应用可以自己打开用户的通道（[见下一节](#应用如何使用自己的-agent)）。Rinx 绘制自己的助手界面；其他应用通过 “Ask <app>” 面板进入。 | + +面板会先征得同意，然后显示两条通道，每条消息都标明说话者。面板上的“停止”只结束用户自己的回合。面板的其他行为见 [docs/architecture.zh-CN.md 第 2 节](docs/architecture.zh-CN.md#2-agent)。 + +### 应用如何使用自己的 Agent + +应用只能经过 Shell 访问自己的 Agent，从不直接使用内核协议；Shell 会在每次调用上标注应用的身份。 -| 入口 | 位置 | 如何打开 | +| 应用类型 | 接口 | 谁在用 | | --- | --- | --- | -| **“Ask ”**（`crates/shell/src/app_chat/`） | Shell 为每个拥有 Agent 的应用提供的面板，不论应用自己是否绘制对话界面：它就是系统对话的窗格，以应用对话的形式绘制（`app_panel: true`）。在桌面端，它位于系统对话的右侧，两条通道并排显示。 | 顶栏的 “Ask ” 按钮（当前聚焦窗口的应用拥有 Agent 时显示）、Shift+F8，或菜单项 “Ask this app's agent”。如果聚焦的应用没有 Agent，Shell 会提示 “No app agent here”。在手机上，这个窗格绘制为全屏面板，但 `main` 上还没有可以打开它的触控入口。 | -| **卡片的卡内对话**（`sys.chat`，`crates/shell/src/glance_chat.rs`） | 声明了对话的 glance 卡片 | 用户在卡片中输入，由发布卡片的应用自己的 Agent 回答，回复标为 AI 撰写。当前可用情况和演示见[卡内对话](#卡内对话)。 | -| **应用自己的界面** | 原生模块的 `open_conversation`、脚本应用的 `octos.session.open`、进程应用的 peer link | 在应用内。Rinx 绘制自己的助手界面；系统应用都不绘制对话界面，目前通过 “Ask ” 面板与它们的 Agent 对话。 | +| 脚本应用 | `host.request("octos.session.open" / "octos.session.history" / "octos.turn.start" / "octos.turn.interrupt")`，限于 manifest 声明的名称 | 商店应用。系统应用都没有声明：它们的 Agent 由 Shell 驱动。 | +| 原生应用（在 Shell 内或作为进程） | Makepad 的 `OctosPeer` 客户端，经由 peer link：先打开链接，再用 `serve_tools` 应答 Agent 对应用自身工具的调用。同一份代码在两种托管方式下都能用。 | Calculator、Clock、Notes、Reminders、Weather、Terminal | +| 使用注入服务的原生应用 | `OctosAppService`：`open_conversation` 打开用户的通道，`open_context` 打开私有上下文 | Rinx | -“Ask ” 面板的行为： +对脚本应用来说，最小可用的接入只需要在 manifest 中申请两个能力： + +```json +"capabilities": ["octos.session.open", "octos.turn.start"] +``` + +再写几行 Splash，打开用户的通道并发送一轮： + +```splash +fn ask(){ + ui.answer.set_text("Waiting for the assistant…") + host.request("octos.session.open", {}, fn(s){ + if !s.is_ok { + ui.answer.set_text("Assistant unavailable: " + s.error) + return + } + host.request("octos.turn.start", {text: ui.prompt.text()}, fn(r){ + if r.is_ok { ui.answer.set_text(r.data.text) } + else { ui.answer.set_text("Assistant unavailable: " + r.error) } + }) + }) +} +``` -- **先征得同意。** 用户还没有决定的 Agent 会先弹出首次使用面板，面板等待用户（“'s assistant is not allowed yet: allow it on the sheet.”）。已关闭的 Agent 会直接说明：“'s assistant is off. Turn it on in Setup › Assistant › Approvals.” -- **两条通道，标明说话者。** 面板跟随两条通道，并加载合并后的历史；每一行都显示说话者（用户、系统 Agent、应用的 Agent）。 -- **发送**会发起用户的一轮（`TurnTrigger::Person`）。它从不等待系统 Agent 的回合：只有系统 Agent 的通道在运行时，“发送”仍然可用。如果应用的 Agent 有一个提问正在等待，输入的文字就作为对它的回答。 -- **停止**（用户自己的回合运行时取代“发送”的位置）只停止这一轮（“Stopped.”，或 “Nothing of yours was running.”）。正在运行的系统 Agent 回合有自己的一行，带 **“Stop the system agent's task”**，只停止那条通道。Shell 的审批和提问面板上的 **“Stop 's agent”** 按钮会停止两条通道（`approvals::stop_agent`）：设备归用户所有。 -- **提问**：用户或应用发起的回合中的提问在面板中显示和回答；系统 Agent 的提问出现在系统对话（F8）中。审批与其他地方一样，都是 Shell 的面板。 -- **关闭**只是隐藏面板。它的上下文和跟随者保持打开，所以再次打开时仍能看到用户自己的那些行。面板为另一个应用打开、Agent 被关闭，或应用的 peer 消失（对邮件来说，就是账号退出登录）时，这个上下文才会关闭。 +在托管了内核的 Shell 中，第一次调用会请用户允许这个应用的 Agent。请把“不可用”当作正常状态处理：设备可能没有内核（iOS）或没有配置提供方，用户也可能拒绝了。这个示例和完整接口见 Design Flow 的 [AI-SERVICES 指南](https://github.com/OctoSense-org/OctoScript-App-Design-Flow/blob/main/docs/AI-SERVICES.zh-CN.md#最小调用示例与不可用状态)。 + +### 应用要给 Agent 提供什么 + +Agent 能做什么，取决于应用交给它什么。脚本应用把这些都声明在应用包里；原生应用则声明在 `native-apps.json` 的条目里。 + +- **声明。** manifest 的 `agent` 块列出 Agent 可用的内核工具（系统应用只申请了 `ask_user_question`）、需要的模型能力（`tool_calling`），以及可选的、写有指令的 `AGENT.md`。原生应用的条目还会说明它自己的 Agent 可以调用它的哪些工具（`own_tools`），系统 Agent 又可以调用哪些（`system_tools`）。 +- **工具。** `tools.json` 描述每个工具（命名为 `.`）：输入 schema、`risk`（`read`、`act` 或 `destructive`）、由谁确认（`confirm: host` 用 Shell 面板，`app` 用应用自己的面板），以及其他应用的 Agent 能否使用（`shareable`）。 +- **执行工具的地方。** 声明了的工具还需要执行者：应用的宿主服务（邮件、日历、新闻）、Shell 的通知服务（其他系统应用的 `.notify`），或原生应用已打开的窗口。商店应用没有宿主服务，而标为 `implemented_by: "app"` 的工具在 Card runner 中还没有执行器。所以目前商店应用的 Agent 能对话、能提问、能读取自己的文件夹，但还不能通过自己的工具做事。 +- **数据。** 默认情况下，Agent 在它所属账号的文件夹 `apps//accounts//` 中工作（不区分账号的应用只有一个 `device` 文件夹），并用宿主的只读工具 `files.list`、`files.read` 和 `files.search`（Unix 上）读取它。应用也可以声明 `storage.agent_workspace: "none"`；Calculator、Clock、Notes、Reminders、Weather 和 Terminal 就是这样，它们的 Agent 只能看到自己的工具返回的内容。任何 Agent 都看不到别的账号的文件夹。 +- **记忆。** 每个 Agent 有自己的记忆命名空间 `app//acct-`，随账号一起清除。 +- **联系用户的方式。** 有了 `glance` 权限，它的工具就能发布卡片。 +- **事件**（目前只有邮件）。一个 `triggers.events` 条目、一项技能和 `AGENT.md`，让邮件的 Agent 不等人开口就能处理新邮件；在这些回合里，Shell 会附上已接纳的指令和技能文本。 + +新增一个工具的步骤（manifest、`tools.json`、授权、处理代码、审批路径）见 [AGENTS.md（英文）](AGENTS.md#architecture-documentation-and-code-walkthroughs)，设计见 [ADR 0002（英文）](docs/adr/0002-event-driven-app-agents.md)。 ### 一次带审批的工具调用 @@ -246,62 +298,51 @@ sequenceDiagram -- **工具调用**：octos 把 `peer/tool/call` 发给 Shell 的中转（`crates/shell/src/host_tools/`）。中转按（拥有工具的应用，工具）和调用方检查授权，按工具的 schema 检查参数，检查调用方的预算，再把调用路由到拥有工具的应用的执行器：进程内模块的执行器、脚本应用的宿主服务、进程应用的 peer link，或 AI bus 上 Terminal 的 `run`。由宿主确认的工具（`confirm: host`，如 Terminal 的 `run`），若应用同时在 AI bus 上提供它，就不经过应用的 peer link，由宿主的审批面板决定。应用自己的 Agent 只能调用其 `native-apps.json` 条目允许的工具（`agent.own_tools`；条目未收窄时为它的全部工具）：Terminal 自己的 Agent 只读取屏幕和回滚内容，输入命令仍是系统 Agent 的 `terminal.run`，受 Setup 中的开关控制。 -- **审批**交给 `crates/shell/src/approvals/`：外部客户端保留自己的提示；开发者模式批准覆盖应用的调用；`confirm: app` 使用所属应用注册的确认面板；必须现场决定的请求跳过规则；随后常设规则可决定符合条件的调用，其余由 Shell 面板询问用户。系统 Agent 无法批准。[导读](docs/architecture-walkthrough.zh-CN.md#审批顺序)列出完整顺序、时限与审计行为。 -- **时限与停止**（[#167](https://github.com/OctoSense-org/OctoSense/pull/167)）：Shell 为应用 peer 持有的审批或提问在 10 分钟后过期（`OCTOSENSE_PROMPT_DEADLINE_SECS`）：审批路由拒绝它，提问被婉拒，两者都保持显示为 "Expired: no answer in 10 min"。如果 30 秒后这一轮仍在运行，代理会中断它，好让下一轮开始。面板上的 “Stop 's agent” 会结束两条通道上正在运行的回合，包括用户的和系统 Agent 的；“Ask ” 面板的“停止”只结束用户自己的回合（[见上文](#直接与应用的-agent-对话)）。 -- **外部客户端的提示**留在客户端：Shell 不回答、也不让 Talk to Octos 客户端各轮的审批过期（octos#2624）。 +- **中转**（[`crates/shell/src/host_tools/`](crates/shell/src/host_tools/)）接收每一个 `peer/tool/call`。它先检查这个调用方能否使用这个工具，按工具的 schema 校验参数，并从调用方的预算中扣除（默认每轮 32 次、每天 1000 次工具调用）；之后才在拥有该工具的应用中运行它：原生应用已打开的窗口、脚本应用的宿主服务，或经 peer link 交给进程应用。结果同样要按 schema 检查。 +- **审批路由**（[`crates/shell/src/approvals/`](crates/shell/src/approvals/)）按固定顺序做决定。开发者模式只能由用户打开，它会批准所覆盖应用的一切调用。`confirm: app` 的工具交给应用自己的面板。必须每次都问的调用（例如 Terminal 的命令）直接弹出面板。其余调用可以由用户的常设规则决定，否则由 Shell 面板展示确切参数，请用户确认。每个决定都会写入审计日志。完整顺序见[导读](docs/architecture-walkthrough.zh-CN.md#审批顺序)。 +- **时限。** 10 分钟内无人回答的审批或提问会被拒绝，绝不会被批准。如果 30 秒后这一轮仍在运行，Shell 会中断它，好让下一轮开始。 ### 卡片与提问 -#### 发布与打开卡片 - -拥有 `glance` 权限的应用通过 `glance.publish` 以自己的身份发布卡片，也可以使用 `glance.withdraw` 和 `glance.list`。Shell 从调用方取得发布者，从不读取参数中的发布者。卡片可以是由 `data` 填充的 L0 `source`（只用于呈现，由 Octoscript 的 L0 检查器检查），也可以是 Splash `script`。 - -每个应用每分钟最多发布 6 次、保留 4 张卡片。Shell 总共保留 32 张，按优先级、再按时间排序：手机的 glance 页面显示前 6 张，桌面的面板列出全部卡片。详见 [`glance.rs`](crates/shell/src/glance.rs)。 - -| 界面 | 行为 | -| --- | --- | -| 桌面面板 | 新卡片会打开 glance 面板，除非已有卡片窗口打开。顶栏铃铛或 F9 也可以打开面板。点击卡片上自身控件以外的地方，会在卡片窗口中打开它。鼠标悬停的卡片会显示打开和移除操作，刚到的卡片旁会有几秒钟的强调色标记。放不下的那张卡片会在列表末尾露出一部分。卡片在面板里有高度上限，更高的卡片可以用滚轮在原处滚动；带卡内对话的卡片停在最新的消息处，输入框和最近一轮对话始终可见。用 F9（或在面板里点击）打开时，面板接管键盘：方向键在卡片之间移动焦点环，Return 打开卡片，Delete 移除卡片，Esc 关闭面板。 | -| 桌面通知 | 以 `notify` 发布的卡片还会弹出 toast，显示应用的图标和名称、卡片标题及其 `summary`（没有时用卡片自带的摘要）。点击 toast 会在独立窗口中打开卡片。同时最多显示三条 toast，其余的由下方的“+N more”标签展开。面板打开时，toast 叠放在面板左侧。 | -| 桌面关闭操作 | 鼠标悬停的卡片会显示移除按钮：`glance::dismiss` 会移除卡片，效果如同应用撤回了它。“Clear all”会移除所有卡片。移除的卡片可以在报告这次移除的 toast 上撤销（Undo），面板接管键盘时也可以按 ⌘Z 撤销。面板本身另有关闭按钮。 | -| 手机 | `notify` 在通知栏发出通知，点击后打开 glance 页面。 | - -未声明主题的卡片使用 Shell 的浅色或深色配色。toast 和面板会滑入；设置 `OCTOSENSE_REDUCE_MOTION=1` 则保持静止。桌面界面实现在 [`glance_panel.rs`](crates/shell/src/glance_panel.rs)、[`glance_sheet.rs`](crates/shell/src/glance_sheet.rs) 和 [`notifications.rs`](crates/shell/src/shell/notifications.rs) 中（`keep_clear_of`，[#273](https://github.com/OctoSense-org/OctoSense/pull/273)）。 - -#### 交互式卡片 - -卡片在应用自己的策略下运行，与应用界面在 Card runner 中一样。用户在卡片上的操作经过应用的能力闸门和宿主服务。这属于应用操作，因此不需要额外进行 Agent 工具调用的 Shell 审批（[#153](https://github.com/OctoSense-org/OctoSense/pull/153)）。 +拥有 `glance` 权限的应用以自己的身份发布卡片（`glance.publish`、`glance.withdraw`、`glance.list`）；Shell 从调用方取得发布者，从不读取参数中的发布者。卡片在应用自己的权限下运行，所以在卡片上按下按钮是应用自己的操作，不是 Agent 的工具调用。卡片显示在桌面端的[一览面板](desktop/README.zh-CN.md#一览面板)中，桌面端 README 对它有详细说明。 #### 卡内对话 -**当前可用情况：**`main` 上应用 Agent 发布的卡片都没有声明对话。唯一带对话的内置卡片是 [`mail-request.card`](crates/shell/resources/glance/mail-request.card)，通过 `OCTOSENSE_GLANCE_DEMO=mail` 启用，使用固定的演示回复。 +卡片可以声明 `sys.chat(app, thread, fields)`。用户在卡片里输入，只有发布卡片的应用自己的 Agent 回答，回答在用户的通道里进行。对话记录归 Shell 所有，保存在应用的账号文件夹中；只有用户亲手输入的内容才记为用户的话。模型写的文字会标为 AI 撰写，且从不作为操作执行。目前还没有 Agent 发布的卡片声明对话；演示卡片 [`mail-request.card`](crates/shell/resources/glance/mail-request.card) 用固定的回复展示了这一功能。详见 [`crates/l0-chat`](crates/l0-chat/README.md)。 -L0 卡片可以声明 `sys.chat(app, thread, fields)` 并绘制 `ChatEntry` 行。它也可以显示模型撰写的文字（`class: model-copy`），这些文字会标为 AI 撰写，且从不作为操作执行（[#263](https://github.com/OctoSense-org/OctoSense/pull/263)）。 +#### 提问 -对话记录归宿主所有。只有发布卡片的应用自己的 Agent 可以回答，回答在用户通道中进行；只有用户亲手输入的内容才会记为用户的话。线程保存在 `apps//accounts//chat/.json`。详见 [`crates/l0-chat`](crates/l0-chat/README.md) 和 [`glance_chat.rs`](crates/shell/src/glance_chat.rs)。 +Agent 的 `ask_user_question` 出现在这一轮的来处：用户和应用自己发起的回合在应用的对话中提问，系统 Agent 发起的回合在系统对话中提问。只有用户能回答，而且只能在 Shell 界面上回答。 -#### 提问 +## 为什么它省内存、省算力 + +一部手机要同时运行 Shell、内核、最多十来个应用 Agent 以及它们的应用。下面这些设计让这件事并不昂贵： -提问（`ask_user_question`）按这一轮的触发方路由。来自用户通道或应用的回合在应用对话中提问；来自系统 Agent 通道的回合在系统对话中提问。只有用户能回答，而且只能在 Shell 界面上回答。 +- **一个内核，Agent 只是会话。** 所有 Agent 都在同一个 octos 内核里。在内核中，一个 Agent 就是一份存储的状态（对话记录、记忆和黑板条目），加上运行回合时的 Tokio 任务。没有哪个 Agent 独占一个进程。 +- **到内核只有一条连接。** 系统对话和每个应用 Agent 的代理共用一条通往内核的流。一个小小的路由器（[`crates/kernel/src/router.rs`](crates/kernel/src/router.rs)）为每个请求分配唯一的 id，并把每条通知只发给关心该会话的使用方。 +- **线程按服务分配，而不是按回合。** 回合是 Tokio 任务。在 Shell 中，内核服务运行在一个有两个工作线程的 Tokio 运行时上，每个活跃应用 Agent 的代理有自己的单线程运行时（所以一个慢吞吞的 Agent 拖不住别的 Agent，也拖不住界面），系统对话则用一个线程，只在有变化时才通过 Makepad 的 `SignalToUI` 唤醒界面。octos 子进程（桌面端和 Android）使用 Tokio 的默认运行时，每个 CPU 核心一个工作线程。 +- **按需启动，空闲即停。** 内核在第一个连接到来时启动：如果用户已经允许过某个脚本应用的 Agent，就在 Shell 启动时；否则在某个 Agent 或系统对话第一次需要它时。系统对话只在面板打开或有回合运行时才保持连接。Talk to Octos 关闭时，最后一个连接断开，内核就停止。没有获准 Agent 的应用既没有代理，也没有 peer。 +- **应用共享 Shell 进程。** 脚本应用是同一个 Card runner 中的隔离环境，原生应用是模块，所以大多数应用不单独占用进程。脚本出错只影响它自己的隔离环境，原生模块的 panic 在模块边界被捕获。只有需要系统沙箱的应用才作为独立进程运行。 +- **进程应用的画面零拷贝。** Terminal 的画面以共享 GPU 表面的形式到达 Shell：macOS 上是 IOSurface，Windows 上是 D3D11 共享句柄，Linux 配 Vulkan 时是 DMA_BUF。在只能靠 CPU 拷贝的环境（Linux 配 OpenGL）中，所有应用都改在 Shell 内运行。 ## 目录结构 | 路径 | 内容 | | --- | --- | | [`desktop/`](desktop/README.zh-CN.md) | 桌面端打包，package `octosense`：入口（只有 `src/main.rs`）、应用目录（`config/apps.json`）、从上游 Makepad 同步窗口管理器（`upstream/`、`scripts/upstream.py`），以及桌面端的系统应用选择。 | -| [`phone/`](phone/README.zh-CN.md) | Home 应用，package `octosense-home`（APK id `dev.makepad.octosense`）：包装 Shell 的入口（`src/main.rs`）、内置设置应用（`src/settings_*.rs`、`src/android_settings.rs`、`resources/settings/`）、Android、OpenHarmony 和 iOS 打包、系统桥的手机端（`android/`），以及手机端的系统应用选择。 | -| [`rom/`](rom/README.zh-CN.md) | 仅 OnePlus 6 ROM 镜像：`vendor/`（产品定义、特权权限、overlay、设置后端、特权 agent）、`patches/`、镜像/刷机/OTA 脚本、Home APK 构建脚本、`web-installer/`、产品测试。 | -| `crates/shell/` | 唯一的一份 Shell，package `octosense-shell`，两个包都链接它：窗口管理器（desk、样式、平铺、场景）、托管（子进程、进程内模块、App Hub、AI 面板）、手机层（主屏页面、下拉面板、手势、Android 启动器桥），以及主题、壁纸和图标（`resources/`）。 | -| [`crates/ai-host/`](crates/ai-host/README.md) | Shell 的 AI 服务，统一入口，package `octosense-ai-host`：octos 内核服务、带平台二维码导入的 `llm` 宿主服务、`model` 服务（`model.complete`）、为每个脚本应用提供 Agent（`card.<应用 id>`）的 `octos` 宿主服务，以及原生应用访问助手的通道。 | -| [`crates/kernel/`](crates/kernel/README.md) | octos 内核服务，package `octosense-kernel`：把 [octos](https://github.com/octos-org/octos) Agent 内核作为 Shell 服务，每个进程一个，由 AI 服务商配置，供所有使用方共享；以及系统 Agent 的精确工具列表。 | -| [`crates/app-peers/`](crates/app-peers/README.md) | 应用 Agent 的代理，package `octosense-app-peers`：每个（应用，账号）一个由宿主拥有的 octos peer，它的两条通道、工具、`peer/input`、时限和清除（[Rinx ADR 0007（英文）](https://github.com/hagency-org/Rinx/blob/main/docs/adr/0007-host-owned-octos-app-peers.md)）。 | -| [`crates/l0-chat/`](crates/l0-chat/README.md) | L0 卡片卡内对话（`sys.chat`）的宿主一侧，package `octosense-l0-chat`，由 Shell 的 glance 卡片和 AppCard 共用。 | -| [`crates/toolbox/`](crates/toolbox/README.md) | 系统工具箱，package `octosense-toolbox`：工作流模板、运行器、分支（fork）与评估，以及 `mod.research`；在 `toolbox-peers` 特性下作为宿主工具提供给应用 Agent。 | -| [`apps/`](apps/README.zh-CN.md) | 系统应用（新闻、相册、地图、相机、邮件、日历、AI 服务商、YouTube），均为受隔离约束的脚本应用；它们的宿主服务（`mail`、`calendar`、`news`、`llm`）；`apps/reference`；以及需显式启用的 AppCard 助手（`apps/appcard`）。 | -| `tools/` | `setup.py`（锁定版本的框架源码）、经审查的 Makepad 运行时补丁（`runtime-patches/`）、`kernel-artifact.py`（Android APK 以 `liboctos.so` 形式打包的 octos 内核）、`check-shell-graph.sh`（每个 Shell 构建都要通过的依赖图检查）。 | -| [`docs/adr/`](docs/adr/README.zh-CN.md) | 架构决策记录：本仓库的决策，以及作为历史保留的 Home 决策 0001–0006。 | -| `Cargo.toml`、`Cargo.lock` | 一个工作区。所有外部依赖都只在 `[workspace.dependencies]` 中锁定一次。 | -| `native-runtime.lock.json`、`runtime-patches.lock.json` | OctoScript-Makepad 发行版（并通过它确定 Makepad 和 OctoScript），以及 Makepad 之上经审查的补丁。 | +| [`phone/`](phone/README.zh-CN.md) | Home 应用，package `octosense-home`（APK id `dev.makepad.octosense`）：入口、内置设置应用、Android、OpenHarmony 和 iOS 打包、系统桥的手机端（`android/`），以及手机端的系统应用选择。 | +| [`rom/`](rom/README.zh-CN.md) | 仅 OnePlus 6 ROM 镜像：`vendor/`、`patches/`、镜像/刷机/OTA 脚本、Home APK 构建脚本、`web-installer/`、产品测试。 | +| `crates/shell/` | Shell，package `octosense-shell`：窗口管理器、应用托管、系统对话和 “Ask <app>” 面板、宿主工具中转和审批路由、手机层，以及主题、壁纸和图标。 | +| [`crates/ai-host/`](crates/ai-host/README.md) | Shell 的 AI 服务，统一入口，package `octosense-ai-host`：内核服务、`llm` 与 `model` 服务，以及为每个脚本应用提供 Agent（`card.<应用 id>`）的 `octos` 宿主服务。 | +| [`crates/kernel/`](crates/kernel/README.zh-CN.md) | 作为 Shell 服务的 octos 内核，package `octosense-kernel`：每个进程一个，由 AI providers 配置，供所有使用方共享；以及系统 Agent 的精确工具列表。 | +| [`crates/app-peers/`](crates/app-peers/README.md) | 应用 Agent 的代理，package `octosense-app-peers`：每个（应用，账号）一个 peer，它的两条通道、工具、`peer/input`、时限和清除。 | +| [`crates/l0-chat/`](crates/l0-chat/README.md) | L0 卡片卡内对话（`sys.chat`）的宿主一侧，package `octosense-l0-chat`。 | +| [`crates/toolbox/`](crates/toolbox/README.md) | 系统工具箱，package `octosense-toolbox`：工作流模板和 `mod.research`，在 `toolbox-peers` 特性下提供给应用 Agent。 | +| [`apps/`](apps/README.zh-CN.md) | 系统应用（新闻、相册、地图、相机、邮件、日历、AI providers、YouTube），均为脚本应用；它们的宿主服务（`mail`、`calendar`、`news`、`llm`）；`apps/reference`；以及需显式启用的 AppCard 助手。 | +| [`native-apps.json`](native-apps.json) | 所有原生应用：代码来自哪里、在各平台上如何托管，以及它的 Agent。`tools/native_apps.py` 据此生成 Shell 的代码和 Cargo 条目。 | +| `tools/` | `setup.py`（锁定版本的框架源码）、经审查的 Makepad 运行时补丁（`runtime-patches/`）、`kernel-artifact.py`（构建 octos 内核）、`check-shell-graph.sh`（依赖图检查）。 | +| [`docs/`](docs/architecture.zh-CN.md) | 架构说明、代码导读和[架构决策记录](docs/adr/README.zh-CN.md)。 | +| `Cargo.toml`、`Cargo.lock`、`native-runtime.lock.json`、`runtime-patches.lock.json` | 一个工作区。所有外部依赖都只锁定一次；运行时锁文件指定 OctoScript-Makepad 发行版和 Makepad 之上经审查的补丁。 | Shell 只有一份，位于 `crates/shell`（[ADR 0001（英文）](docs/adr/0001-one-octosense-repository.md)）：桌面端与手机端以目标平台和 feature 区分，而不是各持一份源码副本。若某个 Shell 源文件同时出现在两个 crate 中，CI 会失败。 @@ -311,22 +352,14 @@ Shell 只有一份，位于 `crates/shell`（[ADR 0001（英文）](docs/adr/000 | 仓库 | 作用 | | --- | --- | -| [makepad（OctoSense fork）](https://github.com/OctoSense-org/makepad) | UI 框架和 `cargo-makepad` 打包工具。检出到 `.sources/makepad`，并应用经审查的运行时补丁。 | +| [makepad（OctoSense fork）](https://github.com/OctoSense-org/makepad) | UI 框架、`cargo-makepad` 打包工具，以及原生应用 Calculator、Clock、Notes、Reminders、Weather、Terminal、Sheets 和 Task。检出到 `.sources/makepad`，并应用经审查的运行时补丁。 | | [OctoScript-Makepad](https://github.com/OctoSense-org/OctoScript-Makepad)、[OctoScript](https://github.com/OctoSense-org/OctoScript) | 指定 Makepad 和 OctoScript 版本的运行时发行版（`native-runtime.lock.json`）。 | -| [OctoSense-App-Hub](https://github.com/OctoSense-org/OctoSense-App-Hub) | 签名目录、商店，以及隔离运行每个应用的 Card runner（`octosense-app-hub-app`）。 | -| [octos](https://github.com/octos-org/octos) | Agent 内核。在 Android 上 APK 以 `liboctos.so` 形式内置它；在桌面上内核服务运行 Shell 旁随附的 `octos-kernel`，并核对其版本与此处固定的一致（由 `tools/kernel-artifact.py --host --stage` 构建）；`OCTOS_APP_CORE_BIN` 可覆盖它。 | +| [OctoSense-App-Hub](https://github.com/OctoSense-org/OctoSense-App-Hub) | 签名目录、商店，以及隔离运行每个脚本应用的 Card runner。 | +| [octos](https://github.com/octos-org/octos) | Agent 内核。Android 上 APK 以 `liboctos.so` 形式内置它；桌面端运行 Shell 旁随附的 `octos-kernel`，并核对其版本与此处固定的一致（由 `tools/kernel-artifact.py --host --stage` 构建）。 | | [Rinx](https://github.com/hagency-org/Rinx) | Matrix 聊天与小程序，作为原生模块托管。 | 相关但不参与构建：[OctoScript-App-Design-Flow](https://github.com/OctoSense-org/OctoScript-App-Design-Flow)（如何构建和发布应用）、[OctoScript-Android](https://github.com/OctoSense-org/OctoScript-Android) 和 [OctoScript-OH](https://github.com/OctoSense-org/OctoScript-OH)（其他渲染后端）、[OctoSense 网站](https://github.com/OctoSense-org/octosense-org.github.io)。 -## AI 服务（octos） - -每个 Shell 运行一个 [octos](https://github.com/octos-org/octos) Agent 内核，首次使用时启动：Android 上是 APK 中的 `liboctos.so`，OpenHarmony 上在进程内运行，桌面端运行 Shell 旁随附的 `octos-kernel`（或 `OCTOS_APP_CORE_BIN` 指定的二进制），iOS 上没有。用户在系统应用 **AI providers** 中、在宿主面板上选择模型并输入密钥；密钥保存在平台的密钥存储中，永远不会到达应用。[`crates/ai-host`](crates/ai-host/README.md) 是两个 Shell 的统一入口，[`crates/app-peers`](crates/app-peers/README.md) 为每个获授权的原生应用分配自己的 octos peer（私有的上下文、工作区和记忆 `app//acct-`），归 Shell 的系统 Agent 所有。Peer 工具审批经 Shell 路由，由开发者模式、符合条件的常设规则，或用户在所属应用/Shell 确认面板上决定。系统 Agent 无法代答。 - -目前可用的：原生模块（Rinx）使用自己的 peer；AppCard（需主动开启）直接使用内核。隔离运行的脚本应用，无论系统应用还是商店应用，在托管了内核的 Shell 中通过 `octos` 宿主服务使用助手：每个应用有自己的、由宿主拥有的 peer（`card.<应用 id>`），它的工具审批和其他应用 Agent 一样交给 Shell 的审批面板（[#155](https://github.com/OctoSense-org/OctoSense/pull/155)）。`llm` 服务只为 `os.*` 应用管理提供方。应用自己的 Agent（`tools.json`、`AGENT.md`、skills、触发器、glance 卡片）见 [ADR 0002](docs/adr/0002-event-driven-app-agents.md)；自 [#160](https://github.com/OctoSense-org/OctoSense/pull/160) 起，应用 `tools.json` 中由宿主服务实现的工具已通过 Shell relay 提供给其 Agent。标为 `implemented_by: "app"` 的工具仍缺少 Card runner 执行器；声明不等于实现。 - -架构、信任模型、各类应用能用什么、规划及其状态，以及如何在本地运行和测试：[docs/ai-services.zh-CN.md](docs/ai-services.zh-CN.md)。它在整个系统中的位置：[docs/architecture.zh-CN.md](docs/architecture.zh-CN.md)。面向应用开发者：OctoScript-App-Design-Flow 的 [AI-SERVICES](https://github.com/OctoSense-org/OctoScript-App-Design-Flow/blob/main/docs/AI-SERVICES.zh-CN.md)。 - ## 环境准备 需要稳定版 Rust（`cargo` 位于 `~/.cargo/bin`）、Git、Python 3.9+（`desktop/scripts/upstream.py` 需要 3.11），macOS 上还需要 Xcode Command Line Tools。Makepad 和 OctoScript 解析到 `.sources/`（已被 git 忽略）中的检出，由环境准备脚本按锁定版本准备好： @@ -340,13 +373,15 @@ python3 tools/setup.py --check --cargo # verify: one Makepad, App Hub, octos an 锁文件变化后，`--update` 会把没有本地修改的检出移到新版本；`--cache DIR` 从已有克隆（`DIR/makepad`、`DIR/octoscript`、`DIR/octoscript-makepad`）借用 Git 对象。`.sources/` 中的本地修改会被保留。 -**本机已有这些仓库的克隆？** 每个仓库在本机只保留一个克隆，`.sources/` 中的每一项都作为它的 `git worktree`，这样每个仓库只有一个对象库，不会出现过时的副本。在 `~/.config/octosense/sources.json` 中一次性写明存放克隆的目录（其中为 `/makepad`、`/octoscript`、`/octoscript-makepad`）： +
本机已有这些仓库的克隆？把它们用作 hub。 + +每个仓库在本机只保留一个克隆，`.sources/` 中的每一项都作为它的 `git worktree`，这样每个仓库只有一个对象库，不会出现过时的副本。在 `~/.config/octosense/sources.json` 中一次性写明存放克隆的目录（其中为 `/makepad`、`/octoscript`、`/octoscript-makepad`）： ```json { "hub": "/path/to/clones" } ``` -也可以每次运行时用 `--hub DIR` 或 `OCTOSENSE_SOURCES_HUB=DIR` 指定；`OCTOSENSE_MAKEPAD_HUB=CLONE`（以及 `_OCTOSCRIPT_`、`_OCTOSCRIPT_MAKEPAD_`）或文件中的 `"repositories": {"makepad": "CLONE"}` 可单独指定某一个克隆。之后环境准备脚本会把锁定的版本 fetch 到该克隆，并运行 `git worktree add --detach .sources/ `，而不是重新克隆；`--update` 会移动这些 worktree。未配置时（CI、新机器）仍像以前一样克隆，`--no-hub` 可强制如此。`.sources/` 中已经是完整克隆的项只会被报告，不会被删除；其中没有本地工作时，`--convert` 会把它替换为 worktree。 +也可以每次运行时用 `--hub DIR` 或 `OCTOSENSE_SOURCES_HUB=DIR` 指定。之后环境准备脚本会把锁定的版本 fetch 到该克隆，并运行 `git worktree add --detach .sources/ `，而不是重新克隆；`--update` 会移动这些 worktree。未配置 hub 时（CI、新机器）仍像以前一样克隆，`--no-hub` 可强制如此。`.sources/` 中已经是完整克隆的项只会被报告，不会被删除；其中没有本地工作时，`--convert` 会把它替换为 worktree。 删除本仓库的某个检出之前，先移除它的 `.sources/` worktree，免得各克隆里留下失效的记录： @@ -355,7 +390,7 @@ python3 tools/setup.py --remove-worktrees # git worktree remove + prune in eac git worktree remove # if it is itself a worktree ``` -手动操作等价于 `git -C worktree remove --force .sources/`（经审查的 Makepad 补丁处于暂存状态，因此需要 `--force`；先检查 `git status`）以及 `git -C worktree prune`。 +
## 构建 @@ -367,7 +402,7 @@ cargo check --locked -p octosense --features mobile-apps cargo check --locked -p octosense -p octosense-appcard --features mobile-apps,app-appcard ``` -助手需要 Shell 旁的 octos 内核：`python3 tools/kernel-artifact.py --host --stage target/release` 构建固定版本并放到该处，每个 octos 固定版本做一次；桌面会拒绝版本不符的内核并说明原因（[构建与运行](desktop/README.zh-CN.md#构建与运行)）。没有内核时桌面在没有助手的情况下运行。 +助手需要 Shell 旁的 octos 内核：`python3 tools/kernel-artifact.py --host --stage target/release` 构建固定版本并放到该处，每个 octos 固定版本做一次。桌面端会拒绝版本不符的内核并说明原因（[构建与运行](desktop/README.zh-CN.md#构建与运行)）。没有内核时，桌面端在没有助手的情况下运行。 **手机端**（在 `phone/` 中运行，它会选择手机端的系统应用；详见 [phone/README.zh-CN.md](phone/README.zh-CN.md)）： @@ -380,29 +415,27 @@ python3 ../rom/scripts/build-home.py --help # the Home and B **ROM 镜像**（Linux 构建主机，外部 LineageOS 源码树；不在 CI 中）：[rom/README.zh-CN.md](rom/README.zh-CN.md)。 -托管应用和 UI 测试使用隐藏窗口和本地控制接口运行：`MAKEPAD_HIDE_WINDOWS=1 MAKEPAD_REMOTE=`（路由见 `/help`）。 - ## CI `.github/workflows/` 中的工作流按路径过滤，每次改动只运行其路径需要的任务： | 工作流 | 触发路径 | 检查内容 | | --- | --- | --- | -| `desktop.yml` | `desktop/`、`crates/`、`apps/`、工作区文件、`tools/` | 编译桌面端（默认、`mobile-apps`、`mobile-apps,app-appcard`），Shell 依赖图检查（`tools/check-shell-graph.sh`），每个 Shell 源文件只有一份，`tools/` 的测试 | -| `phone.yml` | `phone/`、`crates/`、`apps/`、工作区文件、`tools/` | 在 macOS 上编译 Home 及其内置模块，Shell 依赖图检查，并运行 Shell、Home、AI 服务、App Hub 准入和运行时策略的测试；耗时最长的任务 | -| `apps.yml` | `apps/`、`crates/`、工作区文件、`tools/setup.py` | 内核服务、app peers、AI 服务商配置、邮件与 `llm` 宿主服务、Shell 的 AI 服务（`crates/ai-host`）、AppCard | +| `desktop.yml` | `desktop/`、`crates/`、`apps/`、工作区文件、`tools/` | 编译桌面端（默认、`mobile-apps`、`mobile-apps,app-appcard`），Shell 依赖图检查，每个 Shell 源文件只有一份，`tools/` 的测试 | +| `phone.yml` | `phone/`、`crates/`、`apps/`、工作区文件、`tools/` | 在 macOS 上编译 Home 及其内置模块，Shell 依赖图检查，并运行 Shell、Home、AI 服务、App Hub 准入和运行时策略的测试 | +| `apps.yml` | `apps/`、`crates/`、工作区文件、`tools/setup.py` | 内核服务、app peers、AI providers 配置、邮件与 `llm` 宿主服务、Shell 的 AI 服务、AppCard | | `rom.yml` | `rom/`、`phone/android/`、手机端的 Android 资源与测试、`tools/kernel-artifact.py` | 产品测试、生成的 Agent Binder 客户端、网页安装器 | -| `release-desktop.yml` | 推送的 `desktop-v*` 标签、手动运行，或修改打包的 pull request（只构建和扫描） | macOS、Windows、Linux 的未签名桌面包和私有路径扫描；对标签还会在 `release` 环境中签名并生成草稿 release（[desktop/README.zh-CN.md](desktop/README.zh-CN.md#发布构建)）。`tools/ci-local.sh` 不运行它。 | +| `release-desktop.yml` | `desktop-v*` 标签、手动运行，或修改打包的 pull request | macOS、Windows、Linux 的未签名桌面包和私有路径扫描；对标签还会签名并生成草稿 release（[desktop/README.zh-CN.md](desktop/README.zh-CN.md#发布构建)） | -每个工作流的依赖图检查（`tools/setup.py --check --cargo`）确保锁定的依赖图中只有一个 Makepad、一个 App Hub、一个 octos 和一个 Rinx。 +每个工作流的依赖图检查（`tools/setup.py --check --cargo`）确保锁定的依赖图中只有一个 Makepad、一个 App Hub、一个 octos 和一个 Rinx。`tools/ci-local.sh` 可以在你自己的机器上运行同样的步骤（[docs/local-ci.md（英文）](docs/local-ci.md)）。 ## 发布 -按 ADR 0001，每个产品单独打标签：`desktop-v*`、`home-v*`（APK）、`rom-v*`（镜像），构建回执记录仓库提交。`desktop-v*` 标签会把桌面包（`.dmg`、Windows 安装包、`.deb`、`.AppImage`）构建到草稿 release 中（[发布构建](desktop/README.zh-CN.md#发布构建)）。系统应用只随 Shell 一起发布、按摘要准入，不单独发布。仓库合并前发布的 ROM 版本 `20260919-j` 现为本仓库的 [`rom-v20260919-j`](https://github.com/OctoSense-org/OctoSense/releases/tag/rom-v20260919-j)。手机从固定移动的 `rom-latest` release 读取 `update.json`，而不是 `releases/latest`（[rom/docs/updates.md（英文）](rom/docs/updates.md)）。`20260919-j` 及更早的镜像检查的是已停用的 OctoSense-ROM 仓库，因此刷了这些镜像的手机需要重新刷写一次，才能收到 OTA 更新。 +每个产品单独打标签：`desktop-v*`、`home-v*`（APK）和 `rom-v*`（镜像），构建回执记录仓库提交。`desktop-v*` 标签会把桌面包（`.dmg`、Windows 安装包、`.deb`、`.AppImage`）构建到草稿 release 中（[发布构建](desktop/README.zh-CN.md#发布构建)）。系统应用只随 Shell 一起发布，按摘要准入。手机如何获取 ROM 更新，见 [rom/docs/updates.md（英文）](rom/docs/updates.md)。 ## 参与贡献 -`main` 受保护：每个改动都要通过 pull request，禁止强制推送。一个改动就是一个 pull request，按需同时修改 `desktop/`、`phone/`、`crates/` 和 `apps/`；没有内部版本锁需要移动。面向人和编码 Agent 的规则见 [AGENTS.md（英文）](AGENTS.md)。 +`main` 受保护：每个改动都要通过 pull request，禁止强制推送。一个改动就是一个 pull request，按需同时修改 `desktop/`、`phone/`、`crates/` 和 `apps/`。面向人和编码 Agent 的规则见 [AGENTS.md（英文）](AGENTS.md)。 ## 许可证 diff --git a/desktop/README.md b/desktop/README.md index b3f3715b..9d2be9a2 100644 --- a/desktop/README.md +++ b/desktop/README.md @@ -125,6 +125,7 @@ App Hub's modules have no process form and always open in-process. | `OCTOS_APP_CORE_BIN`, `OCTOS_APP_CORE_DIR` | The octos kernel binary the shell's kernel service runs, unchecked (unset: the packaged `octos-kernel`, see [Build and run](#build-and-run)) and its core dir (default `~/octos-home/.octos`; the AI providers profile is `/profiles/_main.json`). | | `OCTOSENSE_GLANCE_DEMO=1` | Publish a sample L0 News digest card (as `os.news`) to the glance screen at startup: F9 on a desktop style, the glance page on a phone style. A test path for the `glance` service. | | `OCTOSENSE_GLANCE_DEMO=mail` | Publish two fake Mail action cards (L0, as `os.mail`, with a toast each) at startup: clicking a toast opens that card in the card window, where Reply, Send (demo), Ask and Track work on fake data. They work the same in the glance panel, which opens with them, and on a phone style's glance page. No mail is read and no model is called. `scripts/mail_card_remote.sh` drives it hidden. | +| `OCTOSENSE_REDUCE_MOTION=1` | Keep the glance panel and its toasts still instead of sliding them in and out ([The glance panel](#the-glance-panel)). | | `MAKEPAD_REMOTE`, `MAKEPAD_HIDE_WINDOWS` | Remote-control bridge; hidden windows (see [Demos](#demos)). | ## Application icons @@ -321,6 +322,20 @@ cargo run --release -p octosense --features app-appcard -- --module appcard It starts no kernel of its own: it connects to the shell's. On desktop that is the packaged `octos-kernel` or `OCTOS_APP_CORE_BIN` (and optionally `OCTOS_APP_CORE_DIR`); without a kernel it shows its login / WebSocket screen. Every octos crate comes from octos-org/octos at the one revision the root `Cargo.toml` pins. +## The glance panel + +Apps and their agents publish glance cards with `glance.publish` ([how cards work](../README.md#cards-and-questions)). On the desktop they appear here: + +| Surface | What happens | +| --- | --- | +| Panel | A new card opens the glance panel unless a card window is already open. The bar’s bell or F9 also opens the panel. A press on a card outside its own controls opens it in the card window. The hovered card shows its open and dismiss actions, and a card that just came wears an accent mark for a few seconds. A card that does not fit whole peeks in at the end of the list. A card taller than its tile scrolls inside it with the wheel; a card with an in-card chat stays at its newest rows, so its field and the latest exchange stay in view. Opened with F9 (or a press in it), the panel has the keyboard: the arrows move a focus ring from card to card, Return opens the card, Delete dismisses it and Esc closes the panel. | +| Notifications | A card published with `notify` also posts a toast with the app’s icon and name, the card’s title and its `summary` (else the card’s own). Selecting the toast opens the card in its own window. At most three toasts show at once; a “+N more” chip under them shows the rest. While the panel is open, toasts stack to its left. | +| Dismissal | The hovered card shows a dismiss button: `glance::dismiss` removes the card as if the app had withdrawn it. Clear all dismisses every card. A dismissal can be undone from the toast that reports it, or with ⌘Z while the panel has the keyboard. The panel has a separate close button. | + +On a phone, `notify` posts a shade notification instead, and selecting it opens the glance page. + +A card that names no theme takes the shell’s light or dark palette. Toasts and the panel slide in; `OCTOSENSE_REDUCE_MOTION=1` keeps them still. The code: [`glance_panel.rs`](../crates/shell/src/glance_panel.rs), [`glance_sheet.rs`](../crates/shell/src/glance_sheet.rs) and [`notifications.rs`](../crates/shell/src/shell/notifications.rs). + ## Demos ### Mail without an account diff --git a/desktop/README.zh-CN.md b/desktop/README.zh-CN.md index 09d0d6df..a29ef3fb 100644 --- a/desktop/README.zh-CN.md +++ b/desktop/README.zh-CN.md @@ -125,6 +125,7 @@ App Hub 的模块没有进程形态，总是在进程内打开。 | `OCTOS_APP_CORE_BIN`、`OCTOS_APP_CORE_DIR` | Shell 内核服务运行的 octos 内核二进制，不做检查（未设置：使用随附的 `octos-kernel`，见[构建与运行](#构建与运行)）及其 core 目录（默认 `~/octos-home/.octos`；AI 提供商的 profile 为 `/profiles/_main.json`）。 | | `OCTOSENSE_GLANCE_DEMO=1` | 启动时以 `os.news` 身份向一览屏发布一张示例 L0 新闻摘要卡片：桌面风格下按 F9 查看，手机风格下在一览页查看。用于测试 `glance` 服务。 | | `OCTOSENSE_GLANCE_DEMO=mail` | 启动时以 `os.mail` 身份发布两张假的邮件操作卡片（L0，各带一条通知）：点击通知会在卡片窗口中打开对应卡片，可用假数据试用回复、发送（演示）、提问和跟踪。在随卡片打开的一览面板里，以及手机风格的一览页上，它们同样可用。不读取邮件，也不调用模型。`scripts/mail_card_remote.sh` 以隐藏窗口驱动它。 | +| `OCTOSENSE_REDUCE_MOTION=1` | 一览面板及其 toast 不再滑入滑出，保持静止（见[一览面板](#一览面板)）。 | | `MAKEPAD_REMOTE`、`MAKEPAD_HIDE_WINDOWS` | 远程控制桥；隐藏窗口（见[演示](#演示)）。 | ## 应用图标 @@ -318,6 +319,20 @@ cargo run --release -p octosense --features app-appcard -- --module appcard 它不会自己启动内核，而是连接 Shell 的内核。在桌面上即随附的 `octos-kernel` 或 `OCTOS_APP_CORE_BIN`（`OCTOS_APP_CORE_DIR` 可选）；没有内核时显示登录 / WebSocket 界面。所有 octos crate 都来自 octos-org/octos，且只有根目录 `Cargo.toml` 固定的那一个版本。 +## 一览面板 + +应用及其 Agent 用 `glance.publish` 发布一览卡片（[卡片如何工作](../README.zh-CN.md#卡片与提问)）。在桌面端，它们显示在这里： + +| 界面 | 行为 | +| --- | --- | +| 面板 | 新卡片会打开一览面板，除非已有卡片窗口打开。顶栏铃铛或 F9 也可以打开面板。点击卡片上自身控件以外的地方，会在卡片窗口中打开它。鼠标悬停的卡片会显示打开和移除操作，刚到的卡片旁会有几秒钟的强调色标记。放不下的那张卡片会在列表末尾露出一部分。卡片在面板里有高度上限，更高的卡片可以用滚轮在原处滚动；带卡内对话的卡片停在最新的消息处，输入框和最近一轮对话始终可见。用 F9（或在面板里点击）打开时，面板接管键盘：方向键在卡片之间移动焦点环，Return 打开卡片，Delete 移除卡片，Esc 关闭面板。 | +| 通知 | 以 `notify` 发布的卡片还会弹出 toast，显示应用的图标和名称、卡片标题及其 `summary`（没有时用卡片自带的摘要）。点击 toast 会在独立窗口中打开卡片。同时最多显示三条 toast，其余的由下方的“+N more”标签展开。面板打开时，toast 叠放在面板左侧。 | +| 移除 | 鼠标悬停的卡片会显示移除按钮：`glance::dismiss` 会移除卡片，效果如同应用撤回了它。“Clear all”会移除所有卡片。移除的卡片可以在报告这次移除的 toast 上撤销（Undo），面板接管键盘时也可以按 ⌘Z 撤销。面板本身另有关闭按钮。 | + +在手机上，`notify` 改为在通知栏发出通知，点击后打开一览页。 + +未声明主题的卡片使用 Shell 的浅色或深色配色。toast 和面板会滑入；设置 `OCTOSENSE_REDUCE_MOTION=1` 则保持静止。代码见 [`glance_panel.rs`](../crates/shell/src/glance_panel.rs)、[`glance_sheet.rs`](../crates/shell/src/glance_sheet.rs) 和 [`notifications.rs`](../crates/shell/src/shell/notifications.rs)。 + ## 演示 ### 无账户试用邮件 diff --git a/docs/architecture.md b/docs/architecture.md index 4f5679b1..8d0452b6 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -32,7 +32,7 @@ The decisions behind it are [ADR 0001](adr/0001-one-octosense-repository.md) (on ![OctoSense AI services: one granted cross-app call](images/ai-services-cross-app-call.png) -*Illustrations (generated). Solid lines are on main; dashed ones are planned or in progress (the peer link for process apps has its shell side on main since [#130](https://github.com/OctoSense-org/OctoSense/pull/130) but no process app uses it yet; `peer/input` and host tools are on main since the octos 665209e5 pin, octos#2567). Where a picture and the text below differ, the text is authoritative: for example, `peer_send_input` puts the message in the peer's inbox, and the peer takes it as its next turn.* +*Illustrations (generated). Solid lines are on main; dashed ones are planned or in progress (the peer link for process apps has been on main since [#130](https://github.com/OctoSense-org/OctoSense/pull/130), and the Terminal uses it; `peer/input` and host tools are on main since the octos 665209e5 pin, octos#2567). Where a picture and the text below differ, the text is authoritative: for example, `peer_send_input` puts the message in the peer's inbox, and the peer takes it as its next turn.* ```mermaid flowchart LR @@ -122,7 +122,8 @@ Native apps are first-party Rust crates declared only in [`native-apps.json`](.. | --- | --- | --- | --- | --- | --- | | App Hub (`apphub`: the store and the Card runner) | in process | in process | in process | default / default | – | | Rinx | in process | in process | in process | default / default | granted the four `octos.*` services | -| Terminal | **own process** | own process with a Vulkan build and a Wayland session, else in process | in process | default / off | – (its `run` tool is `confirm: host`, `auto_approvable: false`) | +| Terminal | **own process** | own process with a Vulkan build and a Wayland session, else in process | in process | default / off | its own agent, with its read tools only (`own_tools`); typing a command is the system agent's `terminal.run` (`confirm: host`, `auto_approvable: false`) | +| Calculator, Clock, Notes, Reminders, Weather | in process | in process | in process | default / default | the four `octos.*` services; their read tools also go to the system agent (`system_tools`) | | Sheets, Reference | in process | in process | in process | opt-in / `mobile-apps` | – | | AppCard | in process | in process | in process | opt-in / opt-in | its own kernel connection | @@ -172,7 +173,7 @@ flowchart TB | Agent | What it is | Status | | --- | --- | --- | | **The system agent** | the session `_main:api:octosense#system` on the `_main` profile (`SYSTEM_SESSION` in `crates/kernel/src/network.rs`). It owns every app peer and supervises them. The person reaches it in the shell's **system chat** (`crates/shell/src/system_chat/`, [#132](https://github.com/OctoSense-org/OctoSense/pull/132): Setup → Assistant → Assistant chat, F8, the dock's Assistant icon or the phone home's Assistant tile; on a desktop a medium pane at the left that the person can move and resize, full screen on a phone), or through a paired Talk to Octos client | On main | -| **An app agent** | one host-owned octos **peer** per (app, account), owned by the system agent (octos UPCR-2026-034, Rinx [ADR 0007](https://github.com/hagency-org/Rinx/blob/main/docs/adr/0007-host-owned-octos-app-peers.md)). The person talks to it in the app's own UI, or in the shell's **"Ask " panel** (`crates/shell/src/app_chat/`: the bar's "Ask ", Shift+F8; beside the system chat on a desktop, a full-screen sheet on a phone), which shows both lanes with their speakers and a composer: Send while the person's lane is idle (even while the system agent works), Stop for the person's own turn, and "Stop the system agent's task" on the system agent's running row | On main for Rinx (native) and every script app with an agent (today the system apps News, Mail, Photos, Maps, YouTube, Calendar on the desktop and Camera on phones), after first-use consent; see below | +| **An app agent** | one host-owned octos **peer** per (app, account), owned by the system agent (octos UPCR-2026-034, Rinx [ADR 0007](https://github.com/hagency-org/Rinx/blob/main/docs/adr/0007-host-owned-octos-app-peers.md)). The person talks to it in the app's own UI, or in the shell's **"Ask " panel** (`crates/shell/src/app_chat/`: the bar's "Ask ", Shift+F8; beside the system chat on a desktop, a full-screen sheet on a phone), which shows both lanes with their speakers and a composer: Send while the person's lane is idle (even while the system agent works), Stop for the person's own turn, and "Stop the system agent's task" on the system agent's running row | On main for the native apps with an agent (Rinx, the Terminal, Calculator, Clock, Notes, Reminders, Weather) and every script app with an agent (today the system apps News, Mail, Photos, Maps, YouTube, Calendar on the desktop and Camera on phones), after first-use consent; see below | Each app peer has, separately from every other: @@ -184,10 +185,10 @@ Each app peer has, separately from every other: Who gets a peer today (`crates/ai-host/src/lib.rs`, `Policy::shipped()`; `crates/app-peers/src/hosted.rs`, `effective_services` = declared ∩ supported ∩ policy): -- **Rinx**, the only native app granted the assistant (the four `octos.*` services, from its `native-apps.json` `agent.octos`, generated into `crates/ai-host/src/native_agents.rs`), after first-use consent. +- **Native apps** whose `native-apps.json` entry grants `agent.octos` (the four `octos.*` services, generated into `crates/ai-host/src/native_agents.rs`): Rinx, the Terminal, Calculator, Clock, Notes, Reminders and Weather, after first-use consent. A native app's peer is its open instance's, so it exists while the app is open (`agents::prepare` skips native apps). - **Script apps with an agent**: a manifest that declares `octos.*` or an `agent` block, or a bundle that ships `tools.json` (`crates/shell/src/apps.rs`, `agent_apps`). Today that is every system app except AI providers: News, Mail, Photos, Maps, YouTube, Calendar on the desktop and Camera on phones, each with an `agent` block and `tools.json` and none declaring `octos.*`. One peer per app/account with broker identity `card.`, using `device` for accountless apps or the host-reported active account (Mail requires sign-in) (`crates/ai-host/src/contained.rs`, [#106](https://github.com/OctoSense-org/OctoSense/pull/106)), once the person allowed it at first use (the shipped gate; `OCTOSENSE_CONTAINED_APPS=1` asks nobody, `0` turns it off). The shell prepares it as soon as it is allowed, and at startup (`crates/shell/src/agents.rs`, `contained::prepare`), so the system agent's `peer_list` shows it even when the app never calls `octos`; the app's own `host.request("octos.*")` stays limited to what its manifest declares. - **AppCard** (opt-in) takes its own kernel connection and sessions, not a peer. -- **Process apps** whose `native-apps.json` entry grants `agent.octos` services reach their peer through the **peer link** (`crates/shell/src/peer_link/`, [#130](https://github.com/OctoSense-org/OctoSense/pull/130)), after first-use consent. None does yet: the Terminal, the only process app, grants none. +- **Process apps** whose `native-apps.json` entry grants `agent.octos` services reach their peer through the **peer link** (`crates/shell/src/peer_link/`, [#130](https://github.com/OctoSense-org/OctoSense/pull/130)), after first-use consent. The Terminal does: it opens its agent's link with Makepad's `OctosPeer` and answers its read tools there. Peers are kept, not thrown away: the broker resumes a peer with its host token (stored under `/../app-peers`, 0600), and never calls `peer_close` on sign-out, because octos cannot resume a closed peer or create a replacement for its (app, account) (ADR 0004 §11). @@ -211,7 +212,7 @@ flowchart LR end rinx["Rinx (module)"] -- "OctosAppService" --> broker card["script app"] -- "host.request(octos.*)" --> broker - proc["process app"] -. "peer link (shell side on main, no user yet)" .-> broker + proc["process app"] -- "peer link (the Terminal)" --> broker broker -- "OUP: peer/prepare, peer/context/open, turn/start" --> kernel sys -- "peer_send_input" --> p1 p1 -- "writes" --> bb @@ -251,10 +252,10 @@ For a host-owned peer, a plain `peer_send_input` runs as a kernel continuation w | Hosting | Channel | Status | | --- | --- | --- | -| In-process native module, through the peer link | Makepad's `OctosPeer::open` parks a channel pair; the module host claims it for the instance whose code opened it and the shell serves it with the process apps' **peer link** (`peer_link::module_connected`, `octosense_ai_host::module_peer`), so the app does not know how it is hosted | On main ([#142](https://github.com/OctoSense-org/OctoSense/issues/142)); no module uses it yet | +| In-process native module, through the peer link | Makepad's `OctosPeer::open` parks a channel pair; the module host claims it for the instance whose code opened it and the shell serves it with the process apps' **peer link** (`peer_link::module_connected`, `octosense_ai_host::module_peer`), so the app does not know how it is hosted | On main ([#142](https://github.com/OctoSense-org/OctoSense/issues/142)); used by Calculator, Clock, Notes, Reminders and Weather, and by the Terminal where it runs in process | | In-process native module (Rinx) | the **injected service**: `ai_host::offer` before `create`, `octosense_app_peers::injection::claim` inside it, giving a scoped `OctosAppService` (`Open`, `History`, `Turn`, `Interrupt`, `Approval`); the module never sees the protocol | On main | | Script app | `host.request("octos.session.open" / "octos.session.history" / "octos.turn.start" / "octos.turn.interrupt", …)` to the `octos` host service (`crates/ai-host/src/contained.rs`); gated by the manifest, `Policy::contained_gate` (`ContainedGate::Consent` by default) and first-use consent; tool approvals its peer raises go to the shell's router. None of the system apps calls `octos.*` or draws a chat: the person talks to their agents in the shell's "Ask " panel, on the same peer, and the shell drives them for the system agent | On main ([#106](https://github.com/OctoSense-org/OctoSense/pull/106), consent from [#120](https://github.com/OctoSense-org/OctoSense/pull/120)) | -| Native app in its own process | the **peer link**: its own channel on the app's hub connection (`PeerRequest`, `PeerReply`, `PeerToolCall` with the identity and caller stamped by the shell), never registered with the AI bus; client API in Makepad's `makepad-ai-services` | Shell side on main ([#130](https://github.com/OctoSense-org/OctoSense/pull/130), `crates/shell/src/peer_link/`, frames routed from the hub connection in `lib.rs`); no process app is granted an agent yet | +| Native app in its own process | the **peer link**: its own channel on the app's hub connection (`PeerRequest`, `PeerReply`, `PeerToolCall` with the identity and caller stamped by the shell), never registered with the AI bus; client API in Makepad's `makepad-ai-services` | Shell side on main ([#130](https://github.com/OctoSense-org/OctoSense/pull/130), `crates/shell/src/peer_link/`, frames routed from the hub connection in `lib.rs`); used by the Terminal | ### Makepad's AI services bus vs OctoSense's app agents @@ -265,9 +266,9 @@ Two models live side by side (ADR 0004, "Two AI models"): | Where | the window manager's half in `crates/shell/src/ai_bus.rs`; upstream `libs/ai/services` | `crates/ai-host`, `crates/app-peers`, the octos kernel | | Shape | **one central conversation** (the desktop's AI pane, Makepad's `aichat`) calls typed tools that apps register with a risk level (`Read`, `Act`, `Destructive`) | **one agent per app** with the app's full context (workspace, memory, history, tools), supervised by the system agent | | Routing | the shell stamps each up-frame with the sender's endpoint, forwards registrations to the pane (replayed on reconnect), routes the pane's calls to the app's socket, and answers the `os` service (list, launch, focus, close, open) itself | the broker talks OUP to the kernel directly | -| Used for | the AI pane on the desktop; Rinx's assistant tools today; the Terminal's `run`. Its `confirm: host` calls go through the shell's approval router ([#120](https://github.com/OctoSense-org/OctoSense/pull/120)) | the apps' own assistants; delegation from the system agent | +| Used for | the AI pane on the desktop; Rinx's assistant tools today; the Terminal's `run`; a native app's own tools when the app holds no peer link (`host_tools/relay.rs`, `serves_on_bus`). Its `confirm: host` calls go through the shell's approval router ([#120](https://github.com/OctoSense-org/OctoSense/pull/120)) | the apps' own assistants; delegation from the system agent | -**Why app-agent traffic does not use the bus.** The bus is a narrow, one-way API to a central agent that holds all the context; an app agent needs its app's full context and a private, supervised session, and the system agent must address it through the kernel's peer mechanism. The bus also carries no account, request context or caller, so it cannot stamp the identity ADR 0004 §5 requires on every tool call, and its registrations are visible to the pane. So in OctoSense the bus is not the system agent's channel to apps; it stays for upstream Makepad apps (ADR 0004 §6), and the peer link for process apps is deliberately a separate channel. Neither `crates/ai-host` nor `crates/app-peers` uses the bus. +**Why app-agent traffic does not use the bus.** The bus is a narrow, one-way API to a central agent that holds all the context; an app agent needs its app's full context and a private, supervised session, and the system agent must address it through the kernel's peer mechanism. The bus also carries no account, request context or caller, so it cannot stamp the identity ADR 0004 §5 requires on every tool call, and its registrations are visible to the pane. So in OctoSense the bus is not how the system agent talks to app agents; the shell's relay uses it only to run a native app's tool in the app's open instance (the Terminal's host-confirmed `run`, and an app's own tools when it holds no peer link). It stays for upstream Makepad apps (ADR 0004 §6), and the peer link for process apps is deliberately a separate channel. Neither `crates/ai-host` nor `crates/app-peers` uses the bus. ## 4. Tools and grants @@ -395,7 +396,7 @@ One host-owned layout for every app, declared by its manifest (`storage` block) | Script app ↔ shell | App Hub's nested isolate: no `mod.res` / `mod.run`, a jail and quota, only `host.request` for granted families; password fields inert; secrets on host sheets | On main | | Native module ↔ shell | none in memory: reviewed first-party code only (`native-apps.json`); **panic containment** at the module boundary (`catch_unwind`, [#114](https://github.com/OctoSense-org/OctoSense/pull/114)); limits: a panic while unwinding, `panic = "abort"`, FFI | On main | | Process app ↔ shell | its own address space; an **OS sandbox** from the manifest's `sandbox` and `storage` (macOS sandbox profile, Linux Landlock and seccomp, Windows AppContainer) | macOS on main (`crates/shell/src/sandbox/macos.rs`, a Seatbelt profile, built and tested; [#130](https://github.com/OctoSense-org/OctoSense/pull/130)); Linux Landlock and seccomp on main (`crates/shell/src/sandbox/linux.rs`), validated on a real kernel on 2026-09-30 ([#138](https://github.com/OctoSense-org/OctoSense/issues/138), fixes [#199](https://github.com/OctoSense-org/OctoSense/pull/199), [#200](https://github.com/OctoSense-org/OctoSense/pull/200), [#202](https://github.com/OctoSense-org/OctoSense/pull/202)); their tests run in local CI's Linux-host job `linux-host / sandbox` ([local-ci.md](local-ci.md)), not in GitHub CI; Windows planned ([#137](https://github.com/OctoSense-org/OctoSense/issues/137)): a process app there runs with the user's rights | -| App ↔ kernel | apps never speak the kernel protocol and **never see the host token**: modules get a scoped `OctosAppService`, script apps a host service, process apps the peer link | On main | +| App ↔ kernel | apps never speak the kernel protocol and **never see the host token**: modules get the peer link or a scoped `OctosAppService`, script apps a host service, process apps the peer link | On main | | Peer ↔ peer | octos: separate workspace (overlapping workspaces refused), memory namespace, transcript; request contexts fenced to `contexts//` | On main (octos UPCR-2026-034) | | Agent ↔ secrets | secrets outside every jail and workspace; startup check | On main | | External client ↔ kernel | external token, method and tool allowlists, `Host` and origin checks, no peer access | On main ([ADR 0003](adr/0003-shared-octos-client-access.md)) | diff --git a/docs/architecture.zh-CN.md b/docs/architecture.zh-CN.md index 336f8a4d..39872d40 100644 --- a/docs/architecture.zh-CN.md +++ b/docs/architecture.zh-CN.md @@ -32,7 +32,7 @@ OctoSense 的进程、Agent、工具、审批、存储与信任边界。具体 ![OctoSense AI 服务：一次已授权的跨应用调用](images/ai-services-cross-app-call.png) -*示意图（生成）。实线表示已在 main 上；虚线表示计划中或进行中（进程应用的 peer link 自 [#130](https://github.com/OctoSense-org/OctoSense/pull/130) 起 Shell 一侧已在 main 上，但还没有进程应用使用它；`peer/input` 和宿主工具自锁定 octos 665209e5 起已在 main，octos#2567）。图与下文不一致时以文字为准：例如 `peer_send_input` 把消息放入 peer 的收件箱，peer 在下一轮处理它。* +*示意图（生成）。实线表示已在 main 上；虚线表示计划中或进行中（进程应用的 peer link 自 [#130](https://github.com/OctoSense-org/OctoSense/pull/130) 起已在 main 上，Terminal 正在使用；`peer/input` 和宿主工具自锁定 octos 665209e5 起已在 main，octos#2567）。图与下文不一致时以文字为准：例如 `peer_send_input` 把消息放入 peer 的收件箱，peer 在下一轮处理它。* ```mermaid flowchart LR @@ -122,7 +122,8 @@ flowchart LR | --- | --- | --- | --- | --- | --- | | App Hub（`apphub`：商店和 Card runner） | 进程内 | 进程内 | 进程内 | 默认 / 默认 | – | | Rinx | 进程内 | 进程内 | 进程内 | 默认 / 默认 | 获授权四个 `octos.*` 服务 | -| Terminal | **独立进程** | Vulkan 构建且在 Wayland 会话中时为独立进程，否则进程内 | 进程内 | 默认 / 关闭 | –（其 `run` 工具为 `confirm: host`、`auto_approvable: false`） | +| Terminal | **独立进程** | Vulkan 构建且在 Wayland 会话中时为独立进程，否则进程内 | 进程内 | 默认 / 关闭 | 有自己的 Agent，只有只读工具（`own_tools`）；输入命令是系统 Agent 的 `terminal.run`（`confirm: host`、`auto_approvable: false`） | +| Calculator、Clock、Notes、Reminders、Weather | 进程内 | 进程内 | 进程内 | 默认 / 默认 | 获授权四个 `octos.*` 服务；它们的只读工具也提供给系统 Agent（`system_tools`） | | Sheets、Reference | 进程内 | 进程内 | 进程内 | 可选 / `mobile-apps` | – | | AppCard | 进程内 | 进程内 | 进程内 | 可选 / 可选 | 自己的内核连接 | @@ -172,7 +173,7 @@ flowchart TB | Agent | 是什么 | 状态 | | --- | --- | --- | | **系统 Agent** | `_main` profile 上的会话 `_main:api:octosense#system`（`crates/kernel/src/network.rs` 中的 `SYSTEM_SESSION`）。它拥有并监督所有应用 peer。用户在 Shell 的**系统对话**中与它对话（`crates/shell/src/system_chat/`，[#132](https://github.com/OctoSense-org/OctoSense/pull/132)：Setup → Assistant → Assistant chat、F8、桌面 Dock 上的 Assistant 图标或手机主屏的 Assistant 磁贴；桌面上是左侧一个中等大小、用户可以移动和调整大小的面板，手机上为全屏），或通过已配对的 Talk to Octos 客户端 | 已在 main | -| **应用 Agent** | 每个（应用，账号）一个由宿主拥有的 octos **peer**，归系统 Agent 所有（octos UPCR-2026-034，Rinx [ADR 0007](https://github.com/hagency-org/Rinx/blob/main/docs/adr/0007-host-owned-octos-app-peers.md)）。用户可在应用自己的界面中，或在 Shell 的 **“Ask ” 面板**（`crates/shell/src/app_chat/`：顶栏的 “Ask ”、Shift+F8；桌面上位于系统对话旁边，手机上为全屏面板）与它对话。面板显示两条通道及其发言者，以及一个输入框：用户的通道空闲时可以发送（即使系统 Agent 正在工作），用户自己的回合运行时可以停止，系统 Agent 正在运行的那一行上有 “Stop the system agent's task” | Rinx（原生）和每个带 Agent 的脚本应用（目前是系统应用 News、Mail、Photos、Maps、YouTube、仅桌面端的 Calendar 和仅手机上的 Camera）已在 main，需首次使用同意；见下文 | +| **应用 Agent** | 每个（应用，账号）一个由宿主拥有的 octos **peer**，归系统 Agent 所有（octos UPCR-2026-034，Rinx [ADR 0007](https://github.com/hagency-org/Rinx/blob/main/docs/adr/0007-host-owned-octos-app-peers.md)）。用户可在应用自己的界面中，或在 Shell 的 **“Ask ” 面板**（`crates/shell/src/app_chat/`：顶栏的 “Ask ”、Shift+F8；桌面上位于系统对话旁边，手机上为全屏面板）与它对话。面板显示两条通道及其发言者，以及一个输入框：用户的通道空闲时可以发送（即使系统 Agent 正在工作），用户自己的回合运行时可以停止，系统 Agent 正在运行的那一行上有 “Stop the system agent's task” | 带 Agent 的原生应用（Rinx、Terminal、Calculator、Clock、Notes、Reminders、Weather）和每个带 Agent 的脚本应用（目前是系统应用 News、Mail、Photos、Maps、YouTube、仅桌面端的 Calendar 和仅手机上的 Camera）已在 main，需首次使用同意；见下文 | 每个应用 peer 都独立拥有： @@ -184,10 +185,10 @@ flowchart TB 目前谁有 peer（`crates/ai-host/src/lib.rs`，`Policy::shipped()`；`crates/app-peers/src/hosted.rs`，`effective_services` = 声明 ∩ 支持 ∩ 策略）： -- **Rinx**，唯一获授权使用助手的原生应用（四个 `octos.*` 服务，来自它在 `native-apps.json` 中的 `agent.octos`，生成到 `crates/ai-host/src/native_agents.rs`），需首次使用时的同意。 +- `native-apps.json` 条目授予了 `agent.octos`（四个 `octos.*` 服务，生成到 `crates/ai-host/src/native_agents.rs`）的**原生应用**：Rinx、Terminal、Calculator、Clock、Notes、Reminders 和 Weather，需首次使用时的同意。原生应用的 peer 属于它已打开的实例，所以只在应用打开期间存在（`agents::prepare` 跳过原生应用）。 - **带 Agent 的脚本应用**：清单声明了 `octos.*` 或 `agent` 块，或应用包带有 `tools.json`（`crates/shell/src/apps.rs`，`agent_apps`）。目前就是除 AI providers 之外的每个系统应用：News、Mail、Photos、Maps、YouTube、仅桌面端的 Calendar 和仅手机上的 Camera，每个都有 `agent` 块和 `tools.json`，都没有声明 `octos.*`。每个应用/账号一个 peer，broker 身份为 `card.`；无账号的应用使用 `device`，否则使用宿主报告的当前账号（Mail 需要先登录）（`crates/ai-host/src/contained.rs`，[#106](https://github.com/OctoSense-org/OctoSense/pull/106)），在用户首次使用同意后可用（发布时的闸门；`OCTOSENSE_CONTAINED_APPS=1` 不询问任何人，`0` 关闭）。用户一旦同意，以及每次启动时，Shell 就会准备其 peer（`crates/shell/src/agents.rs`，`contained::prepare`），因此即使应用从不调用 `octos`，系统 Agent 的 `peer_list` 也能看到它；应用自己的 `host.request("octos.*")` 仍限于其清单声明的服务。 - **AppCard**（可选）使用自己的内核连接和会话，而不是 peer。 -- 其 `native-apps.json` 条目授予了 `agent.octos` 服务的**进程应用**，经 **peer link**（`crates/shell/src/peer_link/`，[#130](https://github.com/OctoSense-org/OctoSense/pull/130)）访问自己的 peer，需首次使用同意。目前还没有这样的应用：唯一的进程应用 Terminal 没有授予任何服务。 +- 其 `native-apps.json` 条目授予了 `agent.octos` 服务的**进程应用**，经 **peer link**（`crates/shell/src/peer_link/`，[#130](https://github.com/OctoSense-org/OctoSense/pull/130)）访问自己的 peer，需首次使用同意。Terminal 就是这样：它用 Makepad 的 `OctosPeer` 打开自己 Agent 的链接，并在上面应答自己的只读工具。 peer 会被保留而不是丢弃：broker 用其宿主 token（存放在 `/../app-peers`，0600）恢复 peer，登出时也从不调用 `peer_close`，因为 octos 无法恢复已关闭的 peer，也无法为同一个（应用，账号）创建替代者（ADR 0004 §11）。 @@ -211,7 +212,7 @@ flowchart LR end rinx["Rinx（模块）"] -- "OctosAppService" --> broker card["脚本应用"] -- "host.request(octos.*)" --> broker - proc["进程应用"] -. "peer link（Shell 一侧已在 main，尚无使用者）" .-> broker + proc["进程应用"] -- "peer link（Terminal）" --> broker broker -- "OUP：peer/prepare、peer/context/open、turn/start" --> kernel sys -- "peer_send_input" --> p1 p1 -- "写入" --> bb @@ -251,10 +252,10 @@ flowchart LR | 托管方式 | 通道 | 状态 | | --- | --- | --- | -| 进程内原生模块，经 peer link | Makepad 的 `OctosPeer::open` 暂存一对通道；模块宿主把它归给打开它的实例，Shell 用进程应用同一套 **peer link** 服务它（`peer_link::module_connected`、`octosense_ai_host::module_peer`），应用无需知道自己如何被承载 | 已在 main（[#142](https://github.com/OctoSense-org/OctoSense/issues/142)）；尚无模块使用 | +| 进程内原生模块，经 peer link | Makepad 的 `OctosPeer::open` 暂存一对通道；模块宿主把它归给打开它的实例，Shell 用进程应用同一套 **peer link** 服务它（`peer_link::module_connected`、`octosense_ai_host::module_peer`），应用无需知道自己如何被承载 | 已在 main（[#142](https://github.com/OctoSense-org/OctoSense/issues/142)）；Calculator、Clock、Notes、Reminders 和 Weather 在用，Terminal 在进程内运行时也在用 | | 进程内原生模块（Rinx） | **注入的服务**：`create` 之前调用 `ai_host::offer`，在其中调用 `octosense_app_peers::injection::claim`，得到受限的 `OctosAppService`（`Open`、`History`、`Turn`、`Interrupt`、`Approval`）；模块永远看不到协议 | 已在 main | | 脚本应用 | 向 `octos` 宿主服务（`crates/ai-host/src/contained.rs`）调用 `host.request("octos.session.open" / "octos.session.history" / "octos.turn.start" / "octos.turn.interrupt", …)`；受清单、`Policy::contained_gate`（默认 `ContainedGate::Consent`）和首次使用同意约束；其 peer 发起的工具审批交给 Shell 的审批路由。系统应用都不调用 `octos.*`，也都不绘制对话界面：用户在 Shell 的 “Ask ” 面板中与它们的 Agent 对话，用的是同一个 peer，Shell 也为系统 Agent 驱动它们 | 已在 main（[#106](https://github.com/OctoSense-org/OctoSense/pull/106)，同意机制来自 [#120](https://github.com/OctoSense-org/OctoSense/pull/120)） | -| 独立进程的原生应用 | **peer link**：应用 hub 连接上的独立通道（`PeerRequest`、`PeerReply`，以及由 Shell 盖上身份和调用方的 `PeerToolCall`），从不注册到 AI 总线；客户端 API 在 Makepad 的 `makepad-ai-services` 中 | Shell 一侧已在 main（[#130](https://github.com/OctoSense-org/OctoSense/pull/130)，`crates/shell/src/peer_link/`，帧由 `lib.rs` 从 hub 连接转发）；尚无进程应用被授予 Agent | +| 独立进程的原生应用 | **peer link**：应用 hub 连接上的独立通道（`PeerRequest`、`PeerReply`，以及由 Shell 盖上身份和调用方的 `PeerToolCall`），从不注册到 AI 总线；客户端 API 在 Makepad 的 `makepad-ai-services` 中 | Shell 一侧已在 main（[#130](https://github.com/OctoSense-org/OctoSense/pull/130)，`crates/shell/src/peer_link/`，帧由 `lib.rs` 从 hub 连接转发）；Terminal 在用 | ### Makepad 的 AI 服务总线与 OctoSense 的应用 Agent @@ -265,9 +266,9 @@ flowchart LR | 位置 | 窗口管理器一侧在 `crates/shell/src/ai_bus.rs`；上游 `libs/ai/services` | `crates/ai-host`、`crates/app-peers`、octos 内核 | | 形态 | **一个中心对话**（桌面端的 AI 面板，即 Makepad 的 `aichat`）调用应用以风险等级（`Read`、`Act`、`Destructive`）注册的类型化工具 | **每个应用一个 Agent**，拥有应用的完整上下文（工作区、记忆、历史、工具），由系统 Agent 监督 | | 路由 | Shell 给每个上行帧盖上发送方的端点，把注册转给面板（面板重连时重放），把面板的调用路由到应用的套接字，并自己回答 `os` 服务（list、launch、focus、close、open） | broker 直接用 OUP 与内核通信 | -| 用途 | 桌面端的 AI 面板；目前 Rinx 的助手工具；Terminal 的 `run`。其 `confirm: host` 调用经过 Shell 的审批路由（[#120](https://github.com/OctoSense-org/OctoSense/pull/120)） | 应用自己的助手；系统 Agent 的委派 | +| 用途 | 桌面端的 AI 面板；目前 Rinx 的助手工具；Terminal 的 `run`；原生应用没有 peer link 时它自己的工具（`host_tools/relay.rs`，`serves_on_bus`）。其 `confirm: host` 调用经过 Shell 的审批路由（[#120](https://github.com/OctoSense-org/OctoSense/pull/120)） | 应用自己的助手；系统 Agent 的委派 | -**为什么应用 Agent 的流量不走总线。** 总线是通向一个掌握全部上下文的中心 Agent 的窄而单向的 API；应用 Agent 需要应用的完整上下文和一个私有、受监督的会话，而系统 Agent 必须通过内核的 peer 机制与它通信。总线也不携带账号、请求上下文或调用方，无法按 ADR 0004 §5 的要求在每次工具调用上盖上身份，而且其注册对面板可见。因此在 OctoSense 中，总线不是系统 Agent 通向应用的通道；它留给上游 Makepad 应用（ADR 0004 §6），进程应用的 peer link 也有意设计为独立通道。`crates/ai-host` 和 `crates/app-peers` 都不使用总线。 +**为什么应用 Agent 的流量不走总线。** 总线是通向一个掌握全部上下文的中心 Agent 的窄而单向的 API；应用 Agent 需要应用的完整上下文和一个私有、受监督的会话，而系统 Agent 必须通过内核的 peer 机制与它通信。总线也不携带账号、请求上下文或调用方，无法按 ADR 0004 §5 的要求在每次工具调用上盖上身份，而且其注册对面板可见。因此在 OctoSense 中，总线不是系统 Agent 与应用 Agent 对话的通道；Shell 的中转只用它在原生应用已打开的实例中运行工具（Terminal 由宿主确认的 `run`，以及应用没有 peer link 时它自己的工具）。它留给上游 Makepad 应用（ADR 0004 §6），进程应用的 peer link 也有意设计为独立通道。`crates/ai-host` 和 `crates/app-peers` 都不使用总线。 ## 4. 工具与授权 @@ -395,7 +396,7 @@ flowchart TB | 脚本应用 ↔ Shell | App Hub 的嵌套隔离环境：没有 `mod.res` / `mod.run`，有 jail 和配额，只能对获授权的服务族使用 `host.request`；密码输入框失效；机密在宿主面板上输入 | 已在 main | | 原生模块 ↔ Shell | 内存上没有边界：只接受经评审的第一方代码（`native-apps.json`）；模块边界上的 **panic 隔离**（`catch_unwind`，[#114](https://github.com/OctoSense-org/OctoSense/pull/114)）；局限：展开过程中再次 panic、`panic = "abort"`、FFI | 已在 main | | 进程应用 ↔ Shell | 独立的地址空间；依据清单的 `sandbox` 和 `storage` 生成的**操作系统沙箱**（macOS 沙箱配置、Linux Landlock 和 seccomp、Windows AppContainer） | macOS 已在 main（`crates/shell/src/sandbox/macos.rs`，Seatbelt 配置，已构建并测试；[#130](https://github.com/OctoSense-org/OctoSense/pull/130)）；Linux 的 Landlock 和 seccomp 已在 main（`crates/shell/src/sandbox/linux.rs`），2026-09-30 已在真实内核上验证（[#138](https://github.com/OctoSense-org/OctoSense/issues/138)，修复见 [#199](https://github.com/OctoSense-org/OctoSense/pull/199)、[#200](https://github.com/OctoSense-org/OctoSense/pull/200)、[#202](https://github.com/OctoSense-org/OctoSense/pull/202)）；它们的测试在本地 CI 的 Linux 主机作业 `linux-host / sandbox` 中运行（[local-ci.md](local-ci.md)，英文），不在 GitHub CI 中；Windows 规划中（[#137](https://github.com/OctoSense-org/OctoSense/issues/137)）：那里的进程应用以用户权限运行 | -| 应用 ↔ 内核 | 应用从不使用内核协议，**永远看不到宿主 token**：模块得到受限的 `OctosAppService`，脚本应用得到宿主服务，进程应用使用 peer link | 已在 main | +| 应用 ↔ 内核 | 应用从不使用内核协议，**永远看不到宿主 token**：模块使用 peer link 或受限的 `OctosAppService`，脚本应用得到宿主服务，进程应用使用 peer link | 已在 main | | peer ↔ peer | octos：独立的工作区（拒绝重叠的工作区）、记忆命名空间、对话记录；请求上下文被限定在 `contexts//` 内 | 已在 main（octos UPCR-2026-034） | | Agent ↔ 机密 | 机密位于所有 jail 和工作区之外；启动检查 | 已在 main | | 外部客户端 ↔ 内核 | 外部 token、方法和工具允许列表、`Host` 和 origin 检查、无法访问 peer | 已在 main（[ADR 0003](adr/0003-shared-octos-client-access.md)） | diff --git a/docs/images/agents-processes.png b/docs/images/agents-processes.png index a9193f00..4996828b 100644 Binary files a/docs/images/agents-processes.png and b/docs/images/agents-processes.png differ