Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
| Which apps have an agent, their tools, and the relay that checks and routes every agent tool call | `crates/shell/src/apps.rs` (`agent_apps`), `crates/shell/src/host_tools/` (`script_apps.rs`, `relay.rs`, `files.rs`) | from `phone/`: `cargo test --locked --features mobile-apps -p octosense-shell host_tools`; `phone.yml` |
| Approvals, first-use consent, standing rules, the audit | `crates/shell/src/approvals/` | from `phone/`: `cargo test --locked --features mobile-apps -p octosense-shell approvals`; `phone.yml` |
| Glance cards (`glance.publish`), the desktop's glance panel and card window, a card's in-card chat | `crates/shell/src/glance*.rs`; the shared `sys.chat` host side in `crates/l0-chat/` | `cargo test --locked -p octosense-l0-chat`; from `phone/`: `cargo test --locked --features mobile-apps -p octosense-shell glance`; `apps.yml`, `phone.yml` |
| App Studio: L0 preview, scoped app instrumentation and local developer installs | `crates/shell/src/studio/` (UI/GPU/instrument/module), `crates/shell/src/host_tools/studio.rs` (tools/workspaces), `studio_bundles.rs` beside it (admission/DevTag receipts), `docs/adr/0006-app-studio-on-the-phone.md` | from `phone/`: `cargo test --locked --features mobile-apps,dev-mode -p octosense-shell`; separate-package device checks must cover a freshly authored app, real widget input, measured UX, installed persistence and revocation; unit/GPU fixture tests alone are insufficient |
| The system toolbox's workflow templates (library, runner, forks, evaluation, `mod.research`) | `crates/toolbox/` (templates in `crates/toolbox/templates/<id>/`); offered to app agents by `crates/ai-host/src/toolbox_peers.rs` (feature `toolbox-peers`) | `cargo test --locked -p octosense-toolbox`, its README, `apps.yml` |
| A system app | `apps/<name>/bundle/` | App Hub's `card-host --bundle apps/<name>/bundle --system`; then in a shell |
| A system app's agent: its tools and the cards they publish | `apps/<name>/bundle/tools.json` and the manifest's `agent` block; the tools run on the app's host service | its host service's tests (below); then in a shell, with the system agent asking the app's agent |
Expand All @@ -31,6 +32,12 @@

Start every session with `python3 tools/setup.py` (it prepares `.sources/`, and changes nothing that is already right).

For local shell tests that link Rinx, set `OCTOSENSE_HOME` and `RINX_DATA_DIR`
to separate, empty test directories and `MAKEPAD_HIDE_WINDOWS=1`. Some hosted
module tests initialize Rinx; without the explicit data directory they can
restore the machine's existing account. Keep these directories outside the
checkout and never copy personal account state into them.

On a machine that already has clones of Makepad, OctoScript or Octoscript-Makepad, never let setup clone them again: name the clones as a hub (`~/.config/octosense/sources.json`, `--hub DIR` or `OCTOSENSE_SOURCES_HUB`; see the README's Set up) so every `.sources/` entry is a `git worktree` of the one clone, and run `python3 tools/setup.py --remove-worktrees` before deleting a checkout of this repository.

## Local CI
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ It was OctoSense-Desktop; OctoSense-ROM (retired; merged into this repository) a

New to the codebase? Start with [From an app window to an agent turn](docs/architecture-walkthrough.md). It follows executable entry points, native and script hosting, app data access, human/system conversations, tool routing and the actual Tokio tasks. The [product walkthrough](desktop/docs/code-walkthrough.md) adds desktop, Home, ROM and system-app run recipes.

[App Studio on the phone](docs/adr/0006-app-studio-on-the-phone.md) now has developer-only tools to check, open, inspect, exercise and locally install an offline `main.splash` app. Preview state is disposable; installed app state persists under its developer grant. Follow the [Studio code walkthrough](docs/architecture-walkthrough.md#follow-app-studio-from-the-agent-to-a-working-app).

A fresh model-authored Task Planner passed 129 physical-device tool calls on OnePlus 6, including interaction and installed-state persistence. [Portrait app and keyboard visual review](docs/studio/oneplus6-validation.md) passed. Fault injection remains pending; image generation, the editable toolbox flow and public publishing remain future work. Home's [build options](phone/README.md#build-and-run) provide developer mode and a separate test package.

## How it fits together

One shell process per device, one octos kernel per shell, and every agent is a session in that kernel. App-agent access goes through the shell broker. The shell holds the host connection and the host token, relays every agent tool call to the app that owns the tool, and routes approvals through developer mode, the person's standing rules or confirmation sheets. The opt-in AppCard prototype accesses the shared kernel service directly, bypassing the app-peer broker, as described below. The full picture, with the code paths and what is on `main` versus planned: [docs/architecture.md](docs/architecture.md); the decisions: [ADR 0004](docs/adr/0004-native-apps-hosting-and-peers.md).
Expand Down
4 changes: 4 additions & 0 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@

初次阅读源码?从[从应用窗口到 Agent 回合](docs/architecture-walkthrough.zh-CN.md)开始,沿着入口、原生/脚本托管、数据访问、人类/系统对话、工具路由以及实际 Tokio 任务阅读源码。[产品导读](desktop/docs/code-walkthrough.zh-CN.md)补充桌面、Home、ROM 和系统应用的运行方式。

[手机上的 App Studio](docs/adr/0006-app-studio-on-the-phone.md) 现已提供开发者工具,可检查、打开、观测、操作并本地安装离线 `main.splash` 应用。预览状态可丢弃;已安装应用的状态在开发者授权范围内持久保留。参见 [Studio 代码导读](docs/architecture-walkthrough.zh-CN.md#从-agent-调用跟到-app-studio-中可运行的应用)。

模型从头编写的 Task Planner 已在 OnePlus 6 上通过 129 次真机工具调用,覆盖交互和已安装状态持久化。[竖屏应用与键盘视觉评审](docs/studio/oneplus6-validation.md)已通过。故障注入仍待完成;图像生成、可编辑工具箱流程与公开发布仍是后续工作。Home 的[构建选项](phone/README.zh-CN.md#构建与运行)提供开发者模式和独立测试包。

## 整体如何运作

每台设备一个 Shell 进程,每个 Shell 一个 octos 内核,每个 Agent 都是这个内核中的一个会话。应用 Agent 通过 Shell broker 访问内核。Shell 持有宿主连接和宿主 token,把每个 Agent 工具调用转交给拥有该工具的应用,并通过开发者模式、用户预设的常设规则或确认面板处理审批。下文的可选 AppCard 原型绕过应用 peer broker,直接访问共享内核服务。完整说明,包括代码路径以及哪些已在 `main` 上、哪些还在规划中:[docs/architecture.zh-CN.md](docs/architecture.zh-CN.md);相关决策:[ADR 0004(英文)](docs/adr/0004-native-apps-hosting-and-peers.md)。
Expand Down
4 changes: 4 additions & 0 deletions crates/app-peers/src/broker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@

use std::collections::{BTreeSet, HashMap, VecDeque};
use std::future::Future;
use std::path::PathBuf;
use std::pin::Pin;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::{Arc, Mutex, Weak};
Expand Down Expand Up @@ -661,6 +662,7 @@ fn remember(list: &mut VecDeque<String>, item: String) {

#[derive(Clone)]
struct PeerInfo {
workspace: Option<PathBuf>,
slug: String,
session: String,
/// The host token that controls this peer (UPCR-2026-034).
Expand Down Expand Up @@ -1686,6 +1688,7 @@ impl Inner {
}
}
let peer = PeerInfo {
workspace: result["cwd"].as_str().map(PathBuf::from),
slug,
session: session.clone(),
token,
Expand Down Expand Up @@ -1897,6 +1900,7 @@ impl Inner {
return refuse("workspace_refused", &format!("the account's workspace was refused: {why}"));
}
call.calling_app = self.cfg.app_id.clone();
call.peer_workspace = peer.workspace.clone();
call.account = Some(account);
let occurrence = call.occurrence();
let repeat = {
Expand Down
7 changes: 7 additions & 0 deletions crates/app-peers/src/host_tools.rs
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,9 @@ pub struct HostToolCall {
// --- stamped by the host (the broker, or the system chat) ---
/// The app whose agent calls (`system` for the system agent).
pub calling_app: String,
/// Kernel-confirmed peer workspace, stamped by the broker, never parsed
/// from tool arguments or wire caller fields. Context folders are below it.
pub peer_workspace: Option<std::path::PathBuf>,
/// The calling peer's account (`None`: signed out, or the system agent).
pub account: Option<String>,
/// The client of the calling request context (a Rinx mini app), from
Expand Down Expand Up @@ -302,6 +305,7 @@ impl HostToolCall {
turn_id: s("turn_id").or_else(|| caller.get("turn_id").and_then(Value::as_str).map(str::to_owned)).unwrap_or_default(),
caller_kind,
calling_app: String::new(),
peer_workspace: None,
account: None,
client: None,
origin: if caller_kind == CallerKind::System { CallOrigin::System } else { CallOrigin::PeerOwn },
Expand Down Expand Up @@ -1104,6 +1108,9 @@ mod tests {
"args": {"room": "!r", "text": "hi"}, "risk": "act", "confirm_required": true, "timeout_ms": 5000, "tools_version": 3
});
let call = HostToolCall::parse(&params).unwrap();
let mut forged = params.clone();
forged["peer_workspace"] = json!("/other/app");
assert!(HostToolCall::parse(&forged).unwrap().peer_workspace.is_none(), "only the broker may stamp the workspace");
assert_eq!(call.caller_kind, CallerKind::AppPeer);
assert!(call.confirm_required);
assert_eq!(call.context_id.as_deref(), Some("ab-mini"));
Expand Down
5 changes: 4 additions & 1 deletion crates/app-peers/tests/broker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -775,12 +775,15 @@ fn a_tool_call_is_stamped_run_once_answered_on_its_link_and_never_after_a_cancel
let slug = peer_slug(&script);
let context_id = calls_of(&script, "peer/context/open")[0].1["context_id"].as_str().unwrap().to_owned();

notify(&script, "peer/tool/call", tool_call_params(&slug, "c1", "t1", Some(&context_id)));
let mut incoming = tool_call_params(&slug, "c1", "t1", Some(&context_id));
incoming["peer_workspace"] = json!("/forged/workspace");
notify(&script, "peer/tool/call", incoming);
wait_for("the host to get the call", || host.calls.lock().unwrap().len() == 1);
let (call, reply) = host.calls.lock().unwrap()[0].clone();
assert_eq!(call.account.as_deref(), Some("@alice:x"), "the account is the host's, never the app's");
assert_eq!(call.client.as_deref(), Some("mini.news#1"), "the client comes from the context table");
assert_eq!(call.calling_app, "rinx");
assert_eq!(call.peer_workspace.as_deref(), Some(std::path::Path::new("/kernel/ws")), "workspace comes from peer/prepare, never the tool call");
assert_eq!(call.origin, CallOrigin::Context);
assert!(reply.acknowledge());
assert!(reply.finish(ToolOutcome::Ok(json!({"sent": true}))));
Expand Down
49 changes: 46 additions & 3 deletions crates/shell/src/apps.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ impl Default for AppRegistry {

/// A linked module by id, without a registry: what the launcher asks.
pub fn is_linked(id: &str) -> bool {
linked_modules().iter().any(|m| m.id() == id) || is_card_app(id)
linked_modules().iter().any(|m| m.id() == id) || is_card_app(id) || is_studio_app(id)
}

/// The native apps this build links (`native-apps.json`, generated into
Expand All @@ -60,6 +60,8 @@ fn linked_modules() -> Vec<&'static dyn AppModule> {
let mut out: Vec<&'static dyn AppModule> = Vec::new();
crate::native_apps::link(&mut out);
out.extend(crate::ext::linked_modules());
#[cfg(all(unix, any(feature = "app-hub", native_mobile)))]
out.push(&crate::studio::module::STUDIO_MODULE);
out
}

Expand Down Expand Up @@ -396,6 +398,23 @@ fn cached_installed_apps(key: (std::path::PathBuf, u64), load: impl FnOnce() ->
})
}

/// Local developer installs are a separate registry, never catalog packages.
pub fn studio_apps() -> Vec<crate::clients::AppDef> {
#[cfg(all(unix, any(feature = "app-hub", native_mobile)))]
{ return crate::host_tools::studio_bundles::installed_apps().into_iter()
.map(|bundle| studio_row(bundle.id.clone(), bundle.name.clone(), Vec::new())).collect(); }
#[cfg(not(all(unix, any(feature = "app-hub", native_mobile))))]
{ Vec::new() }
}
fn is_studio_app(id: &str) -> bool {
id.starts_with("dev.studio.") && studio_apps().iter().any(|app| app.id == id)
}
#[cfg(all(unix, any(feature = "app-hub", native_mobile)))]
pub fn studio_row(id: String, label: String, args: Vec<String>) -> crate::clients::AppDef {
crate::clients::AppDef { id, label, bin: "studio-app".into(), package: String::new(),
dir: String::new(), manifest: None, args, policy: crate::clients::LaunchPolicy::OrFocus, target_dir: None }
}

/// Every launcher row the Card runner opens: system apps, then installed ones.
pub fn card_apps() -> Vec<crate::clients::AppDef> {
let mut apps = system_card_apps();
Expand All @@ -416,7 +435,7 @@ fn is_card_app(id: &str) -> bool {
/// Launch-or-focus: a Card app focuses only an instance of that same app,
/// never a built-in (or another installed app) whose name it shares.
pub fn matches_running_app(app: &crate::clients::AppDef, running_id: &str, title: &str) -> bool {
if app.bin == "card" || running_id.starts_with("hub:") {
if matches!(app.bin.as_str(), "card" | "studio-app") || running_id.starts_with("hub:") || running_id.starts_with("dev.studio.") {
running_id == app.id
} else {
crate::clients::word_match(running_id, &app.id) || crate::clients::word_match(title, &app.id)
Expand All @@ -434,6 +453,10 @@ pub fn module_open(module: &'static dyn AppModule, app: &crate::clients::AppDef)
let manifest_id = card_manifest_id(app).ok_or_else(|| format!("{} names no app for the card runner", app.id))?;
return schema.validate(&format!("{{\"app\":{}}}", makepad_strict_json::Value::Str(manifest_id.into()).to_json()), &[]);
}
if module.id() == "studio-app" {
let instance = app.args.iter().find_map(|a| a.strip_prefix("--studio-instance=")).ok_or("Studio launch requires an admitted instance")?;
return schema.validate(&format!("{{\"instance_id\":{}}}", makepad_strict_json::Value::Str(instance.into()).to_json()), &[]);
}
let config = std::env::var("MAKEPAD_APP_CONFIG").ok().and_then(|text| makepad_strict_json::parse(text.as_bytes()).ok());
if let Some(json) = config.as_ref().and_then(|c| c.get("module_open")).and_then(|v| v.get(module.id())).map(|v| v.to_json()) {
return schema.validate(&json, &[]);
Expand All @@ -452,14 +475,15 @@ pub fn module_open(module: &'static dyn AppModule, app: &crate::clients::AppDef)
pub fn bundled_catalog() -> Vec<crate::clients::AppDef> {
let mut catalog = bundled_modules_catalog();
catalog.extend(card_apps());
catalog.extend(studio_apps());
catalog.retain(|app| catalog_visible(&app.id));
catalog
}

/// Whether an id may be a launcher row: the `card` host is internal (the
/// apps it runs are the rows), and the retired empty `appstore` stays out.
pub fn catalog_visible(id: &str) -> bool {
!matches!(id, "card" | "appstore")
!matches!(id, "card" | "appstore" | "studio-app")
}

/// Registry rows a shell surface launches but no list shows as an app:
Expand Down Expand Up @@ -587,6 +611,9 @@ impl AppRegistry {
if let Some(module) = self.modules.iter().copied().find(|m| m.id() == id) {
return Some(module);
}
if is_studio_app(id) {
return self.modules.iter().copied().find(|m| m.id() == "studio-app");
}
if is_card_app(id) {
return self.modules.iter().copied().find(|m| m.id() == "card");
}
Expand All @@ -603,6 +630,7 @@ impl AppRegistry {
if !crate::host::processes_available() {
return if self.module(id).is_some() { Hosting::Module } else { Hosting::Process };
}
if is_studio_app(id) && self.module(id).is_some() { return Hosting::Module; }
// Neither the store nor Settings has a process form.
if matches!(id, "apphub" | "settings") && self.module(id).is_some() {
return Hosting::Module;
Expand Down Expand Up @@ -817,6 +845,21 @@ mod tests {
assert!(!matches_running_app(&system, "mailer", "Mail"), "a Card app focuses only itself");
}

#[cfg(all(unix, any(feature = "app-hub", native_mobile)))]
#[test]
fn developer_apps_require_admitted_instances_and_exact_launch_identity() {
let module: &'static dyn AppModule = &crate::studio::module::STUDIO_MODULE;
let mut app = studio_row("dev.studio.planner".into(), "Planner".into(), Vec::new());
assert!(!catalog_visible(module.id()));
assert!(module_open(module, &app).is_err());
app.args.push("--studio-instance=host-issued-id".into());
assert_eq!(module_open(module, &app).unwrap().text("instance_id"), Some("host-issued-id"));
assert!(matches_running_app(&app, "dev.studio.planner", "Any title"));
assert!(!matches_running_app(&app, "planner", "Planner"));
assert!(!matches_running_app(&app, "dev.studio.planner-copy", "Planner"));
assert!(!matches_running_app(&app, "studio-instance:preview", "Planner"));
}

/// The system apps this build packs (system-apps.json): each a Card app
/// under its short id unless a native module of that id is linked, and
/// each bundle registered with the runner.
Expand Down
4 changes: 3 additions & 1 deletion crates/shell/src/clients.rs
Original file line number Diff line number Diff line change
Expand Up @@ -134,11 +134,13 @@ fn manifest_value(manifest: &str, key: &str) -> Option<String> {
/// apps and installed apps, read fresh so an install needs no restart).
pub fn registry() -> Vec<AppDef> {
let base = crate::octosense::catalog::loaded().as_ref().cloned().unwrap_or_default();
let mut installed = crate::apps::installed_card_apps();
installed.extend(crate::apps::studio_apps());
merge_catalog(
base,
crate::apps::bundled_modules_catalog(),
crate::apps::system_card_apps(),
crate::apps::installed_card_apps(),
installed,
)
}

Expand Down
28 changes: 28 additions & 0 deletions crates/shell/src/glance.rs
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,34 @@ pub fn check_level(source: &str) -> Result<(), String> {
Ok(())
}

/// Prepare an unpublished studio card without touching the glance store.
/// Chat snapshots currently create account folders; refuse them until the
/// studio has a strictly read-only chat snapshot API.
pub(crate) fn prepare_studio(app: &str, source: &str, data: &Value, dark: bool) -> Result<String, String> {
if source.len() > SOURCE_MAX || serde_json::to_vec(data).map_err(|e| e.to_string())?.len() > DATA_MAX {
return Err("card source or data exceeds the glance budget".into());
}
if !data.is_object() { return Err("data must be an object".into()); }
check_level(source)?;
if octoscript_ui_l0::check_ui_l0(source).level != octoscript_ui_l0::Level::L0 {
return Err("studio milestone 1 accepts L0 only".into());
}
crate::glance_chat::check_publisher(source, app)?;
if crate::glance_chat::reads_chat(source) {
return Err("studio_chat_unsupported: read-only chat snapshots are not available yet".into());
}
let mut data = data.clone();
resolve_digests(&Caller::Native(app.into()), source, &mut data,
crate::glance_digest::digest_root().as_deref(), octosense_l0_chat::now_ms())?;
let body = crate::glance_card::lower_in_mode(source, &data, dark)?;
// Images may finish loading after a visually quiet frame. Until their
// pending-resource state is observable, never call such a render settled.
if body.contains("http_resource(") || body.contains("Image {") || body.contains("Image{") || body.contains("crate_resource(\"self:") {
return Err("studio_images_unsupported: milestone 1 renders resource-free L0 cards".into());
}
Ok(body)
}

/// The card's `sys.digest` sources, resolved for `caller` into `data`: the
/// value under the source's name and its lifecycle under `$status`, both
/// replacing anything the publisher sent. Returns the bound run ids and the
Expand Down
Loading
Loading