diff --git a/AGENTS.md b/AGENTS.md index c8548263..a7108b4a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,6 +20,7 @@ | Which apps have an agent, their tools, and the relay that checks and routes every agent tool call | `crates/shell/src/apps.rs` (`agent_apps`), `crates/shell/src/host_tools/` (`script_apps.rs`, `relay.rs`, `files.rs`) | from `phone/`: `cargo test --locked --features mobile-apps -p octosense-shell host_tools`; `phone.yml` | | Approvals, first-use consent, standing rules, the audit | `crates/shell/src/approvals/` | from `phone/`: `cargo test --locked --features mobile-apps -p octosense-shell approvals`; `phone.yml` | | Glance cards (`glance.publish`), the desktop's glance panel and card window, a card's in-card chat | `crates/shell/src/glance*.rs`; the shared `sys.chat` host side in `crates/l0-chat/` | `cargo test --locked -p octosense-l0-chat`; from `phone/`: `cargo test --locked --features mobile-apps -p octosense-shell glance`; `apps.yml`, `phone.yml` | +| App Studio: L0 preview, scoped app instrumentation and local developer installs | `crates/shell/src/studio/` (UI/GPU/instrument/module), `crates/shell/src/host_tools/studio.rs` (tools/workspaces), `studio_bundles.rs` beside it (admission/DevTag receipts), `docs/adr/0006-app-studio-on-the-phone.md` | from `phone/`: `cargo test --locked --features mobile-apps,dev-mode -p octosense-shell`; separate-package device checks must cover a freshly authored app, real widget input, measured UX, installed persistence and revocation; unit/GPU fixture tests alone are insufficient | | The system toolbox's workflow templates (library, runner, forks, evaluation, `mod.research`) | `crates/toolbox/` (templates in `crates/toolbox/templates//`); offered to app agents by `crates/ai-host/src/toolbox_peers.rs` (feature `toolbox-peers`) | `cargo test --locked -p octosense-toolbox`, its README, `apps.yml` | | A system app | `apps//bundle/` | App Hub's `card-host --bundle apps//bundle --system`; then in a shell | | A system app's agent: its tools and the cards they publish | `apps//bundle/tools.json` and the manifest's `agent` block; the tools run on the app's host service | its host service's tests (below); then in a shell, with the system agent asking the app's agent | @@ -31,6 +32,12 @@ Start every session with `python3 tools/setup.py` (it prepares `.sources/`, and changes nothing that is already right). +For local shell tests that link Rinx, set `OCTOSENSE_HOME` and `RINX_DATA_DIR` +to separate, empty test directories and `MAKEPAD_HIDE_WINDOWS=1`. Some hosted +module tests initialize Rinx; without the explicit data directory they can +restore the machine's existing account. Keep these directories outside the +checkout and never copy personal account state into them. + On a machine that already has clones of Makepad, OctoScript or Octoscript-Makepad, never let setup clone them again: name the clones as a hub (`~/.config/octosense/sources.json`, `--hub DIR` or `OCTOSENSE_SOURCES_HUB`; see the README's Set up) so every `.sources/` entry is a `git worktree` of the one clone, and run `python3 tools/setup.py --remove-worktrees` before deleting a checkout of this repository. ## Local CI diff --git a/README.md b/README.md index 13a8a561..7c2a1947 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,10 @@ It was OctoSense-Desktop; OctoSense-ROM (retired; merged into this repository) a New to the codebase? Start with [From an app window to an agent turn](docs/architecture-walkthrough.md). It follows executable entry points, native and script hosting, app data access, human/system conversations, tool routing and the actual Tokio tasks. The [product walkthrough](desktop/docs/code-walkthrough.md) adds desktop, Home, ROM and system-app run recipes. +[App Studio on the phone](docs/adr/0006-app-studio-on-the-phone.md) now has developer-only tools to check, open, inspect, exercise and locally install an offline `main.splash` app. Preview state is disposable; installed app state persists under its developer grant. Follow the [Studio code walkthrough](docs/architecture-walkthrough.md#follow-app-studio-from-the-agent-to-a-working-app). + +A fresh model-authored Task Planner passed 129 physical-device tool calls on OnePlus 6, including interaction and installed-state persistence. [Portrait app and keyboard visual review](docs/studio/oneplus6-validation.md) passed. Fault injection remains pending; image generation, the editable toolbox flow and public publishing remain future work. Home's [build options](phone/README.md#build-and-run) provide developer mode and a separate test package. + ## How it fits together One shell process per device, one octos kernel per shell, and every agent is a session in that kernel. App-agent access goes through the shell broker. The shell holds the host connection and the host token, relays every agent tool call to the app that owns the tool, and routes approvals through developer mode, the person's standing rules or confirmation sheets. The opt-in AppCard prototype accesses the shared kernel service directly, bypassing the app-peer broker, as described below. The full picture, with the code paths and what is on `main` versus planned: [docs/architecture.md](docs/architecture.md); the decisions: [ADR 0004](docs/adr/0004-native-apps-hosting-and-peers.md). diff --git a/README.zh-CN.md b/README.zh-CN.md index 558498bf..82f4bbea 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -18,6 +18,10 @@ 初次阅读源码？从[从应用窗口到 Agent 回合](docs/architecture-walkthrough.zh-CN.md)开始，沿着入口、原生/脚本托管、数据访问、人类/系统对话、工具路由以及实际 Tokio 任务阅读源码。[产品导读](desktop/docs/code-walkthrough.zh-CN.md)补充桌面、Home、ROM 和系统应用的运行方式。 +[手机上的 App Studio](docs/adr/0006-app-studio-on-the-phone.md) 现已提供开发者工具，可检查、打开、观测、操作并本地安装离线 `main.splash` 应用。预览状态可丢弃；已安装应用的状态在开发者授权范围内持久保留。参见 [Studio 代码导读](docs/architecture-walkthrough.zh-CN.md#从-agent-调用跟到-app-studio-中可运行的应用)。 + +模型从头编写的 Task Planner 已在 OnePlus 6 上通过 129 次真机工具调用，覆盖交互和已安装状态持久化。[竖屏应用与键盘视觉评审](docs/studio/oneplus6-validation.md)已通过。故障注入仍待完成；图像生成、可编辑工具箱流程与公开发布仍是后续工作。Home 的[构建选项](phone/README.zh-CN.md#构建与运行)提供开发者模式和独立测试包。 + ## 整体如何运作 每台设备一个 Shell 进程，每个 Shell 一个 octos 内核，每个 Agent 都是这个内核中的一个会话。应用 Agent 通过 Shell broker 访问内核。Shell 持有宿主连接和宿主 token，把每个 Agent 工具调用转交给拥有该工具的应用，并通过开发者模式、用户预设的常设规则或确认面板处理审批。下文的可选 AppCard 原型绕过应用 peer broker，直接访问共享内核服务。完整说明，包括代码路径以及哪些已在 `main` 上、哪些还在规划中：[docs/architecture.zh-CN.md](docs/architecture.zh-CN.md)；相关决策：[ADR 0004（英文）](docs/adr/0004-native-apps-hosting-and-peers.md)。 diff --git a/crates/app-peers/src/broker.rs b/crates/app-peers/src/broker.rs index 2d925945..15ffb5da 100644 --- a/crates/app-peers/src/broker.rs +++ b/crates/app-peers/src/broker.rs @@ -82,6 +82,7 @@ use std::collections::{BTreeSet, HashMap, VecDeque}; use std::future::Future; +use std::path::PathBuf; use std::pin::Pin; use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; use std::sync::{Arc, Mutex, Weak}; @@ -661,6 +662,7 @@ fn remember(list: &mut VecDeque, item: String) { #[derive(Clone)] struct PeerInfo { + workspace: Option, slug: String, session: String, /// The host token that controls this peer (UPCR-2026-034). @@ -1686,6 +1688,7 @@ impl Inner { } } let peer = PeerInfo { + workspace: result["cwd"].as_str().map(PathBuf::from), slug, session: session.clone(), token, @@ -1897,6 +1900,7 @@ impl Inner { return refuse("workspace_refused", &format!("the account's workspace was refused: {why}")); } call.calling_app = self.cfg.app_id.clone(); + call.peer_workspace = peer.workspace.clone(); call.account = Some(account); let occurrence = call.occurrence(); let repeat = { diff --git a/crates/app-peers/src/host_tools.rs b/crates/app-peers/src/host_tools.rs index afeb9d43..89c8518f 100644 --- a/crates/app-peers/src/host_tools.rs +++ b/crates/app-peers/src/host_tools.rs @@ -261,6 +261,9 @@ pub struct HostToolCall { // --- stamped by the host (the broker, or the system chat) --- /// The app whose agent calls (`system` for the system agent). pub calling_app: String, + /// Kernel-confirmed peer workspace, stamped by the broker, never parsed + /// from tool arguments or wire caller fields. Context folders are below it. + pub peer_workspace: Option, /// The calling peer's account (`None`: signed out, or the system agent). pub account: Option, /// The client of the calling request context (a Rinx mini app), from @@ -302,6 +305,7 @@ impl HostToolCall { turn_id: s("turn_id").or_else(|| caller.get("turn_id").and_then(Value::as_str).map(str::to_owned)).unwrap_or_default(), caller_kind, calling_app: String::new(), + peer_workspace: None, account: None, client: None, origin: if caller_kind == CallerKind::System { CallOrigin::System } else { CallOrigin::PeerOwn }, @@ -1104,6 +1108,9 @@ mod tests { "args": {"room": "!r", "text": "hi"}, "risk": "act", "confirm_required": true, "timeout_ms": 5000, "tools_version": 3 }); let call = HostToolCall::parse(¶ms).unwrap(); + let mut forged = params.clone(); + forged["peer_workspace"] = json!("/other/app"); + assert!(HostToolCall::parse(&forged).unwrap().peer_workspace.is_none(), "only the broker may stamp the workspace"); assert_eq!(call.caller_kind, CallerKind::AppPeer); assert!(call.confirm_required); assert_eq!(call.context_id.as_deref(), Some("ab-mini")); diff --git a/crates/app-peers/tests/broker.rs b/crates/app-peers/tests/broker.rs index be8cdc98..3d4fcb62 100644 --- a/crates/app-peers/tests/broker.rs +++ b/crates/app-peers/tests/broker.rs @@ -775,12 +775,15 @@ fn a_tool_call_is_stamped_run_once_answered_on_its_link_and_never_after_a_cancel let slug = peer_slug(&script); let context_id = calls_of(&script, "peer/context/open")[0].1["context_id"].as_str().unwrap().to_owned(); - notify(&script, "peer/tool/call", tool_call_params(&slug, "c1", "t1", Some(&context_id))); + let mut incoming = tool_call_params(&slug, "c1", "t1", Some(&context_id)); + incoming["peer_workspace"] = json!("/forged/workspace"); + notify(&script, "peer/tool/call", incoming); wait_for("the host to get the call", || host.calls.lock().unwrap().len() == 1); let (call, reply) = host.calls.lock().unwrap()[0].clone(); assert_eq!(call.account.as_deref(), Some("@alice:x"), "the account is the host's, never the app's"); assert_eq!(call.client.as_deref(), Some("mini.news#1"), "the client comes from the context table"); assert_eq!(call.calling_app, "rinx"); + assert_eq!(call.peer_workspace.as_deref(), Some(std::path::Path::new("/kernel/ws")), "workspace comes from peer/prepare, never the tool call"); assert_eq!(call.origin, CallOrigin::Context); assert!(reply.acknowledge()); assert!(reply.finish(ToolOutcome::Ok(json!({"sent": true})))); diff --git a/crates/shell/src/apps.rs b/crates/shell/src/apps.rs index 21957c97..aef9966c 100644 --- a/crates/shell/src/apps.rs +++ b/crates/shell/src/apps.rs @@ -45,7 +45,7 @@ impl Default for AppRegistry { /// A linked module by id, without a registry: what the launcher asks. pub fn is_linked(id: &str) -> bool { - linked_modules().iter().any(|m| m.id() == id) || is_card_app(id) + linked_modules().iter().any(|m| m.id() == id) || is_card_app(id) || is_studio_app(id) } /// The native apps this build links (`native-apps.json`, generated into @@ -60,6 +60,8 @@ fn linked_modules() -> Vec<&'static dyn AppModule> { let mut out: Vec<&'static dyn AppModule> = Vec::new(); crate::native_apps::link(&mut out); out.extend(crate::ext::linked_modules()); + #[cfg(all(unix, any(feature = "app-hub", native_mobile)))] + out.push(&crate::studio::module::STUDIO_MODULE); out } @@ -396,6 +398,23 @@ fn cached_installed_apps(key: (std::path::PathBuf, u64), load: impl FnOnce() -> }) } +/// Local developer installs are a separate registry, never catalog packages. +pub fn studio_apps() -> Vec { + #[cfg(all(unix, any(feature = "app-hub", native_mobile)))] + { return crate::host_tools::studio_bundles::installed_apps().into_iter() + .map(|bundle| studio_row(bundle.id.clone(), bundle.name.clone(), Vec::new())).collect(); } + #[cfg(not(all(unix, any(feature = "app-hub", native_mobile))))] + { Vec::new() } +} +fn is_studio_app(id: &str) -> bool { + id.starts_with("dev.studio.") && studio_apps().iter().any(|app| app.id == id) +} +#[cfg(all(unix, any(feature = "app-hub", native_mobile)))] +pub fn studio_row(id: String, label: String, args: Vec) -> crate::clients::AppDef { + crate::clients::AppDef { id, label, bin: "studio-app".into(), package: String::new(), + dir: String::new(), manifest: None, args, policy: crate::clients::LaunchPolicy::OrFocus, target_dir: None } +} + /// Every launcher row the Card runner opens: system apps, then installed ones. pub fn card_apps() -> Vec { let mut apps = system_card_apps(); @@ -416,7 +435,7 @@ fn is_card_app(id: &str) -> bool { /// Launch-or-focus: a Card app focuses only an instance of that same app, /// never a built-in (or another installed app) whose name it shares. pub fn matches_running_app(app: &crate::clients::AppDef, running_id: &str, title: &str) -> bool { - if app.bin == "card" || running_id.starts_with("hub:") { + if matches!(app.bin.as_str(), "card" | "studio-app") || running_id.starts_with("hub:") || running_id.starts_with("dev.studio.") { running_id == app.id } else { crate::clients::word_match(running_id, &app.id) || crate::clients::word_match(title, &app.id) @@ -434,6 +453,10 @@ pub fn module_open(module: &'static dyn AppModule, app: &crate::clients::AppDef) let manifest_id = card_manifest_id(app).ok_or_else(|| format!("{} names no app for the card runner", app.id))?; return schema.validate(&format!("{{\"app\":{}}}", makepad_strict_json::Value::Str(manifest_id.into()).to_json()), &[]); } + if module.id() == "studio-app" { + let instance = app.args.iter().find_map(|a| a.strip_prefix("--studio-instance=")).ok_or("Studio launch requires an admitted instance")?; + return schema.validate(&format!("{{\"instance_id\":{}}}", makepad_strict_json::Value::Str(instance.into()).to_json()), &[]); + } let config = std::env::var("MAKEPAD_APP_CONFIG").ok().and_then(|text| makepad_strict_json::parse(text.as_bytes()).ok()); if let Some(json) = config.as_ref().and_then(|c| c.get("module_open")).and_then(|v| v.get(module.id())).map(|v| v.to_json()) { return schema.validate(&json, &[]); @@ -452,6 +475,7 @@ pub fn module_open(module: &'static dyn AppModule, app: &crate::clients::AppDef) pub fn bundled_catalog() -> Vec { let mut catalog = bundled_modules_catalog(); catalog.extend(card_apps()); + catalog.extend(studio_apps()); catalog.retain(|app| catalog_visible(&app.id)); catalog } @@ -459,7 +483,7 @@ pub fn bundled_catalog() -> Vec { /// Whether an id may be a launcher row: the `card` host is internal (the /// apps it runs are the rows), and the retired empty `appstore` stays out. pub fn catalog_visible(id: &str) -> bool { - !matches!(id, "card" | "appstore") + !matches!(id, "card" | "appstore" | "studio-app") } /// Registry rows a shell surface launches but no list shows as an app: @@ -587,6 +611,9 @@ impl AppRegistry { if let Some(module) = self.modules.iter().copied().find(|m| m.id() == id) { return Some(module); } + if is_studio_app(id) { + return self.modules.iter().copied().find(|m| m.id() == "studio-app"); + } if is_card_app(id) { return self.modules.iter().copied().find(|m| m.id() == "card"); } @@ -603,6 +630,7 @@ impl AppRegistry { if !crate::host::processes_available() { return if self.module(id).is_some() { Hosting::Module } else { Hosting::Process }; } + if is_studio_app(id) && self.module(id).is_some() { return Hosting::Module; } // Neither the store nor Settings has a process form. if matches!(id, "apphub" | "settings") && self.module(id).is_some() { return Hosting::Module; @@ -817,6 +845,21 @@ mod tests { assert!(!matches_running_app(&system, "mailer", "Mail"), "a Card app focuses only itself"); } + #[cfg(all(unix, any(feature = "app-hub", native_mobile)))] + #[test] + fn developer_apps_require_admitted_instances_and_exact_launch_identity() { + let module: &'static dyn AppModule = &crate::studio::module::STUDIO_MODULE; + let mut app = studio_row("dev.studio.planner".into(), "Planner".into(), Vec::new()); + assert!(!catalog_visible(module.id())); + assert!(module_open(module, &app).is_err()); + app.args.push("--studio-instance=host-issued-id".into()); + assert_eq!(module_open(module, &app).unwrap().text("instance_id"), Some("host-issued-id")); + assert!(matches_running_app(&app, "dev.studio.planner", "Any title")); + assert!(!matches_running_app(&app, "planner", "Planner")); + assert!(!matches_running_app(&app, "dev.studio.planner-copy", "Planner")); + assert!(!matches_running_app(&app, "studio-instance:preview", "Planner")); + } + /// The system apps this build packs (system-apps.json): each a Card app /// under its short id unless a native module of that id is linked, and /// each bundle registered with the runner. diff --git a/crates/shell/src/clients.rs b/crates/shell/src/clients.rs index f0b6139a..bddb2687 100644 --- a/crates/shell/src/clients.rs +++ b/crates/shell/src/clients.rs @@ -134,11 +134,13 @@ fn manifest_value(manifest: &str, key: &str) -> Option { /// apps and installed apps, read fresh so an install needs no restart). pub fn registry() -> Vec { let base = crate::octosense::catalog::loaded().as_ref().cloned().unwrap_or_default(); + let mut installed = crate::apps::installed_card_apps(); + installed.extend(crate::apps::studio_apps()); merge_catalog( base, crate::apps::bundled_modules_catalog(), crate::apps::system_card_apps(), - crate::apps::installed_card_apps(), + installed, ) } diff --git a/crates/shell/src/glance.rs b/crates/shell/src/glance.rs index 82230006..689ba53e 100644 --- a/crates/shell/src/glance.rs +++ b/crates/shell/src/glance.rs @@ -206,6 +206,34 @@ pub fn check_level(source: &str) -> Result<(), String> { Ok(()) } +/// Prepare an unpublished studio card without touching the glance store. +/// Chat snapshots currently create account folders; refuse them until the +/// studio has a strictly read-only chat snapshot API. +pub(crate) fn prepare_studio(app: &str, source: &str, data: &Value, dark: bool) -> Result { + if source.len() > SOURCE_MAX || serde_json::to_vec(data).map_err(|e| e.to_string())?.len() > DATA_MAX { + return Err("card source or data exceeds the glance budget".into()); + } + if !data.is_object() { return Err("data must be an object".into()); } + check_level(source)?; + if octoscript_ui_l0::check_ui_l0(source).level != octoscript_ui_l0::Level::L0 { + return Err("studio milestone 1 accepts L0 only".into()); + } + crate::glance_chat::check_publisher(source, app)?; + if crate::glance_chat::reads_chat(source) { + return Err("studio_chat_unsupported: read-only chat snapshots are not available yet".into()); + } + let mut data = data.clone(); + resolve_digests(&Caller::Native(app.into()), source, &mut data, + crate::glance_digest::digest_root().as_deref(), octosense_l0_chat::now_ms())?; + let body = crate::glance_card::lower_in_mode(source, &data, dark)?; + // Images may finish loading after a visually quiet frame. Until their + // pending-resource state is observable, never call such a render settled. + if body.contains("http_resource(") || body.contains("Image {") || body.contains("Image{") || body.contains("crate_resource(\"self:") { + return Err("studio_images_unsupported: milestone 1 renders resource-free L0 cards".into()); + } + Ok(body) +} + /// The card's `sys.digest` sources, resolved for `caller` into `data`: the /// value under the source's name and its lifecycle under `$status`, both /// replacing anything the publisher sent. Returns the bound run ids and the diff --git a/crates/shell/src/glance_card.rs b/crates/shell/src/glance_card.rs index 82947f20..7293d64c 100644 --- a/crates/shell/src/glance_card.rs +++ b/crates/shell/src/glance_card.rs @@ -124,21 +124,26 @@ const MOODS: &[(&str, &str)] = &[ /// general translation re-lowered chips the Material way and dropped their /// text, and drew the shipping card's tiles light under white text). pub fn lower(source: &str, data: &serde_json::Value) -> Result { - lower_report(source, octoscript_ui_l0::realize(source, data, Default::default()), true) + lower_in_mode(source, data, dark()) +} + +/// Studio selects a palette without changing the live shell's mode. +pub fn lower_in_mode(source: &str, data: &serde_json::Value, dark: bool) -> Result { + lower_report(source, octoscript_ui_l0::realize(source, data, Default::default()), true, dark) } /// [`lower`], realized against a card's local state (its `InstanceStore`), /// for the card window. pub fn lower_with_state(source: &str, data: &serde_json::Value, store: &octoscript_ui_l0::InstanceStore) -> Result { - lower_report(source, octoscript_ui_l0::realize_with_state(source, data, store, Default::default()), true) + lower_report(source, octoscript_ui_l0::realize_with_state(source, data, store, Default::default()), true, dark()) } -fn lower_report(source: &str, report: octoscript_ui_l0::RealizeReport, l0_ui: bool) -> Result { +fn lower_report(source: &str, report: octoscript_ui_l0::RealizeReport, l0_ui: bool, dark: bool) -> Result { let root = report.complete_root()?; if octoscript_ui_l0::kit_pack::contains(root) { return Err("native kit components are not offered on a glance tile".into()); } - let mood = octoscript_ui_l0::card_theme(source).unwrap_or_else(|| if dark() { "dark" } else { "light" }.into()); + let mood = octoscript_ui_l0::card_theme(source).unwrap_or_else(|| if dark { "dark" } else { "light" }.into()); let delta = MOODS.iter().find(|(name, _)| *name == mood).map(|(_, d)| *d).ok_or_else(|| format!("theme {mood:?} is not offered on a glance tile"))?; for (axis, value) in octoscript_ui_l0::card_theme_axes(source) { if !matches!(value.as_str(), "neutral" | "regular" | "none" | "soft" | "sans") { @@ -842,7 +847,7 @@ pub const CAN_RENDER: bool = cfg!(any(feature = "app-hub", native_mobile)); /// Give every Splash isolate the Card runner's vocabulary, once: the same /// registration the `card` module makes before it runs an app. -fn ensure_vocabulary(cx: &mut Cx) { +pub(crate) fn ensure_vocabulary(cx: &mut Cx) { thread_local! { static DONE: std::cell::Cell = const { std::cell::Cell::new(false) }; } diff --git a/crates/shell/src/host_tools/mod.rs b/crates/shell/src/host_tools/mod.rs index c982358b..223b69e7 100644 --- a/crates/shell/src/host_tools/mod.rs +++ b/crates/shell/src/host_tools/mod.rs @@ -37,6 +37,9 @@ pub mod dev_run; pub mod files; pub mod relay; pub mod schema; +pub mod studio; +#[cfg(all(unix, any(feature = "app-hub", native_mobile)))] +pub mod studio_bundles; #[cfg(feature = "toolbox-peers")] pub mod toolbox; #[cfg(any(feature = "app-hub", native_mobile))] @@ -360,6 +363,7 @@ impl ToolHost for ShellToolHost { /// with it and are withdrawn the moment it ends (the relay also refuses a /// late call). The peers asked. pub fn developer_mode_changed() -> usize { + crate::system_chat::sync_host_tools(); #[cfg(kernel)] { crate::ai_host::app_peers::broker::reregister_tools_where(|_| true) @@ -376,16 +380,20 @@ pub fn developer_mode_changed() -> usize { pub struct HostExecutor { dev_run: dev_run::DevRunExecutor, workspace: fn(&str, &str) -> Option, + studio: studio::StudioExecutor, } impl HostExecutor { pub fn new(workspace: fn(&str, &str) -> Option) -> HostExecutor { - HostExecutor { dev_run: dev_run::DevRunExecutor::new(workspace), workspace } + HostExecutor { dev_run: dev_run::DevRunExecutor::new(workspace), workspace, studio: studio::StudioExecutor::default() } } } impl ToolExecutor for HostExecutor { fn execute(&self, call: HostToolCall, reply: ToolReply) { + if studio::is_tool(&call.name) { + return self.studio.execute(call, reply); + } if call.name == relay::DEV_RUN { return self.dev_run.execute(call, reply); } @@ -405,6 +413,7 @@ impl ToolExecutor for HostExecutor { fn cancel(&self, call_id: &str) { self.dev_run.cancel(call_id); + self.studio.cancel(call_id); } } diff --git a/crates/shell/src/host_tools/relay.rs b/crates/shell/src/host_tools/relay.rs index 24371ff5..55ea4df2 100644 --- a/crates/shell/src/host_tools/relay.rs +++ b/crates/shell/src/host_tools/relay.rs @@ -74,7 +74,7 @@ pub const HOST_EXECUTOR: &str = "@shell"; /// Whether `tool` is one the shell runs itself, as the calling app's own /// tool, whatever its declaration names (never routed to the app). pub fn is_host_run(tool: &str) -> bool { - tool == DEV_RUN || super::files::TOOLS.contains(&tool) + tool == DEV_RUN || super::studio::is_tool(tool) || super::files::TOOLS.contains(&tool) } /// A tool the shell runs, as `owner` declares it (its schemas). @@ -82,6 +82,9 @@ fn host_run_declaration(owner: &str, tool: &str) -> Option { if tool == DEV_RUN { return Some(super::dev_run::declaration(owner)); } + if super::studio::is_tool(tool) { + return super::studio::declarations(owner).into_iter().find(|d| d["name"] == tool); + } super::files::declarations(owner).into_iter().find(|d| d["name"] == tool) } /// The system toolbox (ADR 0002 §6), the owning app of the toolbox tools @@ -380,6 +383,9 @@ impl Catalog { if dev_all && app != SYSTEM { out.push(super::dev_run::declaration(app)); } + if dev_all && super::studio::SUPPORTED { + out.extend(super::studio::declarations(app)); + } out } } @@ -640,6 +646,11 @@ impl Relay { // 1. Authorize by (owning app, tool) and caller. let host_run = is_host_run(&tool); let (caller, granted) = match call.caller_kind { + _ if super::studio::is_tool(&tool) => ( + if call.caller_kind == CallerKind::System { Caller::SystemAgent } else { Caller::OwnAgent { client: call.client.clone() } }, + super::studio::SUPPORTED && calling == owner && env.grants_all(&calling) + && (call.caller_kind == CallerKind::AppPeer || calling == SYSTEM), + ), // The shell's own: only the app's own agent, on its own peer; // `dev.run` only while developer mode covers the app. _ if host_run => ( diff --git a/crates/shell/src/host_tools/studio.rs b/crates/shell/src/host_tools/studio.rs new file mode 100644 index 00000000..a3a1ef3c --- /dev/null +++ b/crates/shell/src/host_tools/studio.rs @@ -0,0 +1,878 @@ +//! Developer-only L0 glance rendering. The relay authorizes and audits the +//! call; this executor opens files relative to a trusted workspace descriptor. +//! No source/output path can select a different app, account or context. +use super::relay::{app_of_peer, SYSTEM}; +use crate::ai_host::app_peers::host_tools::{ + CallerKind, HostToolCall, ToolExecutor, ToolOutcome, ToolReply, +}; +use serde_json::{json, Value}; +use std::collections::HashMap; +use std::path::{Path, PathBuf}; +use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; +use std::sync::{Arc, Mutex}; +use std::time::{Duration, Instant}; + +pub const RENDER: &str = "studio.render"; +pub const APP_TOOLS: &[&str] = &[ + "studio.bundle_check", + "studio.open", + "studio.input", + "studio.inspect", + "studio.close", + "studio.install", +]; +pub fn is_tool(name: &str) -> bool { + name == RENDER + || cfg!(all(unix, any(feature = "app-hub", native_mobile))) && APP_TOOLS.contains(&name) +} +pub fn declarations(app: &str) -> Vec { + let mut out = vec![declaration(app)]; + if cfg!(all(unix, any(feature = "app-hub", native_mobile))) { + out.extend(APP_TOOLS.iter().map(|name| app_declaration(app, name))); + } + out +} +fn app_declaration(app: &str, name: &str) -> Value { + let (description,properties,required)=match name { + "studio.bundle_check" => ("Stage and validate an offline main.splash developer bundle from your conversation workspace. Computes and stamps the digest only on a private copy; never modifies source, signs a publisher release, or installs.",json!({"bundle_path":{"type":"string","minLength":1,"maxLength":1024}}),vec!["bundle_path"]), + "studio.install" => ("Install an offline main.splash bundle locally as a developer app. Requires dev.studio.* id. It appears in Home while its developer grant remains valid; its own state persists across close/reopen. No catalog or publisher signature is created.",json!({"bundle_path":{"type":"string","minLength":1,"maxLength":1024}}),vec!["bundle_path"]), + "studio.open" => ("Open a visible contained app. Supply exactly one of bundle_path (disposable preview state) or app_id (previous developer install, persistent app state). Returns an instance id after layout; inspect it before targeting input.",json!({"bundle_path":{"type":"string","minLength":1,"maxLength":1024},"app_id":{"type":"string","minLength":1,"maxLength":100}}),vec![]), + "studio.input" => ("Send a real tap or text event to a visible enabled widget in your Studio app instance. Use the widget id from inspect. Never targets another app or host UI.",json!({"instance_id":{"type":"string","minLength":1,"maxLength":100},"widget_id":{"type":"string","minLength":1,"maxLength":512},"action":{"type":"string","enum":["tap","text","scroll"]},"text":{"type":"string","maxLength":2048},"delta_y":{"type":"number","minimum":-2000,"maximum":2000}}),vec!["instance_id","widget_id","action"]), + "studio.inspect" => ("Inspect your Studio app. Returns a compact page of visible widget selectors; pass each exact selector to studio.input. Follow next_offset with offset for more. path is a PNG for view_image; snapshot_path is the full diagnostic JSON.",json!({"instance_id":{"type":"string","minLength":1,"maxLength":100},"offset":{"type":"integer","minimum":0}}),vec!["instance_id"]), + _ => ("Close your Studio app instance. Preview state is discarded; an installed app's state persists.",json!({"instance_id":{"type":"string","minLength":1,"maxLength":100}}),vec!["instance_id"]), + }; + json!({"name":name,"app":app,"description":description, + "input_schema":{"type":"object","properties":properties,"required":required,"additionalProperties":false}, + "output_schema":{"type":"object"},"risk":if name=="studio.install" || name=="studio.input" {"act"} else {"read"},"background":false,"shareable":false}) +} +pub const SUPPORTED: bool = cfg!(unix); +pub const SOURCE_MAX: usize = 16 * 1024; +pub const DATA_MAX: usize = 32 * 1024; +const MAX_WAIT: Duration = Duration::from_secs(25); +const MAX_IN_FLIGHT: usize = 2; +static IN_FLIGHT: AtomicUsize = AtomicUsize::new(0); +static SYSTEM_WORKSPACE: Mutex> = Mutex::new(None); + +/// Called only with the kernel's session/open response by the system driver. +pub fn system_workspace_opened(path: Option<&str>) { + *SYSTEM_WORKSPACE.lock().unwrap_or_else(|e| e.into_inner()) = path + .filter(|p| Path::new(p).is_absolute()) + .and_then(|p| std::fs::canonicalize(p).ok()); +} + +pub fn declaration(app: &str) -> Value { + json!({ + "name": RENDER, "app": app, + "description": "Developer mode only. Render an L0 glance card at this device's actual glance width while Home is foreground. Read source_path and optional data_path from your own conversation workspace. Returns a PNG path for view_image; no bundle install or app execution. Rendering may take up to 20 seconds; the host call has its normal 30-second deadline.", + "input_schema": {"type":"object", "properties": { + "source_path":{"type":"string", "minLength":1,"maxLength":1024}, + "data_path":{"type":"string", "minLength":1,"maxLength":1024}, + "dark":{"type":"boolean"} + }, "required":["source_path"],"additionalProperties":false}, + "output_schema": {"type":"object", "properties": { + "path":{"type":"string"}, "width":{"type":"integer","minimum":1}, + "height":{"type":"integer","minimum":1}, "settled":{"type":"boolean"} + }, "required":["path","width","height","settled"],"additionalProperties":false}, + "risk":"read", "background":false,"shareable":false + }) +} + +fn workspace(call: &HostToolCall) -> Result { + if call.caller_kind == CallerKind::System { + if call.calling_app != SYSTEM + || call.peer.is_some() + || call.context_id.is_some() + || call.session_id != crate::system_chat::session::SYSTEM_SESSION + { + return Err("not the shell's system conversation".into()); + } + return SYSTEM_WORKSPACE + .lock() + .unwrap_or_else(|e| e.into_inner()) + .clone() + .ok_or_else(|| "the kernel has not confirmed the system workspace".into()); + } + app_available(call)?; + // A resumed peer can retain a kernel-owned cwd rather than the account + // directory offered for new peers. Use only the broker's confirmed cwd; + // agent_workspace() is a proposal for new peers, not their actual cwd. + call.peer_workspace + .clone() + .filter(|p| p.is_absolute()) + .ok_or_else(|| "the broker has not confirmed this peer workspace".into()) +} + +fn app_available(call: &HostToolCall) -> Result<(), String> { + let account = call + .account + .as_deref() + .ok_or("the calling app has no account")?; + if super::suspended(&call.calling_app, Some(account)) { + return Err("the calling app account is suspended".into()); + } + if let Some(why) = crate::app_storage::host() + .and_then(|storage| super::workspace_refused_in(storage, &call.calling_app, account)) + { + return Err(why); + } + Ok(()) +} + +fn authorized(app: &str, tag: &crate::dev_mode::DevTag) -> bool { + crate::dev_mode::tag_valid(tag) && crate::dev_mode::grants_all(app) +} + +#[derive(Default)] +pub struct StudioExecutor { + running: Arc>>>, +} +impl ToolExecutor for StudioExecutor { + fn execute(&self, call: HostToolCall, reply: ToolReply) { + let root = match workspace(&call) { + Ok(root) => root, + Err(why) => { + reply.finish(ToolOutcome::error("no_workspace", why)); + return; + } + }; + self.execute_scoped(call, reply, root); + } + fn cancel(&self, id: &str) { + if let Some(flag) = self + .running + .lock() + .unwrap_or_else(|e| e.into_inner()) + .get(id) + { + flag.store(true, Ordering::SeqCst); + } + crate::studio::cancel(id); + #[cfg(all(unix, any(feature = "app-hub", native_mobile)))] + crate::studio::apps::cancel(id); + } +} +impl StudioExecutor { + fn execute_scoped(&self, call: HostToolCall, reply: ToolReply, root: PathBuf) { + let app = app_of_peer(&call.calling_app).to_string(); + let Some(tag) = crate::dev_mode::tag().filter(|t| authorized(&app, t)) else { + reply.finish(ToolOutcome::error( + "not_granted", + "studio.render needs developer mode for the calling agent", + )); + return; + }; + if call.app != app || !is_tool(&call.name) { + reply.finish(ToolOutcome::error( + "not_granted", + "studio.render belongs to the calling agent", + )); + return; + } + let decl = if call.name == RENDER { + declaration(&app) + } else { + app_declaration(&app, &call.name) + }; + if let Err(why) = super::schema::check(&decl["input_schema"], &call.args) { + reply.finish(ToolOutcome::error("invalid_args", why)); + return; + } + crate::dev_mode::audit_tool_call( + &app, + &call.name, + &call.args.to_string(), + &call.calling_app, + ); + if IN_FLIGHT.fetch_add(1, Ordering::SeqCst) >= MAX_IN_FLIGHT { + IN_FLIGHT.fetch_sub(1, Ordering::SeqCst); + reply.finish(ToolOutcome::error( + "busy", + "two studio requests are already in flight", + )); + return; + } + let flag = Arc::new(AtomicBool::new(false)); + { + let mut running = self.running.lock().unwrap_or_else(|e| e.into_inner()); + if running.contains_key(&call.call_id) { + IN_FLIGHT.fetch_sub(1, Ordering::SeqCst); + reply.finish(ToolOutcome::error( + "busy", + "this studio call is already running", + )); + return; + } + running.insert(call.call_id.clone(), flag.clone()); + } + let running = self.running.clone(); + std::thread::spawn(move || { + #[cfg(all(unix, any(feature = "app-hub", native_mobile)))] + let outcome = if call.name != RENDER { + super::studio_bundles::execute(&call, &root, &tag, &flag, &reply) + } else { + render_call(&call, &root, &app, &tag, &flag, &reply) + }; + #[cfg(not(all(unix, any(feature = "app-hub", native_mobile))))] + let outcome = render_call(&call, &root, &app, &tag, &flag, &reply); + reply.finish(outcome); + running + .lock() + .unwrap_or_else(|e| e.into_inner()) + .remove(&call.call_id); + IN_FLIGHT.fetch_sub(1, Ordering::SeqCst); + }); + } +} + +fn render_call( + call: &HostToolCall, + root: &Path, + app: &str, + tag: &crate::dev_mode::DevTag, + cancel: &AtomicBool, + reply: &ToolReply, +) -> ToolOutcome { + #[cfg(not(unix))] + { + let _ = (call, root, app, tag, cancel, reply); + ToolOutcome::error("unsupported", "studio rendering requires a Unix workspace") + } + #[cfg(unix)] + { + let run = || -> Result { + let scope = scoped::Workspace::open(root, call.context_id.as_deref())?; + let source = scope.read(call.args["source_path"].as_str().unwrap_or(""), SOURCE_MAX)?; + let data = match call.args["data_path"].as_str() { + Some(path) => serde_json::from_str(&scope.read(path, DATA_MAX)?) + .map_err(|e| format!("invalid data JSON: {e}"))?, + None => json!({}), + }; + let output = scope.output()?; + if !authorized(app, tag) || cancel.load(Ordering::SeqCst) || !reply.is_open() { + return Err("cancelled or developer mode revoked".into()); + } + let (tx, rx) = std::sync::mpsc::channel(); + crate::studio::submit(crate::studio::RenderJob { + id: call.call_id.clone(), + app: app.to_string(), + source, + data, + dark: call.args["dark"].as_bool().unwrap_or(false), + width: None, + output: output.file.try_clone().map_err(|e| e.to_string())?, + reply: tx, + dev_tag: tag.clone(), + })?; + if call.call_id.starts_with("studio-test-") { + makepad_widgets::log!("studio-test-started: {}", call.call_id); + } + let timeout = + Duration::from_millis(call.timeout_ms.saturating_sub(500).max(1)).min(MAX_WAIT); + let deadline = Instant::now() + timeout; + let result = loop { + if cancel.load(Ordering::SeqCst) || !reply.is_open() || !authorized(app, tag) { + crate::studio::cancel(&call.call_id); + break Err("cancelled or developer mode revoked".to_string()); + } + if Instant::now() >= deadline { + crate::studio::cancel(&call.call_id); + break Err("studio render timed out".to_string()); + } + match rx.recv_timeout(Duration::from_millis(25)) { + Ok(value) => break value, + Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {} + Err(_) => break Err("studio renderer stopped".into()), + } + }?; + if !authorized(app, tag) || cancel.load(Ordering::SeqCst) || !reply.is_open() { + return Err("cancelled or developer mode revoked".into()); + } + if call.caller_kind == CallerKind::AppPeer { + app_available(call)?; + } + scope.verify(root, call.context_id.as_deref())?; + // Pinned octos resolves a peer-context session to binding.cwd, + // /contexts/ (app_binding::resolve_session_app_binding). + // Its view_image is rooted there, not at the parent peer workspace. + let path = output.keep(); + Ok( + json!({"path":path,"width":result.width,"height":result.height,"settled":result.settled}), + ) + }; + match run() { + Ok(data) => ToolOutcome::Ok(data), + Err(why) => ToolOutcome::error("studio_render_failed", why), + } + } +} + +/// Explicit developer test action only. Its workspace comes from a local +/// launch fixture, never from model tool arguments. No provider is required. +pub fn test_action(spec_path: &str) -> Result<(), String> { + let home = crate::octosense::paths::home(); + makepad_widgets::log!( + "studio-test-profile: home={} canonical={} build={:?}", + home.display(), + std::fs::canonicalize(&home) + .unwrap_or(home.clone()) + .display(), + crate::dev_mode::BuildKind::current() + ); + if !crate::dev_mode::grants_all(SYSTEM) { + return Err("studio test action requires developer mode for system".into()); + } + use std::io::Read; + let mut text = String::new(); + std::fs::File::open(spec_path) + .map_err(|e| e.to_string())? + .take(8193) + .read_to_string(&mut text) + .map_err(|e| e.to_string())?; + if text.len() > 8192 { + return Err("studio test fixture is too large".into()); + } + let mut args: Value = serde_json::from_str(&text).map_err(|e| e.to_string())?; + let root = PathBuf::from( + args["workspace"] + .as_str() + .ok_or("fixture needs workspace")?, + ); + if !root.is_absolute() { + return Err("fixture workspace must be absolute".into()); + } + args.as_object_mut() + .ok_or("fixture must be an object")? + .remove("workspace"); + let id = format!("studio-test-{}", uuid::Uuid::new_v4()); + let mut call = HostToolCall::parse(&json!({"call_id":id,"name":RENDER,"app":SYSTEM,"session_id":crate::system_chat::session::SYSTEM_SESSION, + "caller":{"kind":"system"},"args":args,"timeout_ms":30_000,"risk":"read"})).map_err(|e| e.to_string())?; + call.calling_app = SYSTEM.into(); + #[cfg(unix)] + let result_file = scoped::Workspace::open(&root, None)?.result_file()?; + let reply = ToolReply::new(id, move |fields| { + #[cfg(unix)] + { + use std::io::Write; + let data = fields.to_string(); + if data.len() <= 8192 { + let mut file = &result_file; + if let Err(error) = file + .write_all(data.as_bytes()) + .and_then(|_| file.sync_all()) + { + makepad_widgets::log!("studio-test-result-file-error: {}", error); + } + } + } + makepad_widgets::log!("studio-test-result: {}", fields) + }); + StudioExecutor::default().execute_scoped(call, reply, root); + Ok(()) +} + +/// Explicit launch-only instrument transport for an already authored project. +/// It shares the system owner and executor with real agent tools. The private +/// spool never enables developer mode and is not exposed as an agent tool. +#[cfg(unix)] +pub fn test_flow(spec_path: &str) -> Result<(), String> { + use std::io::{Read, Write}; + let Some(tag) = crate::dev_mode::tag().filter(|t| authorized(SYSTEM, t)) else { + return Err("studio flow requires developer mode for system".into()); + }; + let mut text = String::new(); + std::fs::File::open(spec_path) + .map_err(|e| e.to_string())? + .take(8193) + .read_to_string(&mut text) + .map_err(|e| e.to_string())?; + if text.len() > 8192 { + return Err("studio flow config exceeds limit".into()); + } + let config: Value = serde_json::from_str(&text).map_err(|e| e.to_string())?; + let root = PathBuf::from(config["workspace"].as_str().ok_or("flow needs workspace")?); + if !root.is_absolute() { + return Err("flow workspace must be absolute".into()); + } + for name in ["requests", "responses"] { + let dir = root.join(name); + std::fs::create_dir_all(&dir).map_err(|e| e.to_string())?; + if std::fs::symlink_metadata(&dir) + .map_err(|e| e.to_string())? + .file_type() + .is_symlink() + { + return Err("flow directory must not be a symlink".into()); + } + } + let scope = scoped::Workspace::open(&root, None)?; + std::thread::spawn(move || { + let executor = StudioExecutor::default(); + let mut seen = std::collections::HashSet::new(); + while authorized(SYSTEM, &tag) && seen.len() < 1024 { + let Ok(entries) = std::fs::read_dir(root.join("requests")) else { + break; + }; + for entry in entries.flatten() { + if seen.len() >= 1024 { + break; + } + let name = entry.file_name().to_string_lossy().into_owned(); + let Some(id) = name.strip_suffix(".json") else { + continue; + }; + if uuid::Uuid::parse_str(id).is_err() || seen.contains(&name) { + continue; + } + // A durable response (including an interrupted empty claim) + // means this UUID was consumed. Never replay input/install + // when the test process restarts to check persistent state. + if scope + .read(&format!("responses/{name}"), 1024 * 1024) + .is_ok() + { + continue; + } + let Ok(text) = scope.read(&format!("requests/{name}"), 16 * 1024) else { + continue; + }; + let Ok(request) = serde_json::from_str::(&text) else { + continue; + }; + seen.insert(name.clone()); + let tool = request["name"].as_str().unwrap_or(""); + if !is_tool(tool) { + continue; + } + let mut call = match HostToolCall::parse( + &json!({"call_id":id,"name":tool,"app":SYSTEM,"session_id":crate::system_chat::session::SYSTEM_SESSION,"caller":{"kind":"system"},"args":request["args"],"timeout_ms":30_000,"risk":"read"}), + ) { + Ok(call) => call, + Err(_) => continue, + }; + call.calling_app = SYSTEM.into(); + let output = match scope.create_file(&format!("responses/{name}")) { + Ok(file) => file, + Err(_) => continue, + }; + let reply = ToolReply::new(id.to_string(), move |fields| { + let data = fields.to_string(); + if data.len() <= 1024 * 1024 { + let mut out = &output; + let _ = out.write_all(data.as_bytes()).and_then(|_| out.sync_all()); + } + }); + executor.execute_scoped(call, reply, root.clone()); + } + std::thread::sleep(Duration::from_millis(100)); + } + }); + Ok(()) +} + +#[cfg(unix)] +pub(super) mod scoped { + use super::*; + use std::ffi::{CString, OsStr}; + use std::fs::File; + use std::io::Read; + use std::os::fd::{AsRawFd, FromRawFd, OwnedFd}; + use std::os::unix::ffi::OsStrExt; + use std::os::unix::fs::MetadataExt; + use std::path::Component; + + pub struct Workspace { + dir: Arc, + identity: (u64, u64), + } + fn cstr(s: &OsStr) -> Result { + CString::new(s.as_bytes()).map_err(|_| "NUL in path".into()) + } + fn openat(dir: i32, name: &OsStr, flags: i32) -> Result { + let c = cstr(name)?; + let fd = unsafe { + libc::openat( + dir, + c.as_ptr(), + flags | libc::O_NOFOLLOW | libc::O_CLOEXEC | libc::O_NONBLOCK, + 0o600, + ) + }; + if fd < 0 { + return Err(std::io::Error::last_os_error().to_string()); + } + Ok(unsafe { OwnedFd::from_raw_fd(fd) }) + } + fn parts(path: &str) -> Result, String> { + let parts: Result, _> = Path::new(path) + .components() + .map(|p| match p { + Component::Normal(n) => Ok(n), + _ => Err("path must stay relative to your own workspace".to_string()), + }) + .collect(); + let parts = parts?; + if parts.is_empty() { + return Err("empty path".into()); + } + if parts[0].to_string_lossy().eq_ignore_ascii_case("contexts") { + return Err("other conversation folders are not inputs".into()); + } + Ok(parts) + } + impl Workspace { + pub fn create_file(&self, path: &str) -> Result { + let names = parts(path)?; + let mut dir = self.dir.clone(); + for name in &names[..names.len() - 1] { + dir = Arc::new(openat( + dir.as_raw_fd(), + name, + libc::O_RDONLY | libc::O_DIRECTORY, + )?); + } + openat( + dir.as_raw_fd(), + names[names.len() - 1], + libc::O_WRONLY | libc::O_CREAT | libc::O_EXCL, + ) + .map(File::from) + } + /// Copy only bounded regular files through directory descriptors. + /// Neither a symlink swap nor an authored relative path can redirect + /// this snapshot outside the calling conversation's workspace. + pub fn copy_bundle( + &self, + path: &str, + target: &Path, + files: usize, + bytes: usize, + per_file: usize, + ) -> Result<(), String> { + let mut dir = self.dir.clone(); + for name in parts(path)? { + dir = Arc::new(openat( + dir.as_raw_fd(), + name, + libc::O_RDONLY | libc::O_DIRECTORY, + )?); + } + let mut budget = (files, bytes); + copy_directory(dir.as_raw_fd(), target, 0, &mut budget, per_file) + } + pub fn open(root: &Path, context: Option<&str>) -> Result { + let mut fd = openat( + libc::AT_FDCWD, + root.as_os_str(), + libc::O_RDONLY | libc::O_DIRECTORY, + )?; + if let Some(context) = context { + let component = parts(context)?; + if component.len() != 1 { + return Err("invalid trusted context id".into()); + } + fd = openat( + fd.as_raw_fd(), + OsStr::new("contexts"), + libc::O_RDONLY | libc::O_DIRECTORY, + )?; + fd = openat( + fd.as_raw_fd(), + component[0], + libc::O_RDONLY | libc::O_DIRECTORY, + )?; + } + let file = File::from(fd); + let m = file.metadata().map_err(|e| e.to_string())?; + Ok(Self { + identity: (m.dev(), m.ino()), + dir: Arc::new(file.into()), + }) + } + pub fn verify(&self, root: &Path, context: Option<&str>) -> Result<(), String> { + if Self::open(root, context)?.identity != self.identity { + return Err("workspace changed during render".into()); + } + Ok(()) + } + pub fn read(&self, path: &str, cap: usize) -> Result { + let names = parts(path)?; + let mut dir = self.dir.clone(); + for name in &names[..names.len() - 1] { + dir = Arc::new(openat( + dir.as_raw_fd(), + name, + libc::O_RDONLY | libc::O_DIRECTORY, + )?); + } + let mut file = File::from(openat( + dir.as_raw_fd(), + names[names.len() - 1], + libc::O_RDONLY, + )?); + let meta = file.metadata().map_err(|e| e.to_string())?; + if !meta.is_file() || meta.len() > cap as u64 { + return Err("input must be a bounded regular file".into()); + } + let mut out = String::new(); + (&mut file) + .take(cap as u64 + 1) + .read_to_string(&mut out) + .map_err(|e| e.to_string())?; + if out.len() > cap { + return Err("input exceeds render limit".into()); + } + Ok(out) + } + pub fn result_file(&self) -> Result { + openat( + self.dir.as_raw_fd(), + OsStr::new("studio-result.json"), + libc::O_WRONLY | libc::O_CREAT | libc::O_EXCL, + ) + .map(File::from) + } + pub fn output(&self) -> Result { + self.output_extension("png") + } + pub fn output_json(&self) -> Result { + self.output_extension("json") + } + fn output_extension(&self, extension: &str) -> Result { + let name = format!(".studio-{}.{}", uuid::Uuid::new_v4(), extension); + let fd = openat( + self.dir.as_raw_fd(), + OsStr::new(&name), + libc::O_WRONLY | libc::O_CREAT | libc::O_EXCL, + )?; + Ok(Output { + file: File::from(fd), + dir: self.dir.clone(), + name, + keep: false, + }) + } + } + fn copy_directory( + fd: i32, + target: &Path, + depth: usize, + budget: &mut (usize, usize), + per_file: usize, + ) -> Result<(), String> { + use std::os::unix::ffi::OsStringExt; + if depth > 8 { + return Err("bundle directory depth exceeds 8".into()); + } + let duplicate = unsafe { libc::dup(fd) }; + if duplicate < 0 { + return Err(std::io::Error::last_os_error().to_string()); + } + let stream = unsafe { libc::fdopendir(duplicate) }; + if stream.is_null() { + unsafe { + libc::close(duplicate); + } + return Err(std::io::Error::last_os_error().to_string()); + } + struct Dir(*mut libc::DIR); + impl Drop for Dir { + fn drop(&mut self) { + unsafe { + libc::closedir(self.0); + } + } + } + let stream = Dir(stream); + loop { + let entry = unsafe { libc::readdir(stream.0) }; + if entry.is_null() { + break; + } + let name = unsafe { std::ffi::CStr::from_ptr((*entry).d_name.as_ptr()) }.to_bytes(); + if name == b"." || name == b".." { + continue; + } + if budget.0 == 0 { + return Err("bundle exceeds file/directory count limit".into()); + } + budget.0 -= 1; + let name = std::ffi::OsString::from_vec(name.to_vec()); + let input = File::from(openat(fd, &name, libc::O_RDONLY)?); + let meta = input.metadata().map_err(|e| e.to_string())?; + let output = target.join(&name); + if meta.is_dir() { + std::fs::create_dir(&output).map_err(|e| e.to_string())?; + copy_directory(input.as_raw_fd(), &output, depth + 1, budget, per_file)?; + } else if meta.is_file() { + let cap = per_file.min(budget.1); + if meta.len() > cap as u64 { + return Err("bundle exceeds byte limit".into()); + } + let mut bytes = Vec::new(); + input + .take(cap as u64 + 1) + .read_to_end(&mut bytes) + .map_err(|e| e.to_string())?; + if bytes.len() > cap { + return Err("bundle exceeds byte limit".into()); + } + budget.1 -= bytes.len(); + std::fs::write(&output, bytes).map_err(|e| e.to_string())?; + } else { + return Err("bundle accepts only regular files and directories".into()); + } + } + Ok(()) + } + pub struct Output { + pub file: File, + dir: Arc, + name: String, + keep: bool, + } + impl Output { + pub fn path(&self) -> &str { + &self.name + } + pub fn keep(mut self) -> String { + self.keep = true; + self.name.clone() + } + } + impl Drop for Output { + fn drop(&mut self) { + if !self.keep { + if let Ok(name) = CString::new(self.name.clone()) { + unsafe { + libc::unlinkat(self.dir.as_raw_fd(), name.as_ptr(), 0); + } + } + } + } + } +} + +#[cfg(all(test, unix))] +mod tests { + use super::*; + use std::os::unix::fs::symlink; + + #[test] + fn context_inputs_and_outputs_stay_with_the_caller() { + let temp = crate::app_storage::tests::Scratch::new("studio"); + let root = temp.0.as_path(); + std::fs::create_dir_all(root.join("contexts/a")).unwrap(); + std::fs::create_dir_all(root.join("contexts/b")).unwrap(); + std::fs::write(root.join("contexts/a/card.l0"), "own").unwrap(); + std::fs::write(root.join("contexts/b/card.l0"), "other").unwrap(); + let scope = scoped::Workspace::open(root, Some("a")).unwrap(); + assert_eq!(scope.read("card.l0", SOURCE_MAX).unwrap(), "own"); + for path in ["../b/card.l0", "/etc/passwd", "../../card.l0"] { + assert!(scope.read(path, SOURCE_MAX).is_err(), "{path}"); + } + assert!(scoped::Workspace::open(root, Some("../b")).is_err()); + let output = scope.output().unwrap(); + let name = output.keep(); + assert!(root.join("contexts/a").join(&name).is_file()); + assert!(!root.join(&name).exists()); + assert!(!root.join("contexts/b").join(&name).exists()); + let peer = scoped::Workspace::open(root, None).unwrap(); + assert!(peer.read("contexts/b/card.l0", SOURCE_MAX).is_err()); + assert!(peer.read("Contexts/b/card.l0", SOURCE_MAX).is_err()); + } + + #[test] + fn inputs_reject_links_devices_and_oversized_data() { + let temp = crate::app_storage::tests::Scratch::new("studio"); + let outside = crate::app_storage::tests::Scratch::new("studio"); + std::fs::write(outside.0.as_path().join("private"), "private").unwrap(); + symlink(outside.0.as_path(), temp.0.as_path().join("jump")).unwrap(); + symlink( + outside.0.as_path().join("private"), + temp.0.as_path().join("file-link"), + ) + .unwrap(); + symlink("/dev/zero", temp.0.as_path().join("device")).unwrap(); + std::fs::write(temp.0.as_path().join("large"), vec![b'x'; DATA_MAX + 1]).unwrap(); + let scope = scoped::Workspace::open(temp.0.as_path(), None).unwrap(); + for path in ["jump/private", "file-link", "device", "large"] { + assert!(scope.read(path, DATA_MAX).is_err(), "{path}"); + } + std::fs::create_dir_all(temp.0.as_path().join("contexts")).unwrap(); + symlink(outside.0.as_path(), temp.0.as_path().join("contexts/other")).unwrap(); + assert!(scoped::Workspace::open(temp.0.as_path(), Some("other")).is_err()); + } + + #[test] + fn failed_outputs_are_unlinked_and_workspace_replacement_is_detected() { + let temp = crate::app_storage::tests::Scratch::new("studio"); + let root = temp.0.as_path().join("workspace"); + std::fs::create_dir(&root).unwrap(); + let scope = scoped::Workspace::open(&root, None).unwrap(); + { + let _output = scope.output().unwrap(); + let artifact = scope.output_json().unwrap(); + assert!(artifact.path().ends_with(".json")); + } + assert_eq!(std::fs::read_dir(&root).unwrap().count(), 0); + std::fs::rename(&root, temp.0.as_path().join("old")).unwrap(); + std::fs::create_dir(&root).unwrap(); + assert!(scope.verify(&root, None).is_err()); + } + + #[test] + fn schema_does_not_accept_workspace_output_or_bundle_paths() { + let schema = declaration("system")["input_schema"].clone(); + assert!(super::super::schema::check( + &schema, + &json!({"source_path":"main.card","dark":true}) + ) + .is_ok()); + for extra in [ + "workspace", + "app", + "context_id", + "output", + "bundle", + "width", + ] { + let mut args = json!({"source_path":"main.card"}); + args[extra] = json!("untrusted"); + assert!( + super::super::schema::check(&schema, &args).is_err(), + "{extra}" + ); + } + } + + #[test] + fn flow_response_claim_survives_restart_without_replaying_a_request() { + let temp = crate::app_storage::tests::Scratch::new("studio-flow-claim"); + std::fs::create_dir(temp.0.join("responses")).unwrap(); + let path = "responses/11111111-1111-4111-8111-111111111111.json"; + { + let first = scoped::Workspace::open(&temp.0, None).unwrap(); + let _claim = first.create_file(path).unwrap(); + assert!(first.create_file(path).is_err()); + } + let restarted = scoped::Workspace::open(&temp.0, None).unwrap(); + // Even a crash before writing the response consumes the UUID. The + // polling loop sees this before adding it to its per-process limit. + assert_eq!(restarted.read(path, 1024 * 1024).unwrap(), ""); + assert!(restarted.create_file(path).is_err()); + } + + #[test] + fn cancellation_marks_the_running_call() { + let executor = StudioExecutor::default(); + let flag = Arc::new(AtomicBool::new(false)); + let id = format!("cancel-test-{}", uuid::Uuid::new_v4()); + executor + .running + .lock() + .unwrap() + .insert(id.clone(), flag.clone()); + executor.cancel(&id); + assert!(flag.load(Ordering::SeqCst)); + } +} diff --git a/crates/shell/src/host_tools/studio_bundles.rs b/crates/shell/src/host_tools/studio_bundles.rs new file mode 100644 index 00000000..cdcb8bc5 --- /dev/null +++ b/crates/shell/src/host_tools/studio_bundles.rs @@ -0,0 +1,825 @@ +//! Host-owned developer bundle snapshots. These are local installs, never +//! signed catalog releases. Source trees remain in the author's workspace; +//! admitted bytes and DevTag receipts stay outside every app's storage jail. +use super::studio::scoped::Workspace; +use crate::ai_host::app_peers::host_tools::HostToolCall; +use crate::dev_mode::DevTag; +use octosense_app_policy::{AppManifest, AppPolicy, HostLimits, RefuseAllSignatures}; +use serde::{Deserialize, Serialize}; +use serde_json::{json, Value}; +use std::{ + path::{Path, PathBuf}, + sync::Arc, +}; + +pub const MAX_FILES: usize = 128; +pub const MAX_BYTES: usize = 2 * 1024 * 1024; +pub const MAX_FILE_BYTES: usize = 512 * 1024; +pub const MAX_SOURCE_BYTES: usize = 64 * 1024; +static INSTALL_LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(()); + +/// Identity comes from broker/session stamping, never tool arguments. +#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] +pub struct Owner { + pub app: String, + pub account: Option, + pub session: String, + pub context: Option, +} +impl Owner { + pub fn of(call: &HostToolCall) -> Self { + Self { + app: call.calling_app.clone(), + account: call.account.clone(), + session: call.session_id.clone(), + context: call.context_id.clone(), + } + } +} + +pub struct Bundle { + pub id: String, + pub name: String, + pub version: String, + pub digest: String, + pub root: PathBuf, + pub source: String, + pub policy: AppPolicy, + pub owner: Owner, + pub dev_tag: DevTag, + /// Installed snapshots are retained; temporary previews remove theirs. + retained: std::sync::atomic::AtomicBool, +} +impl Bundle { + pub fn valid(&self) -> bool { + crate::dev_mode::tag_valid(&self.dev_tag) + && crate::dev_mode::grants_all(super::app_of_peer(&self.owner.app)) + } + pub fn summary(&self) -> Value { + json!({"app_id":self.id,"name":self.name,"version":self.version, + "digest":self.digest,"source_modified":false,"admission":"local_developer", + "capabilities":self.policy.capabilities,"publisher_signed":false}) + } +} +impl Drop for Bundle { + fn drop(&mut self) { + if !self.retained.load(std::sync::atomic::Ordering::Acquire) { + let _ = std::fs::remove_dir_all(&self.root); + } + } +} +fn base() -> Result { + crate::octosense::paths::private_dir("studio").map_err(|e| e.to_string()) +} +fn private_child(parent: &Path, child: &str) -> Result { + let path = parent.join(child); + std::fs::create_dir_all(&path).map_err(|e| e.to_string())?; + let meta = std::fs::symlink_metadata(&path).map_err(|e| e.to_string())?; + if !meta.is_dir() || meta.file_type().is_symlink() { + return Err("studio directory must be private and real".into()); + } + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o700)) + .map_err(|e| e.to_string())?; + Ok(path) +} +fn valid_id(id: &str) -> Result<(), String> { + if !id.starts_with("dev.studio.") + || id.len() > 100 + || id.len() <= 11 + || !id.bytes().all(|b| { + b.is_ascii_lowercase() || b.is_ascii_digit() || matches!(b, b'.' | b'-' | b'_') + }) + { + return Err("developer app ids must be under dev.studio.*".into()); + } + crate::apps::check_script_app_id(id) +} + +pub fn stage( + scope: &Workspace, + relative: &str, + owner: Owner, + tag: DevTag, +) -> Result, String> { + let snapshots = private_child(&base()?, "bundles")?; + let root = snapshots.join(uuid::Uuid::new_v4().to_string()); + std::fs::create_dir(&root).map_err(|e| e.to_string())?; + let result = (|| { + scope.copy_bundle(relative, &root, MAX_FILES, MAX_BYTES, MAX_FILE_BYTES)?; + admit_snapshot(root.clone(), owner, tag, None) + })(); + if result.is_err() { + let _ = std::fs::remove_dir_all(&root); + } + result.map(Arc::new) +} +fn admit_snapshot( + root: PathBuf, + owner: Owner, + tag: DevTag, + expected: Option<(&str, &str)>, +) -> Result { + let manifest_path = root.join("manifest.json"); + let text = + std::fs::read_to_string(&manifest_path).map_err(|e| format!("manifest.json: {e}"))?; + if text.len() > 64 * 1024 { + return Err("manifest exceeds 64 KiB".into()); + } + let mut raw: Value = serde_json::from_str(&text).map_err(|e| e.to_string())?; + if raw + .pointer("/integrity/signature") + .is_some_and(|s| !s.is_null()) + { + return Err( + "developer bundles must be unsigned; Studio never impersonates a publisher".into(), + ); + } + let digest = octosense_app_policy::digest_dir(&root)?; + if let Some((wanted_digest, wanted_manifest)) = expected { + if digest != wanted_digest || text != wanted_manifest { + return Err("installed developer bundle changed after admission".into()); + } + } else { + if !raw.is_object() { + return Err("manifest must be an object".into()); + } + if raw.get("integrity").is_none() { + raw["integrity"] = json!({}); + } + if !raw["integrity"].is_object() { + return Err("integrity must be an object".into()); + } + raw["integrity"]["bundle_blake3"] = json!(digest); + } + let text = if expected.is_some() { + text + } else { + serde_json::to_string_pretty(&raw).map_err(|e| e.to_string())? + }; + let manifest = AppManifest::parse(&text)?; + valid_id(&manifest.id)?; + if manifest.capabilities.iter().any(|c| c != "storage") + || !manifest.network.hosts.is_empty() + || manifest.agent.is_some() + || manifest.storage.accounts + { + return Err("this Studio release accepts offline, storage-only script apps without accounts or agents".into()); + } + let policy = octosense_app_policy::admit_and_resolve_dir( + &text, + &digest, + &HostLimits::default() + .with_require_signature(false) + .with_max_storage_bytes(1024 * 1024) + .with_max_instruction_budget(5_000_000) + .with_max_memory_bytes(16 * 1024 * 1024), + &RefuseAllSignatures, + )?; + let source = + octosense_app_policy::script_source(&root, "").ok_or("bundle needs main.splash")??; + if source.is_empty() || source.len() > MAX_SOURCE_BYTES { + return Err("main.splash must be nonempty and at most 64 KiB".into()); + } + if source.contains("{{assets}}") || source.contains("http_resource(") { + return Err("this Studio release accepts resource-free script screens; local launcher artwork may still ship in the bundle".into()); + } + if expected.is_none() { + std::fs::write(&manifest_path, &text).map_err(|e| e.to_string())?; + } + Ok(Bundle { + id: manifest.id, + name: manifest.name, + version: manifest.version, + digest, + root, + source, + policy, + owner, + dev_tag: tag, + retained: std::sync::atomic::AtomicBool::new(expected.is_some()), + }) +} + +#[derive(Serialize, Deserialize)] +struct Receipt { + owner: Owner, + profile_id: String, + since: u64, + snapshot: String, + digest: String, + manifest: String, +} +/// Commit only host-owned, already-admitted bytes. Existing data remains for +/// an update by the same owner; a different conversation cannot take its id. +pub fn install(bundle: &Arc) -> Result { + let _guard = INSTALL_LOCK.lock().unwrap_or_else(|e| e.into_inner()); + if !bundle.valid() { + return Err("developer grant expired".into()); + } + let receipts = private_child(&base()?, "installed")?; + let path = receipts.join(format!("{}.json", bundle.id)); + let previous = if path.exists() { + let old: Receipt = + serde_json::from_slice(&std::fs::read(&path).map_err(|e| e.to_string())?) + .map_err(|e| e.to_string())?; + if old.owner != bundle.owner { + return Err("this app id belongs to another Studio conversation".into()); + } + Some(old.snapshot) + } else { + if std::fs::read_dir(&receipts) + .map_err(|e| e.to_string())? + .flatten() + .filter(|e| e.path().extension().is_some_and(|s| s == "json")) + .count() + >= 16 + { + return Err("Studio supports at most 16 local developer installs".into()); + } + None + }; + let receipt = Receipt { + owner: bundle.owner.clone(), + profile_id: bundle.dev_tag.profile_id.clone(), + since: bundle.dev_tag.since, + snapshot: bundle + .root + .file_name() + .ok_or("invalid snapshot")? + .to_string_lossy() + .into_owned(), + digest: bundle.digest.clone(), + manifest: std::fs::read_to_string(bundle.root.join("manifest.json")) + .map_err(|e| e.to_string())?, + }; + let temp = receipts.join(format!(".{}.tmp", uuid::Uuid::new_v4())); + use std::io::Write; + use std::os::unix::fs::OpenOptionsExt; + let mut file = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(&temp) + .map_err(|e| e.to_string())?; + file.write_all(&serde_json::to_vec(&receipt).map_err(|e| e.to_string())?) + .and_then(|_| file.sync_all()) + .map_err(|e| e.to_string())?; + if !bundle.valid() { + let _ = std::fs::remove_file(&temp); + return Err("developer grant expired".into()); + } + std::fs::rename(&temp, &path).map_err(|e| e.to_string())?; + bundle + .retained + .store(true, std::sync::atomic::Ordering::Release); + if let Some(old) = previous.filter(|old| uuid::Uuid::parse_str(old).is_ok()) { + let old = bundle.root.parent().unwrap().join(old); + if old != bundle.root { + let _ = std::fs::remove_dir_all(old); + } + } + let mut result = bundle.summary(); + result["installed"] = json!(true); + Ok(result) +} +/// Launcher calls with None; host tools pass the full calling identity. +pub fn installed(id: &str, owner: Option<&Owner>) -> Result, String> { + valid_id(id)?; + let root = base()?; + let bytes = std::fs::read(root.join("installed").join(format!("{id}.json"))) + .map_err(|e| e.to_string())?; + if bytes.len() > 128 * 1024 { + return Err("invalid developer receipt".into()); + } + let receipt: Receipt = serde_json::from_slice(&bytes).map_err(|e| e.to_string())?; + if owner.is_some_and(|o| o != &receipt.owner) { + return Err("this app belongs to another Studio conversation".into()); + } + let tag = DevTag { + profile_id: receipt.profile_id, + since: receipt.since, + }; + if !crate::dev_mode::tag_valid(&tag) + || !crate::dev_mode::grants_all(super::app_of_peer(&receipt.owner.app)) + { + return Err("developer grant expired".into()); + } + uuid::Uuid::parse_str(&receipt.snapshot).map_err(|_| "invalid developer snapshot")?; + let bundle = admit_snapshot( + root.join("bundles").join(receipt.snapshot), + receipt.owner, + tag, + Some((&receipt.digest, &receipt.manifest)), + )?; + if bundle.id != id { + return Err("developer receipt identity mismatch".into()); + } + Ok(Arc::new(bundle)) +} +pub fn installed_apps() -> Vec> { + let Ok(root) = base() else { return Vec::new() }; + let Ok(entries) = std::fs::read_dir(root.join("installed")) else { + return Vec::new(); + }; + entries + .flatten() + .filter_map(|e| e.path().file_stem()?.to_str().map(str::to_owned)) + .filter_map(|id| installed(&id, None).ok()) + .collect() +} +pub fn data_dir(bundle: &Bundle) -> Result { + if !bundle.valid() { + return Err("developer grant expired".into()); + } + private_child(&private_child(&base()?, "data")?, &bundle.id) +} + +static GENERATION: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0); +pub fn generation() -> u64 { + GENERATION.load(std::sync::atomic::Ordering::Acquire) +} +fn owner_key(owner: &Owner) -> String { + serde_json::to_string(owner).expect("owner serializes") +} +fn open_bundle(bundle: Arc, persistent: bool) -> Result { + if !bundle.valid() { + return Err("developer grant expired".into()); + } + let instance_id = uuid::Uuid::new_v4().to_string(); + let jail = if persistent { + data_dir(&bundle)? + } else { + private_child(&private_child(&base()?, "previews")?, &instance_id)? + }; + let admission = crate::studio::apps::stage_open(crate::studio::apps::OpenSpec { + instance_id: instance_id.clone(), + app_id: if persistent { + Some(bundle.id.clone()) + } else { + None + }, + owner: owner_key(&bundle.owner), + dev_tag: bundle.dev_tag.clone(), + source: bundle.source.clone(), + storage_quota: if bundle.policy.allows("storage") { + bundle.policy.storage_bytes + } else { + 0 + }, + instruction_budget: bundle.policy.instruction_budget, + memory_bytes: bundle.policy.memory_bytes, + jail: jail.clone(), + persistent, + title: bundle.name.clone(), + }); + if let Err(error) = admission { + if !persistent { + let _ = std::fs::remove_dir_all(&jail); + } + return Err(error); + } + Ok(instance_id) +} +/// A person launches a validated installed receipt through the same UI queue. +pub fn request_launch_installed(id: &str) -> Result { + open_bundle(installed(id, None)?, true) +} + +pub fn execute( + call: &HostToolCall, + root: &Path, + tag: &DevTag, + cancel: &std::sync::atomic::AtomicBool, + reply: &crate::ai_host::app_peers::host_tools::ToolReply, +) -> crate::ai_host::app_peers::host_tools::ToolOutcome { + use crate::ai_host::app_peers::host_tools::ToolOutcome; + let run = || -> Result { + let scope = Workspace::open(root, call.context_id.as_deref())?; + let owner = Owner::of(call); + let mut opened: Option = None; + let mut guard = OpenGuard { + instance: None, + owner: owner_key(&owner), + tag: tag.clone(), + }; + let mut opened_summary = None; + let mut output = None; + let (instance_id, action) = match call.name.as_str() { + "studio.bundle_check" | "studio.install" => { + let bundle = stage( + &scope, + call.args["bundle_path"].as_str().unwrap_or(""), + owner, + tag.clone(), + )?; + if cancel.load(std::sync::atomic::Ordering::Acquire) || !reply.is_open() { + return Err("studio_cancelled".into()); + } + scope.verify(root, call.context_id.as_deref())?; + if call.name == "studio.install" { + let result = install(&bundle)?; + GENERATION.fetch_add(1, std::sync::atomic::Ordering::AcqRel); + makepad_widgets::SignalToUI::set_ui_signal(); + return Ok(result); + } + return Ok(bundle.summary()); + } + "studio.open" => { + let path = call.args["bundle_path"].as_str(); + let id = call.args["app_id"].as_str(); + if path.is_some() == id.is_some() { + return Err("supply exactly one of bundle_path or app_id".into()); + } + let persistent = id.is_some(); + let bundle = if let Some(id) = id { + installed(id, Some(&owner))? + } else { + stage(&scope, path.unwrap(), owner.clone(), tag.clone())? + }; + opened_summary = Some(bundle.summary()); + let instance = open_bundle(bundle, persistent)?; + guard.instance = Some(instance.clone()); + opened = Some(instance.clone()); + (instance, crate::studio::apps::Action::Ready) + } + "studio.input" => { + let kind = call.args["action"].as_str().unwrap_or(""); + let text = call.args["text"].as_str().map(str::to_owned); + let delta_y = call.args["delta_y"].as_f64(); + if kind == "scroll" && (delta_y.is_none() || text.is_some()) { + return Err("scroll requires delta_y and no text".into()); + } + if kind != "scroll" && delta_y.is_some() { + return Err("delta_y is only for scroll".into()); + } + if kind == "text" && text.is_none() { + return Err("text action requires text".into()); + } + if kind == "tap" && text.is_some() { + return Err("tap action takes no text".into()); + } + ( + call.args["instance_id"].as_str().unwrap_or("").into(), + crate::studio::apps::Action::Input { + widget_id: call.args["widget_id"].as_str().unwrap_or("").into(), + kind: kind.into(), + text, + delta_y, + }, + ) + } + "studio.inspect" => { + let file = scope.output()?; + let action = crate::studio::apps::Action::Inspect { + output: file.file.try_clone().map_err(|e| e.to_string())?, + }; + output = Some(file); + ( + call.args["instance_id"].as_str().unwrap_or("").into(), + action, + ) + } + "studio.close" => ( + call.args["instance_id"].as_str().unwrap_or("").into(), + crate::studio::apps::Action::Close, + ), + _ => return Err("unsupported Studio operation".into()), + }; + let (tx, rx) = std::sync::mpsc::channel(); + crate::studio::apps::submit(crate::studio::apps::Request { + id: call.call_id.clone(), + owner: owner_key(&owner), + dev_tag: tag.clone(), + instance_id: instance_id.clone(), + action, + reply: tx, + })?; + let deadline = std::time::Instant::now() + + std::time::Duration::from_millis( + call.timeout_ms.saturating_sub(500).clamp(1, 25_000), + ); + let mut result = loop { + if cancel.load(std::sync::atomic::Ordering::Acquire) + || !reply.is_open() + || !crate::dev_mode::tag_valid(tag) + || !crate::dev_mode::grants_all(super::app_of_peer(&owner.app)) + || std::time::Instant::now() >= deadline + { + crate::studio::apps::cancel(&call.call_id); + return Err("studio_cancelled_or_timed_out".into()); + } + match rx.recv_timeout(std::time::Duration::from_millis(25)) { + Ok(result) => break result?, + Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {} + Err(_) => return Err("studio app runner stopped".into()), + } + }; + scope.verify(root, call.context_id.as_deref())?; + if !crate::dev_mode::tag_valid(tag) + || !reply.is_open() + || cancel.load(std::sync::atomic::Ordering::Acquire) + { + return Err("studio_cancelled".into()); + } + if let Some(id) = opened { + result["instance_id"] = json!(id); + if let Some(summary) = opened_summary { + result["bundle"] = summary; + } + } + if let Some(output) = output { + use std::io::Write; + result["path"] = json!(output.path()); + // Line-oriented diagnostics let read_file offset/limit recover a + // bounded section through the kernel's own model-output budget. + let bytes = serde_json::to_vec_pretty(&result).map_err(|e| e.to_string())?; + if bytes.len() > INSPECT_ARTIFACT_BYTES { + return Err("studio_snapshot_too_large".into()); + } + let mut artifact = scope.output_json()?; + artifact + .file + .write_all(&bytes) + .and_then(|_| artifact.file.sync_all()) + .map_err(|e| e.to_string())?; + let offset = call + .args + .get("offset") + .map(|v| { + v.as_u64() + .and_then(|n| usize::try_from(n).ok()) + .ok_or("studio_inspect_offset_invalid") + }) + .transpose()? + .unwrap_or(0); + result = compact_inspect(&result, artifact.path(), offset)?; + // Both guards unlink on any error, including revocation or a + // replaced context directory during artifact serialization. + scope.verify(root, call.context_id.as_deref())?; + if !crate::dev_mode::tag_valid(tag) + || !crate::dev_mode::grants_all(super::app_of_peer(&owner.app)) + || !reply.is_open() + || cancel.load(std::sync::atomic::Ordering::Acquire) + { + return Err("studio_cancelled".into()); + } + output.keep(); + artifact.keep(); + } + guard.instance = None; + Ok(result) + }; + match run() { + Ok(value) => ToolOutcome::Ok(value), + Err(why) => ToolOutcome::error("studio_app_failed", why), + } +} + +// The pinned kernel context manager exposes only a 4096-byte prefix of tool +// output to the model. Keep actionable selectors intact inside that budget; +// verbose geometry/tree data lives in the separate caller-scoped artifact. +const INSPECT_REPLY_BYTES: usize = 3800; +const INSPECT_ARTIFACT_BYTES: usize = 1024 * 1024; +fn compact_inspect(full: &Value, snapshot_path: &str, offset: usize) -> Result { + let widgets: Vec<&Value> = full["snapshot"]["widgets"] + .as_array() + .ok_or("studio_snapshot_widgets_missing")? + .iter() + .filter(|w| w["visible"] == true && w["type"] != "Splash") + .collect(); + if offset > widgets.len() { + return Err("studio_inspect_offset_out_of_range".into()); + } + let findings = full["snapshot"]["checks"]["findings"].as_array(); + let mut result = json!({ + "instance_id": full["instance_id"], "path": full["path"], + "snapshot_path": snapshot_path, "width": full["width"], "height": full["height"], + "settled": full["settled"], "offset": offset, "total_widgets": widgets.len(), + "next_offset": null, + "snapshot": {"widgets": [], "checks": { + "pass": full["snapshot"]["checks"]["pass"], + "finding_count": findings.map_or(0, |f| f.len()), + "error_count": findings.map_or(0, |f| f.iter().filter(|v| v["severity"] == "error").count()) + }} + }); + for (i, widget) in widgets.iter().enumerate().skip(offset) { + let mut row = json!({"selector":widget["selector"], "type":widget["type"], + "visible":true, "enabled":widget["enabled"]}); + for field in ["text", "value"] { + if let Some(text) = widget[field].as_str() { + let mut end = text.len().min(160); + while !text.is_char_boundary(end) { + end -= 1; + } + row[field] = json!(&text[..end]); + if end < text.len() { + row[format!("{field}_truncated")] = json!(true); + } + } + } + for field in ["checked", "selected"] { + if !widget[field].is_null() { + row[field] = widget[field].clone(); + } + } + result["snapshot"]["widgets"] + .as_array_mut() + .unwrap() + .push(row); + result["next_offset"] = if i + 1 < widgets.len() { + json!(i + 1) + } else { + Value::Null + }; + if serde_json::to_vec(&result) + .map_err(|e| e.to_string())? + .len() + > INSPECT_REPLY_BYTES + { + result["snapshot"]["widgets"].as_array_mut().unwrap().pop(); + result["next_offset"] = json!(i); + if i == offset { + return Err("studio_widget_selector_exceeds_reply_budget".into()); + } + break; + } + } + if serde_json::to_vec(&result) + .map_err(|e| e.to_string())? + .len() + > INSPECT_REPLY_BYTES + { + return Err("studio_inspect_metadata_exceeds_reply_budget".into()); + } + Ok(result) +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn inspect_pages_fit_model_budget_without_losing_unicode_selectors() { + let widgets: Vec = (0..100).map(|i| json!({ + "selector":format!("完成任务{i}@0"),"type":"Button", "visible":true,"enabled":true, + "text":"中文😀\"\n".repeat(1000), "value":"记录".repeat(100), "checked":false + })).collect(); + let full = json!({"instance_id":"test", "path":".studio-test.png", "width":1080, + "height":2000,"settled":false,"snapshot":{"widgets":widgets, + "checks":{"pass":true,"findings":[]},"geometry":{"large":"x".repeat(50000)},"tree":"verbose"}}); + let mut offset = 0; + let mut seen = Vec::new(); + loop { + let page = compact_inspect(&full, ".studio-test.json", offset).unwrap(); + let serialized = serde_json::to_vec(&page).unwrap(); + assert!(serialized.len() <= INSPECT_REPLY_BYTES); + let rows = page["snapshot"]["widgets"].as_array().unwrap(); + assert!(!rows.is_empty()); + for row in rows { + assert_eq!(row["text_truncated"], true); + assert_eq!(row["value_truncated"], true); + seen.push(row["selector"].as_str().unwrap().to_owned()); + } + match page["next_offset"].as_u64() { + Some(next) => { + assert!(next as usize > offset); + offset = next as usize; + } + None => break, + } + } + assert_eq!( + seen, + (0..100) + .map(|i| format!("完成任务{i}@0")) + .collect::>() + ); + assert!(compact_inspect(&full, "artifact.json", 101).is_err()); + } + #[test] + fn inspect_paging_excludes_hidden_and_source_widgets_and_refuses_unpageable_selector() { + let mut full = json!({"snapshot":{"widgets":[ + {"type":"Splash","visible":true,"selector":"source@0"}, + {"type":"Button","visible":false,"selector":"hidden@0"}, + {"type":"Button","visible":true,"enabled":false,"selector":"disabled@0","text":"No"} + ],"checks":{"pass":false,"findings":[{"severity":"error"},{"severity":"warning"}]}}}); + let page = compact_inspect(&full, "artifact.json", 0).unwrap(); + assert_eq!(page["total_widgets"], 1); + assert_eq!(page["snapshot"]["widgets"][0]["selector"], "disabled@0"); + assert_eq!(page["snapshot"]["checks"]["error_count"], 1); + assert!( + compact_inspect(&full, "artifact.json", 1).unwrap()["snapshot"]["widgets"] + .as_array() + .unwrap() + .is_empty() + ); + full["snapshot"]["widgets"][2]["selector"] = json!("x".repeat(4000)); + assert!(compact_inspect(&full, "artifact.json", 0).is_err()); + } + + fn owner() -> Owner { + Owner { + app: "system".into(), + account: None, + session: "system-session".into(), + context: None, + } + } + fn tag() -> DevTag { + DevTag { + profile_id: "test-profile".into(), + since: 1, + } + } + fn fixture(root: &Path, edit: impl FnOnce(&mut Value)) { + std::fs::create_dir_all(root).unwrap(); + std::fs::write(root.join("main.splash"), "View{Label{text: \"authored\"}}").unwrap(); + let mut manifest = json!({"schema":1,"id":"dev.studio.planner","version":"0.1.0","name":"Planner","capabilities":["storage"],"integrity":{"bundle_blake3":"pending"}}); + edit(&mut manifest); + std::fs::write(root.join("manifest.json"), manifest.to_string()).unwrap(); + } + #[test] + fn staging_is_bounded_confined_and_never_stamps_author_source() { + let home = crate::app_storage::tests::Scratch::new("studio-bundle"); + let author = home.0.join("author"); + fixture(&author, |_| {}); + let original = std::fs::read(author.join("manifest.json")).unwrap(); + let snapshot = home.0.join("copy"); + std::fs::create_dir(&snapshot).unwrap(); + let scope = Workspace::open(&home.0, None).unwrap(); + scope + .copy_bundle("author", &snapshot, MAX_FILES, MAX_BYTES, MAX_FILE_BYTES) + .unwrap(); + let bundle = admit_snapshot(snapshot, owner(), tag(), None).unwrap(); + assert_eq!( + std::fs::read(author.join("manifest.json")).unwrap(), + original + ); + assert_ne!(bundle.digest, "pending"); + assert!(bundle.policy.hosts.is_empty()); + assert_eq!(bundle.policy.storage_bytes, 1024 * 1024); + let manifest = std::fs::read_to_string(bundle.root.join("manifest.json")).unwrap(); + std::fs::write(bundle.root.join("main.splash"), "different").unwrap(); + assert!(admit_snapshot( + bundle.root.clone(), + owner(), + tag(), + Some((&bundle.digest, &manifest)) + ) + .is_err()); + assert!(scope + .copy_bundle("../escape", &home.0.join("never"), 10, 1024, 1024) + .is_err()); + let tiny = home.0.join("tiny"); + std::fs::create_dir(&tiny).unwrap(); + assert!(scope.copy_bundle("author", &tiny, 10, 1, 1).is_err()); + } + #[test] + fn local_admission_cannot_claim_system_ids_publishers_network_or_agents() { + for change in 0..4 { + let home = crate::app_storage::tests::Scratch::new("studio-refusal"); + fixture(&home.0, |m| match change { + 0 => m["id"] = json!("os.mail"), + 1 => m["integrity"]["signature"] = json!({"key_id":"fake","value":"fake"}), + 2 => m["capabilities"] = json!(["storage", "net"]), + _ => m["storage"] = json!({"accounts":true}), + }); + assert!(admit_snapshot(home.0.clone(), owner(), tag(), None).is_err()); + } + } + #[test] + fn snapshot_refuses_symlinks_and_owner_includes_account_and_context() { + let home = crate::app_storage::tests::Scratch::new("studio-links"); + fixture(&home.0.join("author"), |_| {}); + std::os::unix::fs::symlink("/etc/hosts", home.0.join("author/link")).unwrap(); + let target = home.0.join("copy"); + std::fs::create_dir(&target).unwrap(); + let scope = Workspace::open(&home.0, None).unwrap(); + assert!(scope + .copy_bundle("author", &target, MAX_FILES, MAX_BYTES, MAX_FILE_BYTES) + .is_err()); + let a = owner(); + let mut b = a.clone(); + b.context = Some("other".into()); + assert_ne!(owner_key(&a), owner_key(&b)); + b = a.clone(); + b.account = Some("other-account".into()); + assert_ne!(owner_key(&a), owner_key(&b)); + b = a.clone(); + b.session = "different-peer".into(); + assert_ne!(owner_key(&a), owner_key(&b)); + } +} + +struct OpenGuard { + instance: Option, + owner: String, + tag: DevTag, +} +impl Drop for OpenGuard { + fn drop(&mut self) { + if let Some(id) = &self.instance { + let _ = crate::studio::apps::abandon(id, &self.owner, &self.tag); + } + } +} diff --git a/crates/shell/src/host_tools/tests.rs b/crates/shell/src/host_tools/tests.rs index bf98685b..d2ed8a4d 100644 --- a/crates/shell/src/host_tools/tests.rs +++ b/crates/shell/src/host_tools/tests.rs @@ -556,6 +556,9 @@ fn a_peer_is_offered_exactly_its_granted_toolbox_tools_marked_with_their_owner_a assert!(offered_names(&relay, "calendar", false, true).is_empty()); let mut every = shareable_native_tools(); every.push(super::relay::DEV_RUN.to_string()); + if super::studio::SUPPORTED { every.push(super::studio::RENDER.to_string()); } + #[cfg(all(unix, any(feature="app-hub", native_mobile)))] + every.extend(super::studio::APP_TOOLS.iter().map(|name| name.to_string())); every.sort(); assert_eq!(offered_names(&relay, "calendar", true, true), every); } @@ -1171,3 +1174,73 @@ fn a_cancelled_confirm_app_call_is_withdrawn_from_the_apps_sheet() { assert!(!w.router.is_pending(&id), "withdrawn with the call"); assert!(exec.0.lock().unwrap().is_empty()); } + +/// Studio uses the host executor for only its covered owner, including the +/// system session; an arbitrary cross-app declaration cannot grant it. +#[test] +fn studio_render_is_scoped_revocable_and_schema_checked() { + let mut relay = Relay::default(); + let host = Arc::new(Exec::default()); + relay.set_executor(super::relay::HOST_EXECUTOR, Some(host.clone())); + let mut w = World::new(FixedDevMode::all()); + w.dev_all = true; + let make = |id: &str, caller: &str, owner: &str, system: bool| { + let mut c = call(id, super::studio::RENDER, caller); + c.app = owner.into(); + c.args = json!({"source_path":"draft.card"}); + if system { c.caller_kind = CallerKind::System; } + c + }; + for (id, caller, system) in [("s1", "os.news", false), ("s2", super::SYSTEM, true)] { + let (r, _) = reply(id); + relay.handle(Event::Call { call: make(id, caller, caller, system), reply: r }, &mut w); + } + assert_eq!(host.0.lock().unwrap().len(), if super::studio::SUPPORTED { 2 } else { 0 }); + if !super::studio::SUPPORTED { return; } + let (r, sent) = reply("cross"); + relay.handle(Event::Call { call: make("cross", "os.news", "os.mail", false), reply: r }, &mut w); + assert_eq!(sent.lock().unwrap()[0]["error"]["kind"], "not_granted"); + let mut bad = make("bad", "os.news", "os.news", false); + bad.args["workspace"] = json!("/other-app"); + let (r, sent) = reply("bad"); + relay.handle(Event::Call { call: bad, reply: r }, &mut w); + assert_eq!(sent.lock().unwrap()[0]["error"]["kind"], "invalid_args"); + w.dev_all = false; + let (r, sent) = reply("revoked"); + relay.handle(Event::Call { call: make("revoked", super::SYSTEM, super::SYSTEM, true), reply: r }, &mut w); + assert_eq!(sent.lock().unwrap()[0]["error"]["kind"], "not_granted"); + relay.handle(Event::Cancel { call_id:"s1".into(),reason:"stopped".into() }, &mut w); + assert_eq!(host.1.lock().unwrap().as_slice(), &["s1".to_string()]); + assert!(!relay.catalog.offered("os.news", false, true).iter().any(|d| d["name"] == super::studio::RENDER)); + assert!(relay.catalog.offered("os.news", true, true).iter().any(|d| d["name"] == super::studio::RENDER)); +} + +#[cfg(all(unix, any(feature="app-hub",native_mobile)))] +#[test] +fn studio_app_tools_are_own_caller_only_and_disappear_on_revocation() { + let mut relay=Relay::default(); + let host=Arc::new(Exec::default()); + relay.set_executor(super::relay::HOST_EXECUTOR,Some(host.clone())); + let mut w=World::new(FixedDevMode::all());w.dev_all=true; + for (index,name) in super::studio::APP_TOOLS.iter().enumerate(){ + let id=format!("studio-app-{index}"); + let mut c=call(&id,name,"os.news");c.app="os.news".into(); + c.args=match *name { + "studio.bundle_check"|"studio.install"|"studio.open"=>json!({"bundle_path":"planner"}), + "studio.input"=>json!({"instance_id":"one","widget_id":"button","action":"tap"}), + _=>json!({"instance_id":"one"}), + }; + let (r,_)=reply(&id); + relay.handle(Event::Call{call:c.clone(),reply:r},&mut w); + let (r,sent)=reply(&format!("cross-{index}")); + c.call_id=format!("cross-{index}");c.app="os.mail".into(); + relay.handle(Event::Call{call:c.clone(),reply:r},&mut w); + assert_eq!(sent.lock().unwrap()[0]["error"]["kind"],"not_granted"); + c.app="os.news".into();c.call_id=format!("revoked-{index}");w.dev_all=false; + let (r,sent)=reply(&c.call_id); + relay.handle(Event::Call{call:c,reply:r},&mut w); + assert_eq!(sent.lock().unwrap()[0]["error"]["kind"],"not_granted"); + w.dev_all=true; + } + assert_eq!(host.0.lock().unwrap().len(),super::studio::APP_TOOLS.len()); +} diff --git a/crates/shell/src/lib.rs b/crates/shell/src/lib.rs index 1ae2a82b..434c2635 100644 --- a/crates/shell/src/lib.rs +++ b/crates/shell/src/lib.rs @@ -73,6 +73,8 @@ pub mod wm_reply; pub mod ext; pub mod glance; pub mod glance_card; +pub mod studio; +use studio::Renderer as StudioRenderer; pub mod glance_chat; pub mod glance_digest; #[cfg(any(feature = "app-hub", native_mobile))] @@ -505,6 +507,9 @@ pub struct App { /// at without a screen (the GPU readback does not need one). #[rust] pub test_capture: Option<(Timer, std::path::PathBuf)>, + #[rust] pub studio: StudioRenderer, + #[rust] pub studio_clients: HashMap, + #[rust] pub studio_registry_generation: u64, #[rust] pub test_capture_ticket: Option, #[rust] pub test_recording: bool, /// `--test-action page:`: the home page to jump to, once the phone @@ -662,6 +667,34 @@ pub struct DividerDrag { } impl App { + #[cfg(all(unix, any(feature = "app-hub", native_mobile)))] + fn drain_studio(&mut self, cx: &mut Cx) { + for instance in studio::apps::take_closes() { + if let Some(client) = self.studio_clients.remove(&instance) { + self.close_client(cx, client, false); + } + } + for instance in studio::apps::take_launches() { + let Some((app_id, title)) = studio::apps::launch_info(&instance) else { continue }; + let app = apps::studio_row(app_id.unwrap_or_else(|| format!("studio-instance:{instance}")), + title, vec![format!("--studio-instance={instance}")]); + let client = self.next_id; + self.launch_module_as(cx, &studio::module::STUDIO_MODULE, &app); + if self.state.as_ref().is_some_and(|state| state.clients.contains_key(&client)) { + self.studio_clients.insert(instance, client); + } else { + studio::apps::shutdown(cx, &instance); + } + } + let generation = host_tools::studio_bundles::generation(); + if generation != self.studio_registry_generation { + self.studio_registry_generation = generation; + shell::launcher::invalidate_apps(); + self.redraw_all(cx); + } + } + + fn desk(&self, cx: &mut Cx) -> WidgetRef { self.ui.widget(cx, ids!(desk)) } @@ -789,6 +822,14 @@ impl App { } } + #[cfg(all(unix, any(feature = "app-hub", native_mobile)))] + if app.bin == "studio-app" { + if let Err(error) = host_tools::studio_bundles::request_launch_installed(&app.id) { + self.notify(cx, "Could not open developer app", &error); + } + return; + } + // In-process hosting (aicontrol §3): a linked module the person // switched on (or a dev `--module` run) becomes an instance in an // isolate of its own — never a process, never the pool. @@ -1960,6 +2001,7 @@ impl App { } fn remove_client(&mut self, cx: &mut Cx, client: ClientId) { + self.studio_clients.retain(|_, id| *id != client); log!("wm: removing client {}", client); peer_link::process_gone(client); self.process_close.gone(client); @@ -3466,6 +3508,7 @@ impl App { self.pane_links.reannounce(); // App peers take `dev.run` and developer grants, or lose them. crate::host_tools::developer_mode_changed(); + shell::launcher::invalidate_apps(); for notice in dev_mode::take_notices() { self.notify(cx, "Developer mode", ¬ice); } @@ -5128,22 +5171,17 @@ impl App { self.handle_event(cx, &event); } - /// The test actions' timers: a `capture:` tick writes the next frame; a - /// due `ask-appcard:` sends its text to the appcard instance's executor - /// exactly as the assistant's `ask` call would. - fn fire_test_timers(&mut self, cx: &mut Cx, te: &TimerEvent) { - if let Some((timer, path)) = self.test_capture.clone() { - if timer.is_timer(te).is_some() { - #[cfg(not(target_os = "android"))] - { - let tmp = path.with_extension("part.png"); - cx.capture_next_frame_to_file(tmp); - let _ = std::fs::rename(path.with_extension("part.png"), path); - } - #[cfg(target_os = "android")] - { - for result in cx.try_take_texture_readbacks() { - if Some(result.ticket) != self.test_capture_ticket { continue; } + /// One consumer drains the platform queue; a test capture cannot discard + /// Studio's ticket (or vice versa). Cancelled unowned tickets are drained + /// too, releasing their platform readback reservation. + fn route_texture_readbacks(&mut self, cx: &mut Cx) { + for result in cx.try_take_texture_readbacks() { + let Some(result) = self.studio.readback(cx, result) else {continue}; + #[cfg(all(unix, any(feature = "app-hub", native_mobile)))] + let Some(result) = studio::apps::readback(cx, result) else {continue}; + #[cfg(target_os = "android")] + if Some(result.ticket) == self.test_capture_ticket { + let Some((_, path)) = self.test_capture.clone() else {continue}; self.test_capture_ticket = None; let output = if self.test_recording { let timestamp = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap_or_default().as_millis(); @@ -5166,7 +5204,26 @@ impl App { if std::fs::write(&tmp,png).is_ok() { let _ = std::fs::rename(tmp,output); } } }) { Ok(task) => task.detach(), Err(error) => log!("wm: capture worker unavailable: {error}") } - } + } + #[cfg(not(target_os = "android"))] + let _ = result; + } + } + + /// The test actions' timers: a `capture:` tick writes the next frame; a + /// due `ask-appcard:` sends its text to the appcard instance's executor + /// exactly as the assistant's `ask` call would. + fn fire_test_timers(&mut self, cx: &mut Cx, te: &TimerEvent) { + if let Some((timer, _path)) = self.test_capture.clone() { + if timer.is_timer(te).is_some() { + #[cfg(not(target_os = "android"))] + { + let tmp = _path.with_extension("part.png"); + cx.capture_next_frame_to_file(tmp); + let _ = std::fs::rename(_path.with_extension("part.png"), _path); + } + #[cfg(target_os = "android")] + { if self.test_capture_ticket.is_none() { let focus = self.state_mut().layout.focused_client(); let texture = focus.and_then(|client|self.desk(cx).borrow::().and_then(|desk|desk.phone_client_texture(client))); @@ -5268,6 +5325,18 @@ impl App { i += 2; continue; } + #[cfg(unix)] + if let Some(path) = name.strip_prefix("studio-render:") { + if let Err(error) = host_tools::studio::test_action(path) { log!("studio test: {error}"); } + i += 2; + continue; + } + #[cfg(all(unix, any(feature = "app-hub", native_mobile)))] + if let Some(path) = name.strip_prefix("studio-flow:") { + if let Err(error) = host_tools::studio::test_flow(path) { log!("studio flow: {error}"); } + i += 2; + continue; + } // capture:: write the presented frame to // every 5 s — a scripted run's screen, with no display // (or a locked one) needed. @@ -6209,6 +6278,8 @@ impl MatchEvent for App { // Its tools, grants and kernel tools, as installed (ADR 0004 §7). host_tools::script_app_installed(&id); } + #[cfg(all(unix, any(feature = "app-hub", native_mobile)))] + self.drain_studio(cx); self.drain_hub(cx); self.drain_client_lines(cx); self.drain_module_upstream(); @@ -6311,6 +6382,8 @@ impl App { pub fn shell_handle_event(&mut self, cx: &mut Cx, event: &Event) { self.webview_render.handle_event(cx, event); self.shell_handle_event_inner(cx, event); + // Studio is independent of modal/phone routes that may consume an event. + if let Event::Draw(draw) = event { self.studio.draw(cx, draw); } // Whatever module panicked during this event — in its tile's event // or draw, or in a call the shell made — is contained by now; show // it closed and free it before the next event (module_host.rs). @@ -6327,6 +6400,11 @@ impl App { } fn shell_handle_event_inner(&mut self, cx: &mut Cx, event: &Event) { + #[cfg(all(unix, any(feature = "app-hub", native_mobile)))] + studio::apps::tick(cx, event); + let studio_surface = studio::surface_geometry(cx); + self.studio.event(cx, event, studio_surface); + self.route_texture_readbacks(cx); // Quitting, or closing the shell's window, asks the hosted instances // first (makepad#65). A termination signal is never refused. match event { diff --git a/crates/shell/src/studio/apps.rs b/crates/shell/src/studio/apps.rs new file mode 100644 index 00000000..b1b4c6e0 --- /dev/null +++ b/crates/shell/src/studio/apps.rs @@ -0,0 +1,1162 @@ +//! Developer app runner. Real visible Splash widgets, private app storage, and +//! Makepad's in-process instrument; no global widget-tree or remote server swap. +use super::{rgba, MAX_PNG}; +use crate::dev_mode::{self, DevTag}; +use makepad_widgets::makepad_platform::event::{ + ScrollEvent, TouchPoint, TouchState, TouchUpdateEvent, +}; +use makepad_widgets::{widget_tree::WidgetTree, *}; +use serde_json::{json, Value}; +use std::{ + cell::RefCell, + collections::{HashMap, VecDeque}, + fs::File, + io::Write, + path::PathBuf, + sync::{ + atomic::{AtomicBool, Ordering}, + mpsc::Sender, + Arc, Mutex, OnceLock, + }, + time::{Duration, Instant}, +}; + +const REQUEST_TIMEOUT: Duration = Duration::from_secs(20); +const MAX_REQUESTS: usize = 16; +const MAX_INSTANCES: usize = 4; +static FOREGROUND: AtomicBool = AtomicBool::new(true); + +/// Constructed only by the host's admitted, immutable bundle staging path. +#[derive(Clone)] +pub struct OpenSpec { + pub instance_id: String, + pub app_id: Option, + pub owner: String, + pub dev_tag: DevTag, + pub source: String, + pub jail: PathBuf, + pub persistent: bool, + pub title: String, + pub storage_quota: u64, + pub instruction_budget: u64, + pub memory_bytes: u64, +} +pub enum Action { + Ready, + Input { + widget_id: String, + kind: String, + text: Option, + delta_y: Option, + }, + Inspect { + output: File, + }, + Close, +} +pub struct Request { + pub id: String, + pub owner: String, + pub dev_tag: DevTag, + pub instance_id: String, + pub action: Action, + pub reply: Sender>, +} +struct Pending { + request: Request, + started: Instant, + cancelled: Arc, +} +#[derive(Default)] +struct Queue { + specs: HashMap, + launches: VecDeque, + closes: VecDeque, + requests: VecDeque, + tokens: HashMap>, + request_instances: HashMap, + failures: HashMap, +} +fn queue() -> &'static Mutex { + static QUEUE: OnceLock> = OnceLock::new(); + QUEUE.get_or_init(Default::default) +} +fn granted(owner: &str, tag: &DevTag) -> bool { + let identity: Option = serde_json::from_str(owner).ok(); + let app = identity + .as_ref() + .and_then(|v| v["app"].as_str()) + .unwrap_or(owner); + dev_mode::tag_valid(tag) && dev_mode::grants_all(crate::host_tools::app_of_peer(app)) +} +fn valid(p: &Pending) -> Result<(), String> { + if !FOREGROUND.load(Ordering::Acquire) { + return Err("not_foreground".into()); + } + if p.cancelled.load(Ordering::Acquire) { + return Err("studio_cancelled".into()); + } + if !granted(&p.request.owner, &p.request.dev_tag) { + return Err("studio_grant_expired".into()); + } + if p.started.elapsed() >= REQUEST_TIMEOUT { + return Err("studio_timeout".into()); + } + Ok(()) +} +fn finish(p: Pending, result: Result) { + let id = p.request.id.clone(); + let _ = p.request.reply.send(result); + let mut q = queue().lock().unwrap_or_else(|e| e.into_inner()); + q.tokens.remove(&id); + q.request_instances.remove(&id); +} +/// A close acknowledgement is a cleanup barrier for a caller reopening the app. +fn finish_after_cleanup(p: Pending, cleanup: impl FnOnce()) { + cleanup(); + finish(p, Ok(json!({"closed":true}))); +} +pub fn stage_open(spec: OpenSpec) -> Result<(), String> { + if !FOREGROUND.load(Ordering::Acquire) { + return Err("not_foreground".into()); + } + if !granted(&spec.owner, &spec.dev_tag) { + return Err("studio_grant_expired".into()); + } + if spec.source.len() > 256 * 1024 || spec.source.trim().is_empty() { + return Err("studio_source_size".into()); + } + if spec.storage_quota > 1024 * 1024 + || spec.instruction_budget == 0 + || spec.instruction_budget > 5_000_000 + || spec.memory_bytes == 0 + || spec.memory_bytes > 16 * 1024 * 1024 + { + return Err("studio_resource_limits".into()); + } + // Initial script-app support deliberately has no external resource route. + // This is also enforced by the isolate; this check makes refusal explicit. + if spec.source.contains("{{assets}}") || spec.source.contains("http_resource(") { + return Err("studio_bundle_assets_unsupported".into()); + } + let mut q = queue().lock().unwrap_or_else(|e| e.into_inner()); + if q.specs.len() >= MAX_INSTANCES || q.specs.contains_key(&spec.instance_id) { + return Err("studio_busy".into()); + } + if spec.app_id.as_ref().is_some_and(|id| { + q.specs + .values() + .any(|open| open.app_id.as_ref() == Some(id)) + }) { + return Err("studio_app_already_open".into()); + } + q.launches.push_back(spec.instance_id.clone()); + q.specs.insert(spec.instance_id.clone(), spec); + SignalToUI::set_ui_signal(); + Ok(()) +} +pub fn take_launches() -> Vec { + queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .launches + .drain(..) + .collect() +} +pub fn take_closes() -> Vec { + queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .closes + .drain(..) + .collect() +} +pub fn launch_info(instance_id: &str) -> Option<(Option, String)> { + queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .specs + .get(instance_id) + .map(|s| (s.app_id.clone(), s.title.clone())) +} +pub fn submit(request: Request) -> Result<(), String> { + if !FOREGROUND.load(Ordering::Acquire) { + return Err("not_foreground".into()); + } + if !granted(&request.owner, &request.dev_tag) { + return Err("studio_grant_expired".into()); + } + let mut q = queue().lock().unwrap_or_else(|e| e.into_inner()); + if !q.specs.contains_key(&request.instance_id) { + if let Some((owner, tag, message)) = q.failures.get(&request.instance_id) { + if owner == &request.owner && tag == &request.dev_tag { + return Err(message.clone()); + } + } + return Err("studio_instance_not_found".into()); + } + let spec = &q.specs[&request.instance_id]; + if spec.owner != request.owner || spec.dev_tag != request.dev_tag { + return Err("studio_instance_scope".into()); + } + if q.tokens.len() >= MAX_REQUESTS || q.tokens.contains_key(&request.id) { + return Err("studio_busy".into()); + } + let token = Arc::new(AtomicBool::new(false)); + q.tokens.insert(request.id.clone(), token.clone()); + q.request_instances + .insert(request.id.clone(), request.instance_id.clone()); + q.requests.push_back(Pending { + request, + started: Instant::now(), + cancelled: token, + }); + SignalToUI::set_ui_signal(); + Ok(()) +} +/// Roll back an open that never reached its caller. Cleanup does not require a +/// still-valid grant, but the original owner and generation must match. +pub fn abandon(instance_id: &str, owner: &str, tag: &DevTag) -> Result<(), String> { + let mut q = queue().lock().unwrap_or_else(|e| e.into_inner()); + let Some(spec) = q.specs.get(instance_id) else { + return Ok(()); + }; + if spec.owner != owner || &spec.dev_tag != tag { + return Err("studio_instance_scope".into()); + } + if q.launches.iter().any(|id| id == instance_id) { + let spec = q.specs.remove(instance_id).unwrap(); + q.launches.retain(|id| id != instance_id); + drop(q); + if !spec.persistent { + let _ = std::fs::remove_dir_all(&spec.jail); + } + } else { + if !q.closes.iter().any(|id| id == instance_id) { + q.closes.push_back(instance_id.into()); + } + for (id, instance) in &q.request_instances { + if instance == instance_id { + if let Some(token) = q.tokens.get(id) { + token.store(true, Ordering::Release); + } + } + } + } + SignalToUI::set_ui_signal(); + Ok(()) +} +pub fn cancel(id: &str) { + if let Some(token) = queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .tokens + .get(id) + { + token.store(true, Ordering::Release); + } + SignalToUI::set_ui_signal(); +} +thread_local! { + static ROOTS: RefCell> = RefCell::new(HashMap::new()); + static CAPTURES: RefCell> = const { RefCell::new(Vec::new()) }; +} +struct Capture { + pending: Pending, + snapshot: Value, + frozen: makepad_widgets::view::ViewTextureSnapshot, + ticket: ReadbackTicket, +} + +script_mod! { + use mod.prelude.widgets.* + mod.widgets.StudioApp = set_type_default() do #(StudioApp::register_widget(vm)) { + width: Fill height: Fill flow: Overlay + capture := CachedView { + width: Fill height: Fill flow: Overlay + card := Splash { width: Fill height: Fill } + } + } +} +#[derive(Script, ScriptHook, Widget)] +pub struct StudioApp { + #[deref] + view: View, + #[rust] + spec: Option, + #[rust] + started: bool, + #[rust] + draws: usize, + #[rust] + timer: Timer, + #[rust] + pending: Option, + #[rust] + text_next: Option, + #[rust] + input_draw_pending: bool, +} +pub fn create(vm: &mut ScriptVm, instance_id: &str) -> WidgetRef { + let spec = queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .specs + .get(instance_id) + .cloned(); + let value = script_eval!(vm,{use mod.widgets.* StudioApp{}}); + let root = WidgetRef::script_from_value(vm, value); + if let Some(mut app) = root.borrow_mut::() { + app.spec = spec; + } + ROOTS.with(|roots| roots.borrow_mut().insert(instance_id.into(), root.clone())); + root +} +pub fn shutdown(cx: &mut Cx, instance_id: &str) { + let root = ROOTS.with(|roots| roots.borrow_mut().remove(instance_id)); + if let Some(root) = root { + if let Some(mut app) = root.borrow_mut::() { + app.stop(cx); + } + } + let mut q = queue().lock().unwrap_or_else(|e| e.into_inner()); + q.specs.remove(instance_id); + q.launches.retain(|id| id != instance_id); + let mut rejected = Vec::new(); + let mut keep = VecDeque::new(); + for p in q.requests.drain(..) { + if p.request.instance_id == instance_id { + rejected.push(p) + } else { + keep.push_back(p) + } + } + q.requests = keep; + drop(q); + for p in rejected { + finish(p, Err("studio_closed".into())); + } + CAPTURES.with(|captures| { + for c in captures + .borrow() + .iter() + .filter(|c| c.pending.request.instance_id == instance_id) + { + c.pending.cancelled.store(true, Ordering::Release); + cx.cancel_texture_readback(c.ticket); + } + }); +} +/// Called before shell event routes. Readback workers also keep this token. +pub fn tick(cx: &mut Cx, event: &Event) { + match event { + Event::Pause | Event::Background => FOREGROUND.store(false, Ordering::Release), + Event::Resume | Event::Foreground => FOREGROUND.store(true, Ordering::Release), + _ => {} + } + let background = matches!(event, Event::Pause | Event::Background); + if background { + for token in queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .tokens + .values() + { + token.store(true, Ordering::Release); + } + } + let expired: Vec = queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .specs + .iter() + .filter(|(_, s)| !granted(&s.owner, &s.dev_tag)) + .map(|(id, _)| id.clone()) + .collect(); + for id in expired { + shutdown(cx, &id); + } + let mut expired_requests = Vec::new(); + { + let mut q = queue().lock().unwrap_or_else(|e| e.into_inner()); + let mut keep = VecDeque::new(); + for p in q.requests.drain(..) { + if let Err(error) = valid(&p) { + expired_requests.push((p, error)); + } else { + keep.push_back(p); + } + } + q.requests = keep; + } + for (p, error) in expired_requests { + finish(p, Err(error)); + } + CAPTURES.with(|captures| { + for c in captures.borrow().iter() { + if valid(&c.pending).is_err() { + cx.cancel_texture_readback(c.ticket); + } + } + }); +} +impl StudioApp { + fn stop(&mut self, cx: &mut Cx) { + self.view.splash(cx, ids!(card)).set_text(cx, ""); + cx.stop_timer(self.timer); + if let Some(p) = self.pending.take() { + finish(p, Err("studio_closed".into())); + } + if let Some(spec) = self.spec.take() { + if !spec.persistent { + let _ = std::fs::remove_dir_all(&spec.jail); + } + let mut q = queue().lock().unwrap_or_else(|e| e.into_inner()); + q.specs.remove(&spec.instance_id); + for (id, instance) in &q.request_instances { + if instance == &spec.instance_id { + if let Some(token) = q.tokens.get(id) { + token.store(true, Ordering::Release); + } + } + } + if !q.closes.contains(&spec.instance_id) { + q.closes.push_back(spec.instance_id.clone()); + } + let mut rejected = Vec::new(); + let mut keep = VecDeque::new(); + for p in q.requests.drain(..) { + if p.request.instance_id == spec.instance_id { + rejected.push(p); + } else { + keep.push_back(p); + } + } + q.requests = keep; + drop(q); + for p in rejected { + let error = queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .failures + .get(&spec.instance_id) + .map(|(_, _, e)| e.clone()) + .unwrap_or_else(|| "studio_closed".into()); + finish(p, Err(error)); + } + SignalToUI::set_ui_signal(); + } + self.view.visible = false; + self.view.redraw(cx); + } + fn start(&mut self, cx: &mut Cx) -> Result<(), String> { + let spec = self.spec.as_ref().ok_or("studio_instance_not_found")?; + if !granted(&spec.owner, &spec.dev_tag) { + return Err("studio_grant_expired".into()); + } + if !spec.jail.is_dir() { + return Err("studio_storage_missing".into()); + } + crate::glance_card::ensure_vocabulary(cx); + let card = self.view.splash(cx, ids!(card)); + if let Some(mut s) = card.borrow_mut() { + s.set_allow_net(false); + s.set_debug_name(&format!("studio/{}/main.splash", spec.instance_id)); + } + card.set_sandbox_dir(cx, Some(spec.jail.clone())); + card.set_storage_quota(cx, Some(spec.storage_quota)); + card.set_host_caps(cx, Vec::new()); + card.set_host_prompts(cx, false); + card.set_policy(cx, Some(Vec::new()), Some(spec.instruction_budget)); + card.set_memory_bytes(cx, Some(spec.memory_bytes as usize)); + card.set_text(cx, &spec.source); + let mut children = 0; + if let Some(s) = card.borrow() { + s.children(&mut |_, _| children += 1); + } + let running = card + .borrow_mut() + .and_then(|mut s| s.isolate_heap_key(cx)) + .is_some_and(makepad_widgets::splash_policy::may_run); + if children == 0 || !running { + return Err(format!("studio_eval_failed: {}/main.splash produced no root content or exceeded its admitted instruction budget",spec.instance_id)); + } + self.timer = cx.start_interval(0.08); + Ok(()) + } + fn dispatch(&mut self, cx: &mut Cx, event: &Event) { + let card = self.view.splash(cx, ids!(card)); + let vm = card + .borrow() + .and_then(|s| cx.script_ref_vm_id(&s.view.source)); + if let Some(vm) = vm { + widget_async::with_isolate(cx, vm, |cx| { + self.view.handle_event(cx, event, &mut Scope::empty()) + }); + } + } + fn instrument(&self, cx: &mut Cx) -> Value { + let tree = WidgetTree::default(); + let card = self.view.widget(cx, ids!(card)); + tree.set_root_widget(card.clone()); + let mut names: HashMap = HashMap::new(); + let widgets: Vec = tree + .snapshot(cx) + .into_iter() + .map(|w| { + let occurrence = names.entry(w.id.clone()).or_default(); + let selector = format!("{}@{}", w.id, *occurrence); + *occurrence += 1; + // Splash.text() is source code, not painted UI text. Never + // let source literals satisfy a functional visible-text check. + let text = if w.widget_type == "Splash" { + None + } else { + w.text + }; + let value = if w.widget_type == "Splash" { + None + } else { + w.value + }; + json!({ + "id":w.id,"selector":selector,"type":w.widget_type,"rect":[w.x,w.y,w.width,w.height], + "visible":w.visible,"enabled":w.enabled,"text":text,"value":value, + "checked":w.checked,"selected":w.selected}) + }) + .collect(); + let mut geometry: Value = + serde_json::from_str(&tree.geometry_json(cx)).unwrap_or(Value::Null); + // ScrollYView is a View template, so its Rust type alone cannot tell + // intentional viewport clipping from a layout bug. Read its inherited + // scroll_bars setting in its own VM without evaluating any app code. + let rows = tree.flat_tree(cx); + let mut name_counts = HashMap::::new(); + for row in &rows { + *name_counts.entry(row.name.clone()).or_default() += 1; + } + let vm = card + .borrow::() + .and_then(|s| cx.script_ref_vm_id(&s.view.source)); + let mut scroll_y = Vec::new(); + if let Some(vm) = vm { + widget_async::with_isolate(cx, vm, |cx| { + cx.with_vm(|vm| { + for row in &rows { + if name_counts.get(&row.name) != Some(&1) { + continue; + } + let source = tree.widget(WidgetUid(row.uid)).script_source(); + if source == ScriptObject::ZERO { + continue; + } + if let Some(bars) = vm + .bx + .heap + .value(source, id!(scroll_bars).into(), NoTrap) + .as_object() + { + if vm + .bx + .heap + .value(bars, id!(show_scroll_y).into(), NoTrap) + .as_bool() + == Some(true) + { + scroll_y.push(row.name.clone()); + } + } + } + }) + }); + } + if let Some(nodes) = geometry["widgets"].as_array_mut() { + for node in nodes { + let is_scroll = node["id"] + .as_str() + .is_some_and(|name| scroll_y.iter().any(|id| id == name)); + if is_scroll { + node["scroll_y"] = json!(true); + } + } + } + let checks = check_geometry(&geometry); + json!({"widgets":widgets,"geometry":geometry,"tree":tree.compact_dump(cx),"checks":checks}) + } + fn target(&self, cx: &mut Cx, id: &str) -> Result { + let snapshot = self.instrument(cx); + let widgets = snapshot["widgets"] + .as_array() + .ok_or("studio_snapshot_failed")?; + let candidate = select_widget(widgets, id)?; + let r = candidate["rect"].as_array().ok_or("studio_widget_rect")?; + let n = |i: usize| r[i].as_f64().unwrap_or(0.0); + if n(2) <= 0.0 || n(3) <= 0.0 { + return Err("studio_widget_not_drawn".into()); + } + let abs = dvec2(n(0) + n(2) * 0.5, n(1) + n(3) * 0.5); + Ok(abs) + } + fn redraw_after_input(&mut self, cx: &mut Cx) { + // A redraw request only queues Draw; the old cache can still be clean. + // Inspect must wait until the changed widget state has been recorded. + self.input_draw_pending = true; + self.view.redraw(cx); + } + fn scroll(&mut self, cx: &mut Cx, id: &str, dy: f64) -> Result<(), String> { + let abs = self.target(cx, id)?; + let window_id = cx.windows.id_iter().next().ok_or("studio_no_window")?; + self.dispatch( + cx, + &Event::Scroll(ScrollEvent { + window_id, + abs, + scroll: dvec2(0., dy), + modifiers: Default::default(), + is_mouse: false, + time: cx.seconds_since_app_start(), + handled_x: Default::default(), + handled_y: Default::default(), + phase: Default::default(), + }), + ); + self.redraw_after_input(cx); + Ok(()) + } + fn tap(&mut self, cx: &mut Cx, id: &str) -> Result<(), String> { + let abs = self.target(cx, id)?; + let window_id = cx.windows.id_iter().next().ok_or("studio_no_window")?; + // Same TouchUpdate hit/handler path as physical input, scoped to this + // visible app. Keep synthetic pointer capture out of the live fingers. + let fingers = std::mem::take(&mut cx.fingers); + for (offset, state) in [(0.0, TouchState::Start), (0.03, TouchState::Stop)] { + let time = cx.seconds_since_app_start() + offset; + self.dispatch( + cx, + &Event::TouchUpdate(TouchUpdateEvent { + time, + window_id, + modifiers: Default::default(), + touches: vec![TouchPoint { + uid: 0x53545544, + state, + abs, + time, + rotation_angle: 0.0, + force: 1.0, + radius: dvec2(1., 1.), + handled: Default::default(), + sweep_lock: Default::default(), + }], + }), + ); + } + cx.fingers = fingers; + self.redraw_after_input(cx); + Ok(()) + } + fn pump(&mut self, cx: &mut Cx) { + let Some(spec) = self.spec.as_ref() else { + return; + }; + if self.pending.is_none() { + let mut q = queue().lock().unwrap_or_else(|e| e.into_inner()); + if let Some(i) = q + .requests + .iter() + .position(|p| p.request.instance_id == spec.instance_id) + { + self.pending = q.requests.remove(i); + } + } + let Some(p) = self.pending.take() else { return }; + if let Err(e) = valid(&p) { + self.text_next = None; + finish(p, Err(e)); + return; + } + if self.draws < 2 { + self.pending = Some(p); + self.view.redraw(cx); + return; + } + if let Some(text) = self.text_next.take() { + self.dispatch( + cx, + &Event::TextInput(TextInputEvent { + input: text, + replace_last: false, + was_paste: true, + ..Default::default() + }), + ); + self.redraw_after_input(cx); + finish(p, Ok(json!({"ok":true}))); + return; + } + match &p.request.action { + Action::Ready => finish(p, Ok(json!({"instance_id":spec.instance_id,"ready":true}))), + Action::Input { + widget_id, + kind, + text, + delta_y, + } => { + if kind != "tap" && kind != "text" && kind != "scroll" { + finish(p, Err("studio_input_kind".into())); + return; + } + if kind == "text" && text.as_ref().is_none_or(|t| t.len() > 4096) { + finish(p, Err("studio_input_text".into())); + return; + } + let follow = (kind == "text").then(|| text.clone().unwrap()); + let result = if kind == "scroll" { + match delta_y { + Some(dy) if dy.is_finite() && dy.abs() <= 2000.0 => { + self.scroll(cx, widget_id, *dy) + } + _ => Err("studio_input_scroll".into()), + } + } else { + self.tap(cx, widget_id) + }; + match result { + Err(e) => finish(p, Err(e)), + Ok(()) => { + if let Some(text) = follow { + self.text_next = Some(text); + self.pending = Some(p); + } else { + finish(p, Ok(json!({"ok":true}))); + } + } + } + } + Action::Inspect { .. } => { + if self.input_draw_pending { + // Keep the original request/deadline. The UI remains free + // to draw; valid() above cancels a stalled/background app. + self.pending = Some(p); + self.view.redraw(cx); + return; + } + let snapshot = self.instrument(cx); + if snapshot.to_string().len() > 1024 * 1024 { + finish(p, Err("studio_snapshot_too_large".into())); + return; + } + let frozen = self + .view + .view(cx, ids!(capture)) + .borrow_mut() + .and_then(|mut view| view.take_texture_snapshot(cx)); + if let Some(frozen) = frozen { + match frozen + .texture() + .read_back(cx, ReadbackRequest { next_render: false }) + { + Ok(ticket) => CAPTURES.with(|c| { + c.borrow_mut().push(Capture { + pending: p, + snapshot, + frozen, + ticket, + }) + }), + Err(e) => finish(p, Err(format!("studio_readback: {e}"))), + } + self.view.redraw(cx); + } else { + self.pending = Some(p); + self.view.redraw(cx); + } + } + Action::Close => { + finish_after_cleanup(p, || self.stop(cx)); + } + } + } +} +impl Widget for StudioApp { + fn handle_event(&mut self, cx: &mut Cx, event: &Event, _scope: &mut Scope) { + if !self.started { + self.started = true; + if let Err(e) = self.start(cx) { + let id = self + .spec + .as_ref() + .map(|s| s.instance_id.clone()) + .unwrap_or_default(); + error!("studio app {id}: {e}"); + if let Some(spec) = self.spec.as_ref() { + let mut q = queue().lock().unwrap_or_else(|e| e.into_inner()); + if q.failures.len() >= MAX_REQUESTS { + if let Some(old) = q.failures.keys().next().cloned() { + q.failures.remove(&old); + } + } + q.failures + .insert(id, (spec.owner.clone(), spec.dev_tag.clone(), e)); + } + self.stop(cx); + return; + } + } + if self.spec.is_none() { + return; + } + self.dispatch(cx, event); + self.pump(cx); + } + fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep { + let card = self.view.splash(cx, ids!(card)); + let vm = card + .borrow() + .and_then(|s| cx.script_ref_vm_id(&s.view.source)); + let result = if let Some(vm) = vm { + widget_async::with_isolate(cx, vm, |cx| self.view.draw_walk(cx, scope, walk)) + } else { + self.view.draw_walk(cx, scope, walk) + }; + if result.is_done() { + self.draws += 1; + self.input_draw_pending = false; + } + result + } +} +fn select_widget<'a>(widgets: &'a [Value], id: &str) -> Result<&'a Value, String> { + let mut matches = widgets.iter().filter(|w| { + (w["id"].as_str() == Some(id) || w["selector"].as_str() == Some(id)) + && w["visible"] == true + && w["enabled"] == true + }); + let first = matches.next().ok_or("studio_widget_missing_or_ambiguous")?; + if matches.next().is_some() { + return Err("studio_widget_missing_or_ambiguous".into()); + } + Ok(first) +} +/// Pure measured checks on Makepad's actual ink/clip rectangles; no model score. +fn check_geometry(geometry: &Value) -> Value { + let mut findings = Vec::new(); + if geometry["widgets"] + .as_array() + .is_none_or(|nodes| nodes.is_empty()) + { + findings.push(json!({"code":"empty_app","severity":"error"})); + } + if let Some(nodes) = geometry["widgets"].as_array() { + let by_index: HashMap = nodes + .iter() + .filter_map(|node| node["i"].as_i64().map(|i| (i, node))) + .collect(); + for node in nodes { + let n = |key: &str| node[key].as_f64().unwrap_or(0.0); + let kind = node["kind"].as_str().unwrap_or(""); + if (kind.contains("Label") || kind.contains("TextInput")) + && node["text"].as_str().is_some_and(|s| !s.is_empty()) + && (n("cw") + 1. < n("w") || n("ch") + 1. < n("h")) + && !scroll_explains_clip(node, &by_index) + { + findings.push(json!({"code":"text_clipped","severity":"error","id":node["id"]})); + } + if kind.contains("Button") && (n("w") < 44. || n("h") < 44.) { + findings + .push(json!({"code":"small_tap_target","severity":"warning","id":node["id"]})); + } + } + } + json!({"pass":!findings.iter().any(|f|f["severity"]=="error"),"findings":findings,"instrument":"makepad_widget_tree"}) +} +fn scroll_explains_clip(node: &Value, by_index: &HashMap) -> bool { + let n = |key: &str| node[key].as_f64().unwrap_or(0.0); + // A vertical scroll viewport never excuses horizontal truncation. + if n("cw") + 1.0 < n("w") && n("ch") > 0.0 { + return false; + } + let mut at = node["parent"].as_i64(); + for _ in 0..64 { + let Some(parent) = at.and_then(|i| by_index.get(&i).copied()) else { + return false; + }; + if parent["scroll_y"] == true { + let top = parent["cy"].as_f64().unwrap_or(0.0); + let bottom = top + parent["ch"].as_f64().unwrap_or(0.0); + let expected_top = n("y").max(top); + let expected_bottom = (n("y") + n("h")).min(bottom); + let expected_height = (expected_bottom - expected_top).max(0.0); + if (expected_height - n("ch")).abs() <= 1.0 + && (expected_height == 0.0 || (expected_top - n("cy")).abs() <= 1.0) + { + return true; + } + } + at = parent["parent"].as_i64(); + } + false +} +pub fn readback(cx: &mut Cx, result: TextureReadback) -> Option { + let capture = CAPTURES.with(|all| { + let mut all = all.borrow_mut(); + all.iter() + .position(|c| c.ticket == result.ticket) + .map(|i| all.remove(i)) + }); + let Some(capture) = capture else { + return Some(result); + }; + let bytes = match valid(&capture.pending).and_then(|_| rgba(&result, false)) { + Ok(bytes) => bytes, + Err(e) => { + finish(capture.pending, Err(e)); + return None; + } + }; + let reply = capture.pending.request.reply.clone(); + let id = capture.pending.request.id.clone(); + // `frozen` pins the exact app framebuffer until readback completion above. + drop(capture.frozen); + let task=cx.task_pool().submit(Lane::Heavy,move||{ + let mut p=capture.pending; + let outcome=(||{ + valid(&p)?; + let png=Cx::encode_rgba_as_png(result.width as u32,result.height as u32,&bytes).map_err(|e|format!("studio_png: {e:?}"))?; + if png.len()>MAX_PNG{return Err("studio_png_too_large".into());} + valid(&p)?; + let Action::Inspect{output}=&mut p.request.action else{return Err("studio_capture_kind".into());}; + output.write_all(&png).and_then(|_|output.flush()).map_err(|e|e.to_string())?; + valid(&p)?; + Ok(json!({"instance_id":p.request.instance_id,"width":result.width,"height":result.height,"settled":false,"snapshot":capture.snapshot})) + })();finish(p,outcome); + }); + match task { + Ok(task) => task.detach(), + Err(e) => { + let _ = reply.send(Err(format!("studio_worker: {e}"))); + let mut q = queue().lock().unwrap_or_else(|e| e.into_inner()); + q.tokens.remove(&id); + q.request_instances.remove(&id); + } + } + None +} +#[cfg(test)] +mod tests { + use super::*; + /// Records the actual StudioApp widget through its cached-view draw path. + /// This catches incompatible View flags and stack imbalance; GPU pixels + /// remain the separate device acceptance test. + #[test] + fn studio_widget_records_a_cached_frame_without_background_or_stack_conflicts() { + let mut cx = Cx::new(Box::new(|_, _| {})); + let root = cx.with_vm(|vm| { + makepad_widgets::script_mod(vm); + super::script_mod(vm); + let value = script_eval!(vm,{use mod.widgets.* StudioApp{}}); + WidgetRef::script_from_value(vm, value) + }); + root.splash(&mut cx, ids!(card)) + .set_text(&mut cx, "Label{text: \"Studio render regression\"}"); + let cache = root.view(&mut cx, ids!(capture)); + assert!(cache.borrow().unwrap().texture_caching); + assert!(!cache.borrow().unwrap().show_bg); + record_frame(&mut cx, &root); + } + #[test] + fn input_redraw_cannot_reuse_the_previous_completed_frame() { + let mut cx = Cx::new(Box::new(|_, _| {})); + let root = cx.with_vm(|vm| { + makepad_widgets::script_mod(vm); + super::script_mod(vm); + let value = script_eval!(vm,{use mod.widgets.* StudioApp{}}); + WidgetRef::script_from_value(vm, value) + }); + root.splash(&mut cx, ids!(card)) + .set_text(&mut cx, "Label{text: \"Before input\"}"); + record_frame(&mut cx, &root); + let previous_draws = root.borrow::().unwrap().draws; + { + let mut app = root.borrow_mut::().unwrap(); + // This is the production invalidation used by tap/scroll and the + // second (text delivery) event. A queued redraw is not a frame. + app.redraw_after_input(&mut cx); + assert!(app.input_draw_pending); + assert_eq!(app.draws, previous_draws); + app.redraw_after_input(&mut cx); + assert!(app.input_draw_pending); + } + record_frame(&mut cx, &root); + let app = root.borrow::().unwrap(); + assert!(app.draws > previous_draws); + assert!(!app.input_draw_pending); + // GPU completion is still checked separately by take_texture_snapshot; + // this CPU test does not claim the recorded frame has been painted. + } + #[test] + fn close_acknowledgement_is_sent_only_after_cleanup() { + let (tx, rx) = std::sync::mpsc::channel(); + let pending = Pending { + request: Request { + id: format!("close-order-{}", uuid::Uuid::new_v4()), + instance_id: "test-instance".into(), + owner: "system".into(), + dev_tag: DevTag { + profile_id: "test-only".into(), + since: 0, + }, + action: Action::Close, + reply: tx, + }, + started: Instant::now(), + cancelled: Arc::new(AtomicBool::new(false)), + }; + let mut disposed = false; + finish_after_cleanup(pending, || { + assert!(matches!( + rx.try_recv(), + Err(std::sync::mpsc::TryRecvError::Empty) + )); + disposed = true; + }); + assert!(disposed); + assert_eq!(rx.try_recv().unwrap().unwrap(), json!({"closed":true})); + } + fn record_frame(cx: &mut Cx, root: &WidgetRef) { + let pass = DrawPass::new(cx); + pass.set_size(cx, dvec2(390., 600.)); + let mut list = DrawList2d::new(cx); + let event = DrawEvent::default(); + let mut draw = CxDraw::new(cx, &event); + let mut cx = Cx2d::new(&mut draw); + cx.begin_pass(&pass, Some(1.)); + list.begin_always(&mut cx); + cx.begin_root_turtle(dvec2(390., 600.), Layout::flow_overlay()); + let mark = cx.unwind_mark(); + root.draw_walk_all(&mut cx, &mut Scope::empty(), Walk::fixed(390., 600.)); + assert!(cx.unwind_mark().is_balanced_with(&mark)); + cx.end_pass_sized_turtle(); + list.end(&mut cx); + cx.end_pass(&pass); + } + #[test] + fn production_shutdown_parks_the_vm_reclaims_scratch_and_allows_next_app_draw() { + use makepad_app_module::{ + AppModule, InstanceHandles, InstanceScope, ModuleWindows, ReplySink, Viewport, + }; + let instance = format!("shutdown-test-{}", uuid::Uuid::new_v4()); + let jail = std::env::temp_dir().join(&instance); + std::fs::create_dir(&jail).unwrap(); + let spec = OpenSpec { + instance_id: instance.clone(), + app_id: None, + owner: "system".into(), + dev_tag: DevTag { + profile_id: "test-only".into(), + since: 0, + }, + source: String::new(), + jail: jail.clone(), + persistent: false, + title: "test".into(), + storage_quota: 0, + instruction_budget: 5_000_000, + memory_bytes: 16 * 1024 * 1024, + }; + // No grant is minted: the test seats an internal spec solely to exercise + // the production factory/shutdown boundary and disposal of its jail. + queue().lock().unwrap().specs.insert(instance.clone(), spec); + let mut cx = Cx::new(Box::new(|_, _| {})); + let storage = cx.storage(&instance); + let parts = cx.with_vm(|vm| { + makepad_widgets::script_mod(vm); + let module = &super::super::module::STUDIO_MODULE; + module.register(vm); + let open = module + .open_schema() + .validate(&json!({"instance_id":instance}).to_string(), &[]) + .unwrap(); + module.create( + vm, + open, + InstanceHandles { + scope: InstanceScope::new(1, 1), + storage, + viewport: Viewport { + size: dvec2(390., 600.), + }, + replies: ReplySink::pair().0, + windows: ModuleWindows::new(false), + }, + ) + }); + let card = parts.root.splash(&mut cx, ids!(card)); + card.set_sandbox_dir(&mut cx, Some(jail.clone())); + card.set_text(&mut cx, "Label{text: \"Before shutdown\"}"); + record_frame(&mut cx, &parts.root); + let shutdown = parts.shutdown; + // Invoke the actual FnOnce while Cx's VM is taken, exactly the boundary + // that used to panic. No ModuleHost catch can turn a panic into success. + cx.with_vm(|vm| shutdown(vm)); + assert!(!jail.exists()); + assert!(!ROOTS.with(|roots| roots.borrow().contains_key(&instance))); + assert!(!queue().lock().unwrap().specs.contains_key(&instance)); + assert!(parts.root.borrow::().unwrap().spec.is_none()); + let next = cx.with_vm(|vm| { + let value = script_eval!(vm,{use mod.widgets.* StudioApp{}}); + WidgetRef::script_from_value(vm, value) + }); + next.splash(&mut cx, ids!(card)) + .set_text(&mut cx, "Label{text: \"After shutdown\"}"); + record_frame(&mut cx, &next); + queue().lock().unwrap().closes.retain(|id| id != &instance); + } + #[test] + fn scoped_selectors_refuse_ambiguous_hidden_and_disabled_targets() { + let rows = vec![ + json!({"id":"complete","selector":"complete@0","visible":true,"enabled":true}), + json!({"id":"complete","selector":"complete@1","visible":true,"enabled":true}), + json!({"id":"delete","selector":"delete@0","visible":false,"enabled":true}), + json!({"id":"blocked","selector":"blocked@0","visible":true,"enabled":false}), + ]; + assert!(select_widget(&rows, "complete").is_err()); + assert_eq!(select_widget(&rows, "complete@1").unwrap(), &rows[1]); + assert!(select_widget(&rows, "delete@0").is_err()); + assert!(select_widget(&rows, "blocked@0").is_err()); + assert!(select_widget(&rows, "outside_instance").is_err()); + } + #[test] + fn missing_geometry_never_passes_inspection() { + assert_eq!(check_geometry(&json!({"widgets":[]}))["pass"], false); + } + #[test] + fn scroll_viewport_clipping_is_distinct_from_a_short_row_or_horizontal_overflow() { + let mut report = json!({"widgets":[ + {"i":0,"parent":-1,"id":"list","kind":"View","scroll_y":true,"cy":0,"ch":100}, + {"i":1,"parent":0,"id":"row","kind":"Label","text":"task","y":90,"w":100,"h":20,"cy":90,"cw":100,"ch":10}]}); + assert_eq!(check_geometry(&report)["pass"], true); + report["widgets"][1]["ch"] = json!(5); + assert_eq!(check_geometry(&report)["pass"], false); + report["widgets"][1]["ch"] = json!(10); + report["widgets"][1]["cw"] = json!(50); + assert_eq!(check_geometry(&report)["pass"], false); + } + #[test] + fn instrument_checks_report_real_clipping_and_small_controls() { + let report = check_geometry(&json!({"widgets":[ + {"id":"title","kind":"Label","text":"Tasks","w":100,"h":20,"cw":40,"ch":20}, + {"id":"add","kind":"Button","w":30,"h":30,"cw":30,"ch":30}]})); + assert_eq!(report["pass"], false); + assert_eq!(report["findings"].as_array().unwrap().len(), 2); + } + #[test] + fn empty_or_complete_text_is_not_clipped() { + assert_eq!( + check_geometry( + &json!({"widgets":[{"id":"title","kind":"Label","text":"Tasks","w":100,"h":20,"cw":100,"ch":20}]}) + )["pass"], + true + ); + } +} diff --git a/crates/shell/src/studio/mod.rs b/crates/shell/src/studio/mod.rs new file mode 100644 index 00000000..fb4e5638 --- /dev/null +++ b/crates/shell/src/studio/mod.rs @@ -0,0 +1,725 @@ +//! Foreground-only, unpublished L0 previews. No app policy or live app storage +//! is inherited: each render receives a disposable, zero-quota isolate. +#[cfg(all(unix, any(feature = "app-hub", native_mobile)))] +pub mod apps; +#[cfg(all(unix, any(feature = "app-hub", native_mobile)))] +pub mod module; +use crate::dev_mode::DevTag; +use makepad_widgets::*; +use serde_json::Value; +use std::{ + collections::{HashMap, VecDeque}, + fs::File, + io::Write, + path::PathBuf, + sync::{ + atomic::{AtomicBool, Ordering}, + mpsc::Sender, + Arc, Mutex, OnceLock, + }, + time::{Duration, Instant}, +}; + +const DEADLINE: Duration = Duration::from_secs(20); +const MAX_JOBS: usize = 4; +const MAX_PNG: usize = 5 * 1024 * 1024; + +pub struct RenderJob { + pub id: String, + pub app: String, + pub source: String, + pub data: Value, + pub dark: bool, + /// None uses the current surface's glance width. Reserved for host tests. + pub width: Option, + pub output: File, + pub reply: Sender>, + pub dev_tag: DevTag, +} +#[derive(Debug)] +pub struct RenderResult { + pub width: u32, + pub height: u32, + pub settled: bool, +} +struct Pending { + job: RenderJob, + cancel: Arc, + started: Instant, +} +#[derive(Default)] +struct Queue { + jobs: VecDeque, + tokens: HashMap>, +} +fn queue() -> &'static Mutex { + static Q: OnceLock> = OnceLock::new(); + Q.get_or_init(Default::default) +} +fn authorized(job: &RenderJob) -> bool { + crate::dev_mode::tag_valid(&job.dev_tag) && crate::dev_mode::grants_all(&job.app) +} +pub fn submit(job: RenderJob) -> Result<(), String> { + if !authorized(&job) { + return Err("studio requires a current developer-mode grant".into()); + } + let mut q = queue().lock().unwrap_or_else(|e| e.into_inner()); + if q.tokens.len() >= MAX_JOBS || q.tokens.contains_key(&job.id) { + return Err("studio_busy".into()); + } + let cancel = Arc::new(AtomicBool::new(false)); + q.tokens.insert(job.id.clone(), cancel.clone()); + q.jobs.push_back(Pending { + job, + cancel, + started: Instant::now(), + }); + SignalToUI::set_ui_signal(); + Ok(()) +} +pub fn cancel(id: &str) { + if let Some(cancel) = queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .tokens + .get(id) + { + cancel.store(true, Ordering::Release); + } + SignalToUI::set_ui_signal(); +} +fn retire(id: &str) { + queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .tokens + .remove(id); + SignalToUI::set_ui_signal(); +} +fn validity(p: &Pending) -> Result<(), String> { + if p.cancel.load(Ordering::Acquire) { + return Err("studio_cancelled".into()); + } + if !authorized(&p.job) { + return Err("studio_grant_expired".into()); + } + if p.started.elapsed() >= DEADLINE { + return Err("studio_timeout".into()); + } + Ok(()) +} + +/// Geometry from the platform's created root window, in layout points. The +/// widget lookup and shell's cached DPI can still be empty at a test action. +#[derive(Clone, Copy, Debug, PartialEq)] +pub(crate) struct SurfaceGeometry { + width: f64, + height: f64, + dpi: f64, +} +impl SurfaceGeometry { + fn from_window(created: bool, geom: &WindowGeom) -> Option { + let width = geom.inner_size.x; + let height = geom.inner_size.y; + let dpi = geom.dpi_factor; + (created + && width.is_finite() + && height.is_finite() + && dpi.is_finite() + && width >= 112.0 + && height > 0.0 + && (0.5..=4.0).contains(&dpi)) + .then_some(Self { width, height, dpi }) + } + fn card_width(self) -> f64 { + self.width - 40.0 + } +} +pub(crate) fn surface_geometry(cx: &Cx) -> Option { + // The shell owns the first window; hosted module windows must never supply + // its geometry. Use the current generation rather than an assumed id_zero. + let id = cx.windows.id_iter().next()?; + let window = &cx.windows[id]; + SurfaceGeometry::from_window(window.is_created, &window.window_geom) +} + +struct Active { + pending: Pending, + frame: WidgetRef, + splash: SplashRef, + pass: DrawPass, + list: DrawList2d, + texture: Texture, + jail: PathBuf, + width: f64, + height: f64, + dpi: f64, + surface: SurfaceGeometry, + ticket: Option, + previous: Option>, + stable: usize, + draws: usize, + repaint: bool, +} +pub struct Renderer { + active: Option, + foreground: bool, + timer: Option, +} +impl Default for Renderer { + fn default() -> Self { + Self { + active: None, + foreground: true, + timer: None, + } + } +} +impl Renderer { + /// Called before the shell may consume an event. Background/cancellation + /// revokes publication immediately, even if a GPU lease is still draining. + pub(crate) fn event(&mut self, cx: &mut Cx, event: &Event, surface: Option) { + match event { + Event::Pause | Event::Background => self.foreground = false, + Event::Resume | Event::Foreground => self.foreground = true, + _ => {} + } + if let Some(a) = self.active.as_ref() { + let status = if !self.foreground { + Err("not_foreground".into()) + } else if surface != Some(a.surface) { + Err("studio_surface_changed".into()) + } else { + validity(&a.pending) + }; + if let Err(e) = status { + self.fail(cx, e); + } + } + if !self.foreground { + // Includes jobs whose PNG worker already owns the output file. + for token in queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .tokens + .values() + { + token.store(true, Ordering::Release); + } + } + if self.active.is_none() { + let pending = queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .jobs + .pop_front(); + if let Some(pending) = pending { + let error = if !self.foreground { + Some("not_foreground".into()) + } else { + validity(&pending).err() + }; + if let Some(e) = error { + let _ = pending.job.reply.send(Err(e)); + retire(&pending.job.id); + } else if let Some(surface) = surface { + match Active::new(cx, pending, surface) { + Ok(a) => self.active = Some(a), + Err((p, e)) => { + let _ = p.job.reply.send(Err(e)); + retire(&p.job.id); + } + } + } else { + // Wait for the native surface; do not turn zero geometry + // into a minimum-width successful image. The original + // enqueue deadline still applies while waiting. + queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .jobs + .push_front(pending); + } + } + } + let waiting = !queue() + .lock() + .unwrap_or_else(|e| e.into_inner()) + .jobs + .is_empty(); + if (self.active.is_some() || waiting) && self.timer.is_none() { + self.timer = Some(cx.start_interval(0.08)); + } + if self.active.is_none() && !waiting { + if let Some(timer) = self.timer.take() { + cx.stop_timer(timer); + } + } + if let Some(a) = self.active.as_mut() { + // No input reaches the preview, and no host-service pump runs. + if matches!(event, Event::Signal | Event::Timer(_) | Event::NextFrame(_)) { + a.frame.handle_event(cx, event, &mut Scope::empty()); + } + if let Event::Timer(te) = event { + if self + .timer + .as_ref() + .is_some_and(|t| t.is_timer(te).is_some()) + && a.ticket.is_none() + { + a.repaint = true; + cx.redraw_all(); + } + } + } + } + /// A separate root pass never becomes a live tile or enters its store. + pub fn draw(&mut self, cx: &mut Cx, event: &DrawEvent) { + let Some(a) = self + .active + .as_mut() + .filter(|a| a.repaint && a.ticket.is_none()) + else { + return; + }; + a.repaint = false; + let size = dvec2(a.width, a.height); + a.pass.set_size(cx, size); + { + let mut draw = CxDraw::new(cx, event); + let mut cx = Cx2d::new(&mut draw); + cx.begin_pass(&a.pass, Some(a.dpi)); + a.list.begin_always(&mut cx); + cx.begin_root_turtle(size, Layout::flow_overlay()); + let walk = Walk::abs_rect(Rect { + pos: dvec2(0.0, 0.0), + size, + }); + let vm_id = a + .splash + .borrow() + .and_then(|s| cx.script_ref_vm_id(&s.view.source)); + match vm_id { + Some(id) => widget_async::with_isolate(&mut cx, id, |cx| { + a.frame.draw_walk_all(cx, &mut Scope::empty(), walk) + }), + None => a.frame.draw_walk_all(&mut cx, &mut Scope::empty(), walk), + } + cx.end_pass_sized_turtle(); + a.list.end(&mut cx); + cx.end_pass(&a.pass); + } + a.draws += 1; + let measured = a.splash.area().rect(cx).size.y.clamp( + crate::glance_card::TILE_MIN_HEIGHT, + crate::glance_card::TILE_MAX_HEIGHT, + ); + if (a.height - measured).abs() > 0.5 { + a.height = measured; + a.previous = None; + a.stable = 0; + return; + } + // Shader readiness is checked after at least one platform paint. A + // not-yet-compiled shader makes us request another frame, not a PNG. + if a.draws < 2 || !shaders_ready(cx, a.list.id(), 0) { + return; + } + match a + .texture + .read_back(cx, ReadbackRequest { next_render: true }) + { + Ok(ticket) => a.ticket = Some(ticket), + Err(ReadbackError::Backpressure) => {} + Err(e) => self.fail(cx, format!("studio_readback: {e}")), + } + } + /// Return unowned tickets to the shell's shared readback router. + pub fn readback(&mut self, cx: &mut Cx, result: TextureReadback) -> Option { + if !self + .active + .as_ref() + .is_some_and(|a| a.ticket == Some(result.ticket)) + { + return Some(result); + } + let a = self.active.as_mut().unwrap(); + a.ticket = None; + let bytes = match rgba(&result, a.pending.job.dark) { + Ok(b) => b, + Err(e) => { + self.fail(cx, e); + return None; + } + }; + if a.previous.as_ref() == Some(&bytes) { + a.stable += 1; + } else { + a.stable = 0; + } + a.previous = Some(bytes); + if a.stable < 2 { + return None; + } + let a = self.active.take().unwrap(); + let bytes = a.previous.clone().unwrap(); + a.splash.set_text(cx, ""); + let _ = std::fs::remove_dir_all(&a.jail); + let reply = a.pending.job.reply.clone(); + let id = a.pending.job.id.clone(); + // Compression never stalls the UI. Grants, cancellation and the same + // original deadline are checked again before publishing the file. + let task = cx.task_pool().submit(Lane::Heavy, move || { + let mut p = a.pending; + let outcome = (|| { + validity(&p)?; + let png = Cx::encode_rgba_as_png(result.width as u32, result.height as u32, &bytes) + .map_err(|e| format!("studio_png: {e:?}"))?; + if png.len() > MAX_PNG { + return Err("studio_png_too_large".into()); + } + validity(&p)?; + p.job.output.write_all(&png).map_err(|e| e.to_string())?; + p.job.output.flush().map_err(|e| e.to_string())?; + validity(&p)?; + Ok(RenderResult { + width: result.width as u32, + height: result.height as u32, + settled: true, + }) + })(); + let _ = p.job.reply.send(outcome); + retire(&p.job.id); + }); + match task { + Ok(task) => task.detach(), + Err(e) => { + let _ = reply.send(Err(format!("studio_worker: {e}"))); + retire(&id); + } + } + None + } + fn fail(&mut self, cx: &mut Cx, reason: String) { + if let Some(a) = self.active.take() { + if let Some(ticket) = a.ticket { + cx.cancel_texture_readback(ticket); + } + a.splash.set_text(cx, ""); + let _ = std::fs::remove_dir_all(&a.jail); + let _ = a.pending.job.reply.send(Err(reason)); + retire(&a.pending.job.id); + } + } +} +impl Active { + fn new( + cx: &mut Cx, + pending: Pending, + surface: SurfaceGeometry, + ) -> Result { + let body = match crate::glance::prepare_studio( + &pending.job.app, + &pending.job.source, + &pending.job.data, + pending.job.dark, + ) { + Ok(b) => b, + Err(e) => return Err((pending, e)), + }; + if !crate::glance_card::CAN_RENDER { + return Err(( + pending, + "studio renderer needs the App Hub vocabulary".into(), + )); + } + let width = pending.job.width.unwrap_or(surface.card_width()); + let dpi = surface.dpi; + if !width.is_finite() + || !(72.0..=2048.0).contains(&width) + || !dpi.is_finite() + || !(0.5..=4.0).contains(&dpi) + { + return Err((pending, "invalid studio surface geometry".into())); + } + log!("studio-test-geometry: surface_width={} surface_height={} dpi={} card_width={} pixel_width={}", + surface.width, surface.height, dpi, width, (width * dpi).round()); + let base = cx + .get_data_dir() + .map(PathBuf::from) + .unwrap_or_else(std::env::temp_dir); + let jail = base.join(format!(".studio-isolate-{}", uuid::Uuid::new_v4())); + if let Err(e) = std::fs::create_dir(&jail) { + return Err((pending, e.to_string())); + } + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + if let Err(e) = std::fs::set_permissions(&jail, std::fs::Permissions::from_mode(0o700)) + { + let _ = std::fs::remove_dir(&jail); + return Err((pending, e.to_string())); + } + } + crate::glance_card::ensure_vocabulary(cx); + let frame = cx.with_vm(|vm| { + let value = script_eval!(vm,{use mod.widgets.* GlanceTileFrame {}}); + WidgetRef::script_from_value(vm, value) + }); + let splash = frame.splash(cx, ids!(card)); + if let Some(mut s) = splash.borrow_mut() { + s.set_allow_net(false); + } + splash.set_sandbox_dir(cx, Some(jail.clone())); + splash.set_storage_quota(cx, Some(0)); + splash.set_host_caps(cx, Vec::new()); + splash.set_host_prompts(cx, false); + splash.set_policy(cx, Some(Vec::new()), Some(5_000_000)); + splash.set_memory_bytes(cx, Some(16 * 1024 * 1024)); + splash.set_text(cx, &body); + let mut children = 0; + if let Some(s) = splash.borrow() { + s.children(&mut |_, _| children += 1); + } + let running = splash + .borrow_mut() + .and_then(|mut s| s.isolate_heap_key(cx)) + .is_some_and(makepad_widgets::splash_policy::may_run); + if children == 0 || !running { + splash.set_text(cx, ""); + let _ = std::fs::remove_dir_all(&jail); + return Err((pending, "studio_eval_failed: preview has no root content or exhausted its instruction budget".into())); + } + let pass = DrawPass::new_with_name(cx, "studio_l0_preview"); + let texture = Texture::new_with_format( + cx, + TextureFormat::RenderBGRAu8 { + size: TextureSize::Auto, + initial: true, + }, + ); + pass.set_color_texture( + cx, + &texture, + DrawPassClearColor::ClearWith(vec4(0.0, 0.0, 0.0, 0.0)), + ); + let list = DrawList2d::new(cx); + cx.redraw_all(); + Ok(Self { + pending, + frame, + splash, + pass, + list, + texture, + jail, + width, + height: crate::glance_card::TILE_DEFAULT_HEIGHT, + dpi, + surface, + ticket: None, + previous: None, + stable: 0, + draws: 0, + repaint: true, + }) + } +} +fn shaders_ready(cx: &Cx, id: DrawListId, depth: usize) -> bool { + if depth > 64 || cx.draw_shaders_pending() { + return false; + } + let items = &cx.draw_lists[id].draw_items; + (0..items.len()).all(|i| { + let kind = &items[i].kind; + if let Some(id) = kind.sub_list() { + return shaders_ready(cx, id, depth + 1); + } + if let Some(call) = kind.draw_call() { + #[cfg(any(target_os = "android", target_os = "linux", target_os = "windows"))] + { + return cx.is_draw_shader_window_ready(call.draw_shader_id); + } + #[cfg(not(any(target_os = "android", target_os = "linux", target_os = "windows")))] + { + let _ = call; + } + } + true + }) +} +/// Platform readbacks are premultiplied. Flatten before PNG encoding rather +/// than interpreting premultiplied bytes as straight-alpha PNG samples. +fn rgba(result: &TextureReadback, dark: bool) -> Result, String> { + let bytes = result + .data + .as_ref() + .map_err(|e| format!("studio_readback: {e}"))?; + let row = result + .width + .checked_mul(4) + .ok_or("invalid readback dimensions")?; + let total = result + .stride + .checked_mul(result.height) + .ok_or("invalid readback dimensions")?; + if result.stride < row || bytes.len() < total { + return Err("invalid readback stride".into()); + } + let mut out = Vec::with_capacity(row * result.height); + let backdrop = if dark { 24u16 } else { 255u16 }; + for y in 0..result.height { + let y = match result.origin { + ReadbackOrigin::TopLeft => y, + ReadbackOrigin::BottomLeft => result.height - 1 - y, + }; + for p in bytes[y * result.stride..y * result.stride + row].chunks_exact(4) { + let rgb = match result.channel_order { + ReadbackChannelOrder::Rgba => [p[0], p[1], p[2]], + ReadbackChannelOrder::Bgra => [p[2], p[1], p[0]], + }; + for c in rgb { + out.push( + (u16::from(c) + (backdrop * (255 - u16::from(p[3])) + 127) / 255).min(255) + as u8, + ); + } + out.push(255); + } + } + Ok(out) +} + +#[cfg(test)] +mod tests { + use super::*; + fn pixels(order: ReadbackChannelOrder, origin: ReadbackOrigin, data: &[u8]) -> TextureReadback { + TextureReadback { + ticket: ReadbackTicket(1), + allocation_generation: 1, + producer_serial: 1, + width: 1, + height: 2, + stride: 8, + channel_order: order, + origin, + data: Ok(Arc::from(data)), + } + } + #[test] + fn png_input_flattens_premultiplied_rows_and_respects_padding_and_origin() { + let result = pixels( + ReadbackChannelOrder::Bgra, + ReadbackOrigin::BottomLeft, + &[0, 0, 128, 128, 9, 9, 9, 9, 30, 20, 10, 255, 9, 9, 9, 9], + ); + assert_eq!( + rgba(&result, false).unwrap(), + [10, 20, 30, 255, 255, 127, 127, 255] + ); + assert_eq!( + rgba(&result, true).unwrap(), + [10, 20, 30, 255, 140, 12, 12, 255] + ); + } + #[test] + fn bad_readback_payloads_fail_before_encoding() { + assert!(rgba( + &pixels(ReadbackChannelOrder::Rgba, ReadbackOrigin::TopLeft, &[0; 8]), + false + ) + .is_err()); + let mut result = pixels( + ReadbackChannelOrder::Rgba, + ReadbackOrigin::TopLeft, + &[0; 16], + ); + result.data = Err(ReadbackError::Cancelled); + assert!(rgba(&result, false).unwrap_err().contains("Cancelled")); + } + #[test] + fn preview_is_l0_only_and_explicit_modes_do_not_depend_on_global_dark() { + let (source, data) = crate::glance::demo_digest(); + let light = crate::glance::prepare_studio("os.news", &source, &data, false).unwrap(); + let dark = crate::glance::prepare_studio("os.news", &source, &data, true).unwrap(); + assert_ne!(light, dark); + assert_eq!( + light, + crate::glance::prepare_studio("os.news", &source, &data, false).unwrap() + ); + assert!(crate::glance::prepare_studio("os.news", "View{}", &data, false).is_err()); + assert!(crate::glance::prepare_studio("os.news", &source, &Value::Null, false).is_err()); + } + #[test] + fn digest_values_are_host_owned_and_chat_snapshots_fail_closed() { + let source = include_str!("../../resources/glance/news-brief.card"); + let data = serde_json::json!({"brief":{"summary":"FORGED","points":[],"sources":[]},"$status":{"brief":"ready"}}); + let body = crate::glance::prepare_studio("os.news", source, &data, false).unwrap(); + assert!(!body.contains("FORGED")); + assert!( + crate::glance::prepare_studio("os.mail", source, &data, false) + .unwrap_err() + .contains("own app") + ); + let (_, _, source, data) = crate::glance::demo_mail().remove(0); + assert!( + crate::glance::prepare_studio("os.mail", &source, &data, false) + .unwrap_err() + .contains("studio_chat_unsupported") + ); + } + #[test] + fn asynchronous_images_are_refused_instead_of_reported_as_settled() { + let (source, mut data) = crate::glance::demo_digest(); + let source = source.replace( + "TextTitle(text: status.message, width: .fill)", + "Photo(src: status.message)", + ); + data["status"]["message"] = serde_json::json!("https://example.invalid/image.png"); + let error = crate::glance::prepare_studio("os.news", &source, &data, false).unwrap_err(); + assert!(error.contains("studio_images_unsupported"), "{error}"); + } + + #[test] + fn unowned_readback_is_preserved_for_other_shell_consumers() { + let mut cx = Cx::new(Box::new(|_, _| {})); + let mut renderer = Renderer::default(); + let original = pixels( + ReadbackChannelOrder::Rgba, + ReadbackOrigin::TopLeft, + &[0; 16], + ); + assert_eq!( + renderer.readback(&mut cx, original).unwrap().ticket, + ReadbackTicket(1) + ); + } + #[test] + fn geometry_waits_for_created_surface_and_uses_its_actual_scale() { + let mut geom = WindowGeom { + inner_size: dvec2(0.0, 0.0), + dpi_factor: 1.0, + ..Default::default() + }; + assert!(SurfaceGeometry::from_window(true, &geom).is_none()); + geom.inner_size = dvec2(432.0, 912.0); + geom.dpi_factor = 2.5; + assert!(SurfaceGeometry::from_window(false, &geom).is_none()); + let surface = SurfaceGeometry::from_window(true, &geom).unwrap(); + assert_eq!(surface.card_width(), 392.0); + assert_eq!(surface.card_width() * surface.dpi, 980.0); + for invalid in [0.0, f64::NAN, f64::INFINITY] { + geom.dpi_factor = invalid; + assert!(SurfaceGeometry::from_window(true, &geom).is_none()); + } + } + #[test] + fn background_events_close_admission_and_resume_reopens_it() { + let mut cx = Cx::new(Box::new(|_, _| {})); + let mut renderer = Renderer::default(); + renderer.event(&mut cx, &Event::Background, None); + assert!(!renderer.foreground); + renderer.event(&mut cx, &Event::Foreground, None); + assert!(renderer.foreground); + } +} diff --git a/crates/shell/src/studio/module.rs b/crates/shell/src/studio/module.rs new file mode 100644 index 00000000..5c422865 --- /dev/null +++ b/crates/shell/src/studio/module.rs @@ -0,0 +1,26 @@ +//! Internal native wrapper for an admitted developer script instance. +use makepad_widgets::*; +use makepad_app_module::{AppModule, ExecOutcome, InstanceHandles, InstanceParts, + OpenArgKind, OpenSchema, ServiceExecutor, ValidatedOpen, + makepad_ai_services::wire::{ServiceCall, ServiceManifest, ToolResult}}; +pub struct StudioModule; +pub static STUDIO_MODULE: StudioModule = StudioModule; +impl AppModule for StudioModule { + fn id(&self) -> &'static str { "studio-app" } + fn label(&self) -> &'static str { "App Studio" } + fn register(&self, vm: &mut ScriptVm) { super::apps::script_mod(vm); } + fn open_schema(&self) -> OpenSchema { OpenSchema::new(1).arg("instance_id", OpenArgKind::Text, true) } + fn capabilities(&self) -> &'static [&'static str] { &[] } + fn create(&self, vm: &mut ScriptVm, open: ValidatedOpen, _handles: InstanceHandles) -> InstanceParts { + let id = open.text("instance_id").unwrap_or_default().to_owned(); + InstanceParts { root: super::apps::create(vm, &id), executor: Box::new(StudioExecutor), + shutdown: Box::new(move |cx| cx.with_cx_mut(|cx| super::apps::shutdown(cx, &id))) } + } +} +struct StudioExecutor; +impl ServiceExecutor for StudioExecutor { + fn manifest(&self) -> ServiceManifest { ServiceManifest::new("studio-app", "App Studio", "An isolated local developer app.") } + fn execute(&mut self, _cx: &mut Cx, call: &ServiceCall) -> ExecOutcome { + ExecOutcome::Done(ToolResult::unavailable(&call.call_id, "Use the scoped studio tools")) + } +} diff --git a/crates/shell/src/system_chat/session.rs b/crates/shell/src/system_chat/session.rs index 196ac14d..1b6297db 100644 --- a/crates/shell/src/system_chat/session.rs +++ b/crates/shell/src/system_chat/session.rs @@ -90,6 +90,8 @@ pub trait SystemHost: Send { /// The host tools to register on the system session now (empty: none /// granted, or the grant was withdrawn). fn declarations(&self) -> Vec; + /// Workspace confirmed by this connection, never a model-supplied path. + fn workspace_opened(&self, _workspace: Option<&str>) {} } /// The shell's: Setup's grants and the app peers' host state. @@ -107,8 +109,17 @@ impl SystemHost for ShellSystemHost { // Which apps have an agent, and asking the person to allow one // (ADR 0004 §4; answered by the chat itself, `crate::agents`). decls.extend(crate::agents::declarations()); + // octos apply_turn_host_tool_rosters applies the kernel allowlist + // BEFORE adding this connection-owned host set. No baseline kernel + // tool grant is needed, and external turns cannot inherit this set. + if crate::host_tools::studio::SUPPORTED && crate::dev_mode::grants_all(crate::host_tools::SYSTEM) { + decls.extend(crate::host_tools::studio::declarations(crate::host_tools::SYSTEM)); + } decls } + fn workspace_opened(&self, workspace: Option<&str>) { + crate::host_tools::studio::system_workspace_opened(workspace); + } } /// What the UI asks of the driver. @@ -438,6 +449,7 @@ impl Driver { } fn drop_link(&mut self) { + self.system_host.workspace_opened(None); self.link = None; self.pending.clear(); self.opened = false; @@ -586,6 +598,7 @@ impl Driver { self.retry_at = Some(Instant::now() + Duration::from_secs(3)); } None => { + self.system_host.workspace_opened(result.and_then(|r| r["opened"]["workspace_root"].as_str())); self.opened = true; self.model.set_phase(Phase::Ready); self.sync_tools(); diff --git a/crates/shell/src/system_chat/tests.rs b/crates/shell/src/system_chat/tests.rs index 4e1a178f..bfd5df3d 100644 --- a/crates/shell/src/system_chat/tests.rs +++ b/crates/shell/src/system_chat/tests.rs @@ -27,6 +27,7 @@ struct Script { unavailable: Option, /// `session/open` fails with this message. open_error: Option, + opened_workspace: Option, } #[derive(Clone, Default)] @@ -57,7 +58,7 @@ impl Link for FakeLink { let reply = match method.as_str() { "session/open" => match &s.open_error { Some(e) => json!({"jsonrpc": "2.0", "id": id, "error": {"code": -32000, "message": e}}), - None => json!({"jsonrpc": "2.0", "id": id, "result": {"opened": {"session_id": SYSTEM_SESSION}}}), + None => json!({"jsonrpc": "2.0", "id": id, "result": {"opened": {"session_id": SYSTEM_SESSION, "workspace_root": s.opened_workspace}}}), }, "session/hydrate" => json!({"jsonrpc": "2.0", "id": id, "result": {"messages": s.history.clone()}}), _ => json!({"jsonrpc": "2.0", "id": id, "result": {}}), @@ -1070,3 +1071,22 @@ fn the_system_agent_gets_the_read_tools_of_the_native_apps_that_run_here_never_t assert_eq!(calculator.system_tools, ["calculator.eval"]); assert_eq!(crate::native_apps::find("notes").unwrap().system_tools, ["notes.search", "notes.read"]); } + +#[test] +fn kernel_confirmed_system_workspace_is_delivered_and_cleared_on_disconnect() { + struct Workspaces(Arc>>>); + impl SystemHost for Workspaces { + fn declarations(&self) -> Vec { vec![] } + fn workspace_opened(&self, path: Option<&str>) { self.0.lock().unwrap().push(path.map(str::to_owned)); } + } + let fake = Fake::default(); + fake.s().opened_workspace = Some("/kernel/system-workspace".into()); + let seen = Arc::new(Mutex::new(Vec::new())); + let mut driver = Driver::with_system_host(Box::new(FakeConnector(fake.clone())), Box::new(Workspaces(seen.clone()))); + driver.command(Command::Open); + settle(&mut driver); + assert_eq!(seen.lock().unwrap().last().cloned(), Some(Some("/kernel/system-workspace".into()))); + driver.command(Command::Close); + settle(&mut driver); + assert_eq!(seen.lock().unwrap().last().cloned(), Some(None)); +} diff --git a/docs/adr/0006-app-studio-on-the-phone.md b/docs/adr/0006-app-studio-on-the-phone.md index 72b21fbd..679da8e9 100644 --- a/docs/adr/0006-app-studio-on-the-phone.md +++ b/docs/adr/0006-app-studio-on-the-phone.md @@ -1,20 +1,58 @@ # ADR 0006: App Studio on the phone - **Date:** 2026-10-02 -- **Status:** Accepted (2026-10-03). Implementation has not started; milestone 1 comes first. +- **Status:** Accepted (2026-10-03). Implementation in progress: a fresh model-authored app passed physical-device functional checks; portrait visual review passed; fault injection and the full image/workflow pipeline remain pending. - **Scope:** How an agent on the phone turns an image (a generated design or a screenshot of an existing app) into an OctoSense app or glance card, looks at its own result and improves it, entirely on the phone, at first in developer mode. Covers the inputs, the in-process renderer, the checks, the rules that can change without a build, the tools agents get, and what of the OctoScript App Design Flow moves to the phone. There is no compile in the loop and no Mac. - **Relates to:** [ADR 0002](0002-event-driven-app-agents.md) (§6 the `card_render` and `card_critique_payload` toolbox tools; §7 a card is rendered, critiqued and revised before it is published; milestone M6); [ADR 0004](0004-native-apps-hosting-and-peers.md) (app agents, host tools, approvals, §13 developer mode); [ADR 0005](0005-app-contract.md) (the app contract and bundles); [Home ADR 0004](home/0004-system-apps-are-contained-script-apps.md) (contained script apps); [Home ADR 0005](home/0005-settings-octoscript-controller.md) and [Home ADR 0006](home/0006-builtin-settings.md) (Settings and its developer options); App Hub's [`card-studio`](https://github.com/OctoSense-org/OctoSense-App-Hub/tree/main/crates/card-studio) crate and [skill](https://github.com/OctoSense-org/OctoSense-App-Hub/tree/main/skills/card-studio); the [OctoScript App Design Flow](https://github.com/OctoSense-org/OctoScript-App-Design-Flow) (`flows/image-to-card`, `flows/image-lib`); octos issue #1149, closed, which added the `image_generation` stub (its backend needs a new issue). +## Implementation status + +The implementation now covers L0 glance rendering and a local, offline script-app authoring path. A fresh Task Planner authored by **DeepSeek V4 Flash passed 129 instrument-driven tool calls on a physical OnePlus 6**. This establishes the functional path from a new design to a working local app. Portrait app and keyboard visual review also passed; fault injection remains pending. A fixture PNG, a passing unit test or a successful build alone would not establish that result. + +The shell registers the following tools only while developer mode covers the calling system or app agent: + +| Implemented tool | Current behavior | +| --- | --- | +| `studio.render` | L0 glance source/data → PNG, using the real glance width and height bounds. Arguments are relative `source_path`, optional `data_path` and `dark`. | +| `studio.bundle_check` | Copies a workspace-relative bundle into host-private staging, computes its digest and admits that copy. The author's source is unchanged. | +| `studio.open` | Opens a visible script app from either `bundle_path` (disposable preview) or `app_id` (local developer install with persistent state). | +| `studio.inspect` | Captures app pixels and scoped Makepad diagnostics. Returns PNG `path`, compact `snapshot.widgets`/check summary, and a full diagnostic JSON `snapshot_path`; optional `offset` follows `next_offset` pagination. | +| `studio.input` | Sends tap, text or scroll events to visible, enabled widget selectors within the caller's own instance. | +| `studio.close` | Closes that instance; preview state is discarded and installed state is retained. | +| `studio.install` | Records a local developer install under `dev.studio.*`, available from Home while its `DevTag` is valid. | + +Inspection replies are bounded to 3,800 UTF-8 bytes so actionable selectors survive the pinned kernel's 4 KiB model-output limit. Pages retain exact selectors, report `total_widgets` and `next_offset`, and explicitly mark shortened text/value fields. Pass the returned selector to `studio.input`; a painted label is not an identity. Each page observes the current UI. The full original result is written as pretty-printed JSON (at most 1 MiB) in the caller's workspace, retaining all widget rectangles, geometry, tree and findings. `read_file` can read bounded line ranges through `snapshot_path`. Both artifacts are removed if completion fails, the grant expires or the workspace scope changes. + +Tools are not shareable or available to background turns. The relay supplies the trusted caller and workspace: the system session's kernel-confirmed root, or the app peer's confirmed root and own conversation context. The model supplies relative input paths and instance ids, never another app's identity or an output directory. `studio.install` and `studio.input` are `act` tools. A private install receipt binds the authoring app/account/session/context, developer activation, manifest and digest. Launch revalidates it; revocation closes the app and removes launcher availability. A second instance of the same installed app is refused to prevent concurrent state writes. + +The first script-app path accepts `main.splash` with storage-only permissions: no app agent, accounts, external network, host services or in-screen resource loader. An original launcher icon may be present in the bundle; rich artwork routes are still future work. Staging permits 128 files/directories, eight directory levels, 2 MiB total, 512 KiB per file and 64 KiB for `main.splash`. The manifest's resolved ceilings are at most 1 MiB app storage, five million script instructions and a 16 MiB heap. These limits are installed before evaluation. Private preview writes are allowed for functional tests and discarded on close; developer installs write only their own persistent jail. No signed catalog or publisher identity is fabricated, and local admission does not count as App Hub publishing approval. + +L0 glance rendering remains narrower: 16 KiB source, 32 KiB JSON data, zero storage quota, no chat or image resources. It reports `settled: true` after three matching pixel samples with shaders ready. Interactive app inspection returns `settled: false`, since a current frame does not prove an arbitrary script has stopped changing. PNGs are limited to 5 MiB. UI/render requests have a 20-second deadline inside a 25-second host wait and the normal 30-second kernel call deadline; cancellation, foreground state and the developer grant are checked through readback and encoding. + +Android still has no HTTP remote instrument at the pinned Makepad revision. Studio calls Makepad's underlying widget instrument **in process**, rooted only at its own app; it does not inspect other apps or the shell's controls. Current checks report empty geometry, clipped text and small buttons. The shared `card-studio` check/critique port, image comparison, `mod.studio` toolbox adapter, editable workflow templates, image generation and capture sessions remain unimplemented. + +Start at [host tools and admission](../../crates/shell/src/host_tools/studio_bundles.rs), then [the visible app runner](../../crates/shell/src/studio/apps.rs) and [the bilingual architecture walkthrough](../architecture-walkthrough.md#follow-app-studio-from-the-agent-to-a-working-app). The [fresh-app brief](../studio/task-planner-brief.md) and [functional device harness](../../tools/studio-flow-device-test.py) define acceptance against a newly authored project. The older [L0 device probe](../../tools/studio-device-probe.py) covers denial, palette/PNG behavior and background cancellation only. The Android wrapper's `--dev-mode` enables developer options; `--development` selects signing and `--package-name` selects an isolated test identity. The physical functional result is recorded below, separately from build and unit results. + +### Physical functional evidence + +The `adr0006-task-planner-acceptance-v2` run used the isolated `dev.makepad.octosense.studio` test package. Its generation receipt identifies `deepseek/deepseek-v4-flash` and binds the model-authored design, manifest, `main.splash` and icon to bundle digest `7892501e8b69d752b002130879dde11af8f7c7e7b7e620a6854b97892fc95f3b`. The model called `view_image`; its tool result reported `shown_to_model: true`, and the model continued its turn. This records image delivery and continuation, not an independent guarantee that every visual detail was understood. + +The 129-call harness reported `functional_passed_visual_review_required`. It exercised task entry, completion and All/Active/Done filters; discarded preview state; verified separate installed storage; and retained installed state through close/reopen and process restart. It also verified exact Chinese text after native input and reopening, and scrolled through seven additional rows. The harness made no direct source or app-storage writes. + +Subsequent manual review passed portrait readability, long-title wrapping, bottom-row reachability, Chinese glyphs, app contrast and absence of unintended app overlap. An Android-wide screenshot also confirmed keyboard presentation and correct viewport shrinkage; app-texture PNGs alone exclude keyboard pixels. Generous spacing leaves about one row above the keyboard. A shell floating overlay and low-contrast status bar remain separate observations; rotation/landscape was not established. See the [validation report](../studio/oneplus6-validation.md) for provenance and limits. Malformed saved JSON preservation and reported save failures were **not tested** because they require isolated fault injection. The full image-generation/refinement workflow and `mod.studio` adapter remain unimplemented. + +The remaining decision describes the complete intended design. Some local app support from milestone 3 is implemented ahead of the shared checks and workflow adapter; the milestones as a whole are not complete. + ## Context -Making an OctoSense app or card from a picture works today only on a desktop: +At the time of the proposal, the image-driven app/card flow was provided by these desktop tools. The phone implementation above does not yet port that image pipeline: - **The OctoScript App Design Flow** is about 12,150 lines of Python tooling (about 16,000 with its tests), plus per-app examples. It runs on a Mac with cargo, Makepad Studio and App Hub's `card-host --remote`, and also needs Python 3.12 with numpy, OpenCV, Pillow and fontTools, Swift and Node. Its two image paths are: - the Sketch kit (`flows/kits/sketch`), which needs `sketchtool`, Swift and a licensed kit, and has its own gates (`gate_structure`, `gate_composition`, `gate_fill`, `gate_visual`); - image-to-card (`flows/image-to-card`, `flows/image-lib`): crop scenes from an image, read their text with Apple Vision OCR (through Swift), map regions to widgets, generate L0, render and compare. Its gate is `flows/image-lib/gate.py`: native geometry, OCR text, ink and colour checks, and the visual-review receipt. - **App Hub's `card-studio`** renders a card in a hidden `card-host --remote`, grabs a PNG, runs measured checks (hidden, clipped or truncated text, overflow, overlap, empty or failed states, fit, lint, realize) and builds a critique payload for a vision model. The checks and the payload are plain Rust that depends only on serde and serde_json. They read the remote instrument's snapshot and dump formats and card-host's widget ids, and the critique prompt is written for an L0 glance card. The rendering needs the remote instrument and a separate process. -On the phone none of that runs: +The desktop process/HTTP pipeline cannot run unchanged on the phone: - There is no Python, and no Apple Vision for OCR. - Makepad's remote instrument is compiled out on Android (`platform/src/remote.rs`, the `target_os = "android"` stub), and there is no second process to host a card. @@ -35,15 +73,15 @@ What the phone already has: - start an app (`startActivity`, `startTask`); - inject input (`tap`, `swipe`, `typeText`, `pressKey`), which it refuses while the keyguard is showing. `captureScreen` and `startActivity` do not check the keyguard. - **No image generation yet.** octos registers an `image_generation` tool but binds no backend, so every call fails with `no_backend_bound`. The tool is on no client's tool list: not octos's external clients', not the system agent's, not the developer app agents'. -- **No developer mode on a phone today.** A development build (`cfg(dev_mode)`) honours it from Settings; a release build only with the `--dev-grant-all` launch flag, which a phone cannot pass. The Home APK is built `--release` without the feature. In a home with real accounts developer mode ends after 8 hours; only a home marked as a developer profile keeps it ([`dev_mode.rs`](../../crates/shell/src/dev_mode.rs)). A phone has one home. +- **Phone developer mode.** A development build (`cfg(dev_mode)`) honours it from Settings; an ordinary release build only with the `--dev-grant-all` launch flag, which a phone cannot pass. The Home APK still defaults to `--release` without the feature; the milestone 1 `--dev-mode` build option includes the existing feature while retaining release optimization. In a home with real accounts developer mode ends after 8 hours; only a home marked as a developer profile keeps it ([`dev_mode.rs`](../../crates/shell/src/dev_mode.rs)). A phone has one home. -For many people the phone is the only computer they have. An agent that can make and refine its own apps there should not need a Mac or a server. +For many people the phone is the only computer they have. An agent that can make and refine its own apps there should not need a desktop or a build/render server. ## Decision ### 1. An App Studio that runs on the phone, developer mode first -The whole loop runs on the device: +The authoring workspace and rendering loop live on the device: 1. **Take an image.** 2. **Draft** an app or card. @@ -53,7 +91,7 @@ The whole loop runs on the device: 6. **Critique** it with the agent's own vision. 7. **Edit** it, then render again. -There is no compile in the loop and no Mac or server anywhere. The first release is gated to developer mode (section 8). Normal mode follows once approvals cover screen capture and installs. +There is no compile in the loop and no desktop or build/render server is required. Generation and vision still call the model providers configured by the person. The first release is gated to developer mode (section 8). Normal mode follows once approvals cover screen capture and installs. ### 2. The input is an image; there is no Sketch path on the phone @@ -62,7 +100,8 @@ An image comes from one of two places. - **Generated designs.** The person, or the agent, asks for a design. Generation goes through a provider the person configured in AI providers, and the PNG lands in the studio project's folder. octos's `image_generation` tool gets this backend instead of a separate OctoSense tool, and joins the tool lists that need it (octos's external clients, the system agent, developer app agents), so every client sees one tool. The person types the image provider's key on AI providers' host sheet, and it is kept like every provider key, where the kernel reads it: in the profile's `env_vars` on a phone (app-private, mode 0600), behind a `keychain:` marker on a desktop. octos's backend resolves it from there as it resolves an LLM key, and no app ever sees it. - **Screenshots of existing apps, to clone them.** - On every phone, the person picks screenshots with the system document picker, as AI providers' QR import does (one PNG or JPEG at a time today; picking several is new), or shares images to OctoSense (Home's share target accepts only text today; image shares are new). - - On the OctoSense ROM, in developer mode and after the person approves a capture session, the studio drives the agent service. It opens the app (`startActivity`), walks its screens (`tap`, `swipe`) and captures each one (`captureScreen`). A session is bound to one app and ends when another app comes to the front. It only navigates, never types (`typeText` is not used), refuses to start or continue while the phone is locked, and shows a stop control the person can use at any time. + - On the OctoSense ROM, in developer mode and after the person approves a capture session, the studio drives the agent service. It opens the app (`startActivity`) and captures its screens (`captureScreen`); automated input has the additional guard below. A session is bound to one app and ends when another app comes to the front. It refuses to start or continue while the phone is locked and shows a stop control the person can use at any time. Coordinates alone cannot distinguish navigation from sending, purchasing or changing a setting. Autonomous `tap`/`swipe` stays deferred until a trusted host guard can identify the target action and require the person’s approval for actions that change external state; omitting `typeText` is insufficient. Until that guard exists, the person navigates the target app and the approved session only captures its screens. + - The ROM capture transport must also change before these sessions ship: full PNGs must travel through a file descriptor or another bounded transfer, rather than an unbounded Binder byte-array reply. The host checks dimensions and byte limits, copies the image into the project folder, and closes or cancels the transfer on timeout or session end. This is future ROM/client work, not a milestone 1 dependency. - On stock Android, a capture session through MediaProjection, with Android's own consent prompt, comes later. The Sketch kit, its gates included, stays a desktop tool and is not ported. @@ -70,7 +109,7 @@ The Sketch kit, its gates included, stays a desktop tool and is not ported. ### 3. The output is a glance card or a contained script app - **A glance card:** L0 only, as for every generated card, published through `glance.publish` as the app. -- **A script app:** a bundle per [ADR 0005](0005-app-contract.md), with `main.splash`, Octoscript controllers, its kit and its data. In developer mode it installs locally as a developer bundle. App Hub installs only from the signed catalog today, so this adds a local install path: a developer bundle carries developer mode's `DevTag`, runs only while that tag is valid, and is never offered through the catalog. Publishing goes through App Hub's usual gate and signing. +- **A script app:** a bundle per [ADR 0005](0005-app-contract.md), with `main.splash`, Octoscript controllers, its kit and its data. In developer mode it installs locally as a developer bundle. App Hub's ordinary install path remains tied to its signed catalog. Studio now adds a separate local path for offline, storage-only `main.splash` apps: a developer bundle carries developer mode's `DevTag`, runs only while that tag is valid, and is never offered through the catalog. Broader bundle support remains planned. Publishing goes through App Hub's usual gate and signing. A cloned app is the person's own prototype. Section 9 says what may and may not be copied. @@ -87,7 +126,7 @@ It then: 2. reads the pixels back, one readback at a time (makepad's 32 MiB readback budget is shared by every pending readback), composes the premultiplied RGBA onto the surface's backdrop and writes an opaque PNG; 3. builds the widget-tree capture the checks read, in the capture format `card-studio` reads (section 5). -There is no remote instrument, no `card-host` process and no window grab. Studio renders never go through the glance store, so its publish rate and card limits do not apply. Script cards render under a no-side-effect policy: a `host.request` that would change something is refused during a studio render. +There is no HTTP remote server, `card-host` process or whole-window grab. The local app path uses Makepad's in-process widget instrument, scoped to its own root. Studio renders never go through the glance store, so its publish rate and card limits do not apply. Every preview uses a policy that confines side effects before source evaluation: a fresh disposable storage jail (or a bounded snapshot copied into it), no live app/account files, and disabled external network access. Bundle artwork is staged by the host through a bounded local asset path; service data comes from explicit preview fixtures. Mutating `host.request` calls are refused, but that gate alone is insufficient: Splash’s direct `fs.write`/`append`/`remove` and network APIs must remain confined or disabled too. A glance render discards its state on completion or cancellation. An interactive preview may write within its disposable jail for functional tests, then discards that state on close; an installed developer app keeps its own private state. The target’s layout and lowering are reused; its live account storage and service authority are not. ### 5. Checks and comparison are Rust, shared by phone and desktop @@ -118,7 +157,7 @@ The system toolbox's runner becomes the studio runner. - **What moves from the design flow to Octoscript** (rules and text, not pixels): - the role-first mapping policy (`semantics`, `core/policy`); - review records; - - the code generation of image-to-card (`compile`, `extract`, `register`), without its desktop assumptions: fonts read from a `splash-makepad` checkout, artwork fetched from a local server, and the single 406×776 artboard; + - the image-to-card code generation (`flows/image-lib/compile.py`, `flows/image-lib/register.py` and `flows/image-to-card/extract.py`), without its desktop assumptions: fonts read from a `splash-makepad` checkout, artwork fetched from a local server, and the single 406×776 artboard; - repair plans. - **What moves to Rust** (in `card-studio`): - pixel work, including `extract`'s crop and `core/policy`'s crop comparison; @@ -130,10 +169,11 @@ The system toolbox's runner becomes the studio runner. ### 7. Agents drive the loop through host tools -Host tools on the shell: +The complete design includes these host tools; the implementation table above identifies the current subset: - `studio.render`, `studio.check`, `studio.compare`, `studio.critique_payload` and `studio.bundle_check`, declared `risk: read`; -- `studio.install`, for a developer bundle, declared `risk: act`, because it changes what is installed. +- `studio.install`, for a developer bundle, declared `risk: act`, because it changes what is installed; +- the implemented interactive app path also has `studio.open`, `studio.inspect` and `studio.close` (`read`), plus `studio.input` (`act`) for stateful behavior tests within the caller's own app. None is a background tool, because a render needs Home in front (section 10). Each registers a call timeout that covers settle and readback; octos's default is 30 s. @@ -155,21 +195,21 @@ An octos change adds a media field to `peer/tool/result`, mapped onto octos's in ### 8. Developer mode for the studio -- **The developer build.** Home is built with `--features dev-mode`, through a new `--dev-mode` option of [`build-home.py`](../../rom/scripts/build-home.py). Its existing `--development` option only picks the signing keystore. On the ROM the developer build is platform-signed, because the agent service accepts only platform-signed OctoSense packages. +- **The developer build.** Home is built with `--features dev-mode`, through the `--dev-mode` option of [`build-home.py`](../../rom/scripts/build-home.py). Its existing `--development` option only picks the signing keystore; ordinary release builds keep developer options disabled. Standalone tests use a separate `--package-name`, such as `dev.makepad.octosense.studio`, without renaming or replacing the installed Bridge. Build receipts record both choices and ROM staging rejects developer-enabled receipts. On the ROM the developer build is platform-signed, because the agent service accepts only platform-signed OctoSense packages. - **The person's own home keeps its 8-hour limit.** The studio never marks a home that holds real accounts as a developer profile. A persistent developer profile on the phone needs a second home with test accounts, as ADR 0004 §13 intends; how the phone hosts one is open. - **One new grant.** Developer mode gives app agents the `studio.*` tools. -- **One exception to ADR 0004 §13.** A capture session's approval is a new approval kind that developer mode never answers by itself. The person answers it once per session, because the session sees other apps' screens, and the answer goes to the developer-mode audit log. +- **Person-only approvals amend ADR 0004 §13.** Developer mode never answers a capture-session approval by itself. The person answers once per session because the session sees other apps' screens. If the future trusted input guard permits an autonomous action that changes external state, its action-specific approval must also come from the person; the capture-session approval does not authorize that action. Both answers go to the developer-mode audit log. Until that guard and approval route exist, the person operates the target app. ### 9. Privacy, safety and other people's work - **Screenshots can hold personal data.** They stay in the project's folder and are sent only to the model provider the person chose. Deleting a project deletes its folder and purges the conversation sessions that viewed its images, because octos keeps viewed images as message media. - **Other apps are captured only in an approved session** started by the person (section 2). A session is visible on screen, never runs in the background, and leaves secure windows out. -- **A clone is for the person's own use or as a prototype.** Studio projects never ship logos, brand marks, icons or copyrighted images in a bundle. The studio records where every image came from, and `studio.bundle_check` refuses any raster asset derived from a captured screenshot. The studio regenerates or draws artwork instead. Publishing a clone needs the publisher's own assets and goes through App Hub's review. +- **A clone is for the person's own use or as a prototype.** Studio projects never ship logos, brand marks, icons or copyrighted images in a bundle. The planned image workflow records where every image came from and adds a provenance gate refusing raster assets derived from captured screenshots. The current `studio.bundle_check` checks local developer admission and resource limits; it does not establish image provenance. The studio regenerates or draws artwork instead. Publishing a clone needs the publisher's own assets and goes through App Hub's review. - **Generated cards stay L0** and script apps stay contained. The studio widens no policy. ### 10. Limits on the device -- **The screen must be on.** Makepad paints only while Home has a drawable surface, and without one a readback does not fail; it waits. The renderer watches for Home going to the background, cancels the readback and fails the render with `not_foreground`. Rendering in the background needs makepad changes, a surfaceless or pbuffer EGL context and its drawable-surface checks; that stays an open question. +- **Home must have a drawable surface.** Makepad paints only while Home has a drawable surface, and without one a readback does not fail; it waits. The renderer watches for Home going to the background, cancels the readback and fails the render with `not_foreground`. Rendering in the background needs makepad changes, a surfaceless or pbuffer EGL context and its drawable-surface checks; that stays an open question. - **Size and memory.** A glance tile reads back from under 1 MB to about 8 MB of RGBA, depending on its height and the screen's density. A full app screen reads back 10–18 MB, which reserves 19–25 MiB of the 32 MiB readback budget once rows are padded, so renders run one at a time. PNGs for `view_image` stay under 5 MiB and are scaled down if needed. - **Vulkan.** On Android, readback is implemented for GLES only. A Vulkan build of Home would need Vulkan readback first. @@ -189,9 +229,9 @@ An octos change adds a media field to `peer/tool/result`, mapped onto octos's in ## Milestones -1. **Render on the phone.** Renderer, readback router, `studio.render`, developer build. Verified on a device: pixels and orientation, settle timing, cost of the repaint, the readback when Home leaves the front, and an image reaching DeepSeek V4 Flash (`shown_to_model`). +1. **Render on the phone.** Renderer, readback router, `studio.render`, developer build. Acceptance requires device evidence for: pixels and orientation, settle timing, cost of the repaint, the readback when Home leaves the front, and an image reaching DeepSeek V4 Flash (`shown_to_model`). 2. **Check and compare.** `card-studio` as a library, the new checks, comparison with a screenshot, the OCR choice. End-to-end: a screenshot of an app becomes a glance card refined in three iterations. -3. **Rules in Octoscript.** The studio runner and `mod.studio`; the `card-refine` template; image-to-card code generation; script-app output and developer install. +3. **Rules in Octoscript.** The studio runner and `mod.studio`; the `card-refine` template; image-to-card code generation. Local offline script-app output, instrumentation and developer install now have a Rust path; the template/adapter port and fault injection remain pending. Fresh-app functional and portrait visual checks have passed on OnePlus 6. 4. **Images in.** The image-generation backend; capture sessions on the ROM; clone a multi-screen app from captured screens. 5. **Normal mode.** Approvals for capture and install; MediaProjection on stock Android; background rendering if it is solved. @@ -199,9 +239,9 @@ An octos change adds a media field to `peer/tool/result`, mapped onto octos's in - One implementation of checks and comparison for desktop and phone, so a card judged on a Mac and on a phone gets the same report. - The design flow's Python shrinks to the Sketch kit and desktop-only tools as the image-to-card parts move to Octoscript and Rust. -- The shell gains a renderer that can show any card offscreen. It is a new attack surface for script cards, which is why studio renders run without side effects. +- The shell gains a renderer that can show any card offscreen. It is a new attack surface for script cards, which is why preview effects stay in disposable storage with external services disabled. - Rules and templates change on the phone without a build, under the toolbox's digest and budget rules. -- Developer mode gains the `studio.*` tools and one approval it never answers by itself. +- Developer mode gains the `studio.*` tools, with person-only approvals for capture sessions and any future guarded input that changes external state. - ADR 0002 is amended (its amendment of 2026-10-03): its §6 toolbox tools `card_render` and `card_critique_payload` become the `studio.*` host tools, and its §7 rule that the phone evaluates only while charging does not apply to renders the person starts. ## Open questions diff --git a/docs/architecture-walkthrough.md b/docs/architecture-walkthrough.md index 0e97e8e6..46b2988f 100644 --- a/docs/architecture-walkthrough.md +++ b/docs/architecture-walkthrough.md @@ -279,6 +279,43 @@ Read `mpsc` as a many-sender mailbox, `oneshot` as one correlated answer, and `w The broker's synchronous `bind`/`host_request` wrappers wait for channel replies: do not call such blocking interfaces from a rendering callback. Async model I/O can overlap across sessions; synchronous filesystem work still needs the existing worker boundary. One peer per app and account is an identity rule. The number of tasks and worker threads follows the runtimes and active operations above. +### Follow App Studio from the agent to a working app + +App Studio adds shell-owned tools to the existing system or app agent while developer mode covers that caller. Their [declarations and executor](../crates/shell/src/host_tools/studio.rs) use the same relay, audit and cancellation path as other host tools. A system call uses the workspace confirmed by `session/open`; an app call uses the broker-confirmed peer workspace, narrowed to its own `contexts//` for a human conversation. Arguments cannot replace that identity or choose an output directory. + +An agent can write a fresh `manifest.json` and `main.splash` with its normal file tools, then follow this sequence: + +| Tool | What it does | +| --- | --- | +| `studio.bundle_check {bundle_path}` | Copies bounded files from the conversation workspace, computes the digest and admits a private developer snapshot. It leaves the author's files unchanged. | +| `studio.open {bundle_path}` | Opens a visible preview with disposable app state; returns `instance_id`. | +| `studio.inspect {instance_id, offset?}` | Returns a PNG `path`, a compact page of widget selectors and checks, and `snapshot_path` for full diagnostic JSON. | +| `studio.input {instance_id, widget_id, action, …}` | Sends a real `tap`, `text` or `scroll` event to an inspected widget. Text uses `text`; scrolling uses `delta_y`. | +| `studio.close {instance_id}` | Closes the app and discards preview state. | +| `studio.install {bundle_path}` | Records a local developer install, visible in Home. Open it with `studio.open {app_id}` or its launcher tile; its own state survives closing and reopening. | + +The first full-app path accepts offline, storage-only `main.splash` bundles under `dev.studio.*`. These apps have no agent of their own, account access, network or host-service grants. A bundle may contain an original launcher icon, but in-screen resource routes are not supported yet. In [studio_bundles.rs](../crates/shell/src/host_tools/studio_bundles.rs), admission limits the bundle to 128 files/directories, eight directory levels and 2 MiB total, with at most 512 KiB per file and 64 KiB for `main.splash`. The resolved policy allows at most 1 MiB of private app storage, five million script instructions and a 16 MiB heap; a lower manifest limit remains lower. + +A developer install is separate from App Hub's signed catalog. Its host-private receipt binds the admitted bytes to the authoring app, account, session, context and `DevTag`. The tag identifies the developer profile and the activation that authorized it. Opening an installed app rechecks its receipt and digest. Ending that grant removes its launcher availability and stops its running instance. Another conversation cannot inspect, control or replace it. One installed app runs in one instance at a time, avoiding concurrent writes to its state. + +Now follow the Rust execution boundaries: + +1. The host executor runs bounded file reads and admission on a worker thread, then queues an `OpenSpec` or instrument request. It waits for a reply channel without blocking Makepad's UI thread. +2. The shell drains the launch queue into a normal window-manager client. [StudioModule](../crates/shell/src/studio/module.rs) creates the [StudioApp](../crates/shell/src/studio/apps.rs) widget and owns its shutdown. Splash evaluates the source after the private jail and admitted limits are installed. Preview writes stay in a disposable jail; installed writes stay in that app's persistent jail. +3. The UI thread builds a Makepad `WidgetTree` rooted at that app alone. Inspection reads its real rectangles and control state; input resolves a visible, enabled widget and follows the event path. Duplicate names have unique `selector` values. GPU readback uses the shell's shared ticket router; PNG compression runs on the heavy worker pool. The reply returns to the same agent call. + +The model receives at most 3,800 UTF-8 bytes from `studio.inspect`. `snapshot.widgets` lists visible non-Splash widgets with exact `selector` values; pass one as `studio.input.widget_id`, rather than guessing a button's painted label. Long text/value fields carry `text_truncated` or `value_truncated`. If `next_offset` is an integer, call inspect again with that `offset` to read the next page. Each call observes the current UI, so page through a stable app state. `snapshot.checks` summarizes the pass result and finding/error counts. + +The `snapshot_path` file contains the full original result, including all widgets, rectangles, geometry, tree and findings, plus the PNG path. This pretty-printed JSON is limited to 1 MiB and stored alongside the PNG in the caller's conversation workspace; `read_file` can retrieve bounded line ranges. Full diagnostics stay available without pushing selectors beyond the kernel's 4 KiB model-visible tool-output limit. + +The Android HTTP remote instrument is unavailable at this pin. Studio uses its underlying Makepad widget APIs in process, scoped to its own app. The current checks detect empty geometry, clipped text and small buttons; they do not prove the app's behavior or overall UX. App inspection returns `settled: false`: it captures a current frame without claiming that an arbitrary interactive script has finished changing. The agent can call `view_image` on the returned PNG and then test the app's behavior with further inputs. + +`studio.render` remains the separate L0 glance path: relative `source_path`, optional `data_path` and `dark`, with 16/32 KiB source/data limits. It uses the actual glance width and 72–440 point height bounds, a zero-quota disposable jail, and no network or host capabilities. Three matching readbacks after shader readiness produce `settled: true`; chat and image resources are explicitly unsupported. Both capture paths cap PNGs at 5 MiB and check cancellation, foreground state and the developer grant. UI requests have a 20-second deadline inside the host's 25-second wait and normal 30-second kernel call deadline. + +None of this creates an agent peer or assigns one Tokio task to each app. The existing agent invokes a host tool; Rust workers handle files and encoding, and the UI thread owns widgets and GPU submission. + +A fresh Task Planner authored by DeepSeek V4 Flash passed **129 tool calls on a physical OnePlus 6**: task entry/completion/filtering, disposable preview state, separate installed state, close/reopen, process restart, exact Chinese text input and scrolling. The harness did not modify app source or storage directly. Portrait app and keyboard visual review passed, with generous spacing and separate shell overlay/status-bar observations; malformed-storage/save-failure fault injection remains pending. See the [validation report](studio/oneplus6-validation.md), [ADR 0006](adr/0006-app-studio-on-the-phone.md) and the [fresh-app brief](studio/task-planner-brief.md). The `mod.studio` toolbox adapter, image generation/comparison, richer asset routes and public publishing are not implemented by this slice. + ## 11. Tests and remaining implementation [Broker tests](../crates/app-peers/tests/broker.rs) provide executable protocol examples: `a_persons_message_runs_while_the_system_agents_input_runs`, `a_lane_stop_leaves_the_other_lane_running`, `a_kernel_without_shared_history_is_refused_for_the_conversation`, and `removing_an_account_purges_its_recorded_peer_and_drops_the_record`. [Shell relay scenario tests](../crates/shell/src/host_tools/scenario_tests.rs) cover caller/tool/approval boundaries. Start with these when changing a conversation or executor path. diff --git a/docs/architecture-walkthrough.zh-CN.md b/docs/architecture-walkthrough.zh-CN.md index 5344511c..b2416151 100644 --- a/docs/architecture-walkthrough.zh-CN.md +++ b/docs/architecture-walkthrough.zh-CN.md @@ -273,6 +273,43 @@ flowchart LR Broker 同步 `bind` / `host_request` 会等待通道回复，不应在绘制回调中调用。不同 session 的异步模型 I/O 可重叠，同步文件工作仍走已有的工作线程边界。每应用/账号一个 peer 是身份规则；task 和线程数量由上表的 runtime 与活跃操作决定。 +### 从 Agent 调用跟到 App Studio 中可运行的应用 + +开发者模式覆盖当前调用方时，App Studio 会向已有的系统或应用 Agent 提供 Shell 所有的工具。[工具声明与执行器](../crates/shell/src/host_tools/studio.rs)沿用宿主工具的 relay、审计与取消路径。系统调用使用 `session/open` 确认的 workspace；应用调用使用 broker 确认的 peer workspace，人类对话进一步限定到自己的 `contexts//`。工具参数不能替换调用身份，也不能选择任意输出目录。 + +Agent 可先用普通文件工具从头编写 `manifest.json` 和 `main.splash`，再执行以下步骤： + +| 工具 | 行为 | +| --- | --- | +| `studio.bundle_check {bundle_path}` | 从当前对话 workspace 复制大小受限的文件，计算摘要，对宿主私有副本做开发者准入；作者的文件不变。 | +| `studio.open {bundle_path}` | 打开可见预览，使用可丢弃的应用状态，返回 `instance_id`。 | +| `studio.inspect {instance_id, offset?}` | 返回 PNG `path`、一页精简的控件 selector 与检查摘要，以及完整诊断 JSON 的 `snapshot_path`。 | +| `studio.input {instance_id, widget_id, action, …}` | 向检查到的控件发送真实 `tap`、`text` 或 `scroll` 事件。输入文字用 `text`，滚动用 `delta_y`。 | +| `studio.close {instance_id}` | 关闭应用，丢弃预览状态。 | +| `studio.install {bundle_path}` | 登记本地开发者安装，在 Home 中显示。随后可用 `studio.open {app_id}` 或启动器图块打开；应用自己的状态在关闭、重开后保留。 | + +首个完整应用路径接受 `dev.studio.*` 命名空间下的离线、仅存储权限 `main.splash` 包。应用本身没有 Agent、账户访问、网络或宿主服务授权。包中可携带原创启动器图标，但暂不支持屏幕内的资源加载路径。[studio_bundles.rs](../crates/shell/src/host_tools/studio_bundles.rs)把包限制为 128 个文件/目录、八层目录、总计 2 MiB；单文件最多 512 KiB，`main.splash` 最多 64 KiB。解析后的策略最多允许 1 MiB 私有应用存储、五百万脚本指令和 16 MiB 堆；清单中更低的限制仍然生效。 + +开发者安装独立于 App Hub 的签名目录。宿主私有收据把准入后的字节绑定到作者所属的应用、账户、session、context 与 `DevTag`。该标记记录开发者 profile 和授予权限的那次启用。打开已安装应用时会重新检查收据和摘要；授权结束后，应用从启动器可用列表移除，运行中的实例停止。另一个对话不能检查、操控或替换它。同一已安装应用只允许一个实例运行，避免同时写入其状态。 + +再沿着 Rust 的执行边界阅读： + +1. 宿主执行器在工作线程读取受限文件并做准入，然后将 `OpenSpec` 或检查请求入队。它通过回复 channel 等待结果，不阻塞 Makepad UI 线程。 +2. Shell 把启动队列中的请求变成普通窗口管理器 client。[StudioModule](../crates/shell/src/studio/module.rs)创建 [StudioApp](../crates/shell/src/studio/apps.rs) 控件并负责关闭。Splash 在私有 jail 与准入限制设置完毕后才求值源码。预览写入可丢弃的 jail；安装后的写入保存在该应用自己的持久 jail。 +3. UI 线程以该应用为根构建 Makepad `WidgetTree`。检查读取真实矩形与控件状态；输入先定位可见、启用的控件，再走事件路径。重名控件有唯一的 `selector` 值。GPU 读回复用 Shell 的 ticket 路由，PNG 压缩运行在重型工作线程池；结果回到原来的 Agent 工具调用。 + +`studio.inspect` 返回给模型的内容最多为 3,800 个 UTF-8 字节。`snapshot.widgets` 列出可见的非 Splash 控件及其准确 `selector`；把该值传给 `studio.input.widget_id`，不要猜测按钮显示文字就是控件 ID。较长的 text/value 会附带 `text_truncated` 或 `value_truncated` 标记。若 `next_offset` 是整数，用该值作为 `offset` 再调用 inspect 获取下一页。每次调用都观察当前 UI，因此翻页时应保持应用状态稳定。`snapshot.checks` 汇总是否通过以及发现项和错误数量。 + +`snapshot_path` 指向完整原始结果文件，包含所有控件、矩形、geometry、tree、检查详情和 PNG 路径。该 JSON 按行缩进，最多 1 MiB，与 PNG 一起存入调用方的对话 workspace；可用 `read_file` 按有限行数读取。这样既保留完整诊断，也避免重要 selector 被内核对模型工具结果的 4 KiB 截断隐藏。 + +当前 pin 在 Android 上没有 HTTP 远程 instrument。Studio 直接在进程内调用底层 Makepad 控件 API，范围限定为自身应用。现有检查能发现空几何信息、文字裁切与过小按钮，不能据此证明功能或整体 UX。应用检查返回 `settled: false`：它捕获当前帧，不声称任意交互脚本已经停止变化。Agent 可用 `view_image` 查看返回的 PNG，再发送输入测试实际行为。 + +`studio.render` 仍是独立的 L0 glance 路径：参数为相对 `source_path`、可选 `data_path` 和 `dark`，源码/数据上限为 16/32 KiB。它使用真实 glance 宽度与 72–440 点高度范围，临时 jail 配额为零，没有网络或宿主能力。着色器就绪且连续三次读回一致后返回 `settled: true`；聊天与图像资源明确返回不支持。两条截图路径都把 PNG 限制为 5 MiB，并检查取消、前台状态与开发者授权。UI 请求期限为 20 秒，处于宿主 25 秒等待和内核默认 30 秒工具期限之内。 + +这些操作不会创建 Agent peer，也不是每个应用对应一个 Tokio task。已有 Agent 调用宿主工具，Rust 工作线程负责文件与编码，UI 线程持有控件并提交 GPU 工作。 + +DeepSeek V4 Flash 从头编写的 Task Planner 已在**真实 OnePlus 6 上通过 129 次工具调用**，覆盖任务输入、完成与筛选、预览状态丢弃、独立安装状态、关闭重开、进程重启、准确的中文输入及滚动。测试工具没有直接修改应用源码或存储。竖屏应用与键盘视觉评审已通过；间距较宽，Shell 浮层和状态栏另有观察记录。损坏存储与保存失败的故障注入仍待完成，详见[验收报告](studio/oneplus6-validation.md)。见 [ADR 0006](adr/0006-app-studio-on-the-phone.md) 和[新应用需求](studio/task-planner-brief.md)。本次实现不包含 `mod.studio` 工具箱适配器、图像生成/比较、更完整的资源路径或公开发布。 + ## 11. 测试与尚待实现的部分 [Broker 测试](../crates/app-peers/tests/broker.rs)提供可执行协议示例：`a_persons_message_runs_while_the_system_agents_input_runs`、`a_lane_stop_leaves_the_other_lane_running`、`a_kernel_without_shared_history_is_refused_for_the_conversation`、`removing_an_account_purges_its_recorded_peer_and_drops_the_record`。[Shell relay 场景测试](../crates/shell/src/host_tools/scenario_tests.rs)覆盖调用方、工具与审批边界；修改对话或执行路径时从这些测试开始。 diff --git a/docs/studio/evidence/task-planner-keyboard.png b/docs/studio/evidence/task-planner-keyboard.png new file mode 100644 index 00000000..881eb4de Binary files /dev/null and b/docs/studio/evidence/task-planner-keyboard.png differ diff --git a/docs/studio/evidence/task-planner-titles.png b/docs/studio/evidence/task-planner-titles.png new file mode 100644 index 00000000..d2ae31c6 Binary files /dev/null and b/docs/studio/evidence/task-planner-titles.png differ diff --git a/docs/studio/oneplus6-validation.md b/docs/studio/oneplus6-validation.md new file mode 100644 index 00000000..7b25d67d --- /dev/null +++ b/docs/studio/oneplus6-validation.md @@ -0,0 +1,58 @@ +# App Studio: OnePlus 6 validation + +A fresh Task Planner authored by DeepSeek V4 Flash passed physical-device functional testing and manual app visual review in the isolated `dev.makepad.octosense.studio` package. This validates the implemented local, offline app path; it does not complete ADR 0006's image-generation and editable-workflow design. + +## Scope and provenance + +The model authored `DESIGN.md`, the bundle manifest, `main.splash` and an original icon. The shared kernel drove Studio tools; five recorded `view_image` calls returned `shown_to_model: true`, followed by continued model work. That establishes image delivery and continuation, not perfect visual understanding. The independent [device harness](../../tools/studio-flow-device-test.py) then exercised the resulting bundle without directly modifying its source or app storage. + +Evidence identifiers from the retained local run artifacts: + +| Record | Digest | +| --- | --- | +| Admitted bundle, BLAKE3 | `7892501e8b69d752b002130879dde11af8f7c7e7b7e620a6854b97892fc95f3b` | +| Model transcript, SHA-256 | `2d35937d1165d22e9f1ee8a1509064afe33df7f311ee7e2b80420ff4af34e219` | +| Generation receipt, SHA-256 | `495461c440b451047bd32114a47ba69e778a9cb25eafdb72c1f99094fb0da6db` | +| Acceptance contract, SHA-256 | `58a2091f2593a231982f755f643559b9dc29649b8d657b67c4edc3d3cbf715` | + +The final functional run is `adr0006-task-planner-acceptance-final`, using runtime commit `50639fa2528baa9e6263ba603fb66eb3505e72a1` after rebasing onto `bf3c2180` and fixing both review findings. APK SHA-256: `d57dc55f1380372a37228ba37a4c6f4d3221599d5566783ae85d5b2d7e9122f3`. The unchanged model-authored bundle passed all 129 calls again. Before rerunning, the previous test-only installed data directory was archived by rename; the harness then created fresh installed state through native input. Production Home and Bridge were not replaced. After the final checks, the original developer-profile files and screen timeout were restored, the temporary provider profile was removed, the test package was stopped and adbd remained unprivileged. + +The generation receipt is `adr0006-generated-app-final/generation-receipt.json`. Independent review replayed the model's successful write/edit operations and matched all four final authored files byte for byte. Generation was supervised across retries and runtime repairs. DeepSeek drove its own preview interactions; the separate Python acceptance harness drove the complete 129-call sequence. Manual visual review was performed by the coding agent. This is not evidence of one-shot autonomous model acceptance. + +Local checks passed: 863 shell tests, 89 app-peer unit/connector tests, 108 tools Python tests and 100 ROM tests with JDK 17 configured. The 15 optional real-kernel tests returned early without their executable and are not counted as end-to-end passes. Desktop default/mobile-apps and phone mobile-apps checks, both shell dependency graphs, setup consistency and native-app generation checks passed. The final Android build and physical rerun used the runtime commit above; later report-only changes do not alter it. + +Raw provider configuration, personal paths and device identifiers are not published here. + +## Functional result + +Version 2 and the final post-review rerun each completed **129 tool calls**. It verified task entry, completion, All/Active/Done filtering, disposable preview state, separate installed storage, installed close/reopen and process-restart persistence. Exact Chinese input survived reopening. Seven additional rows made a real scrollable list; the instrument reached the last row. The full diagnostic artifacts retained native widget values, geometry and screenshots while compact tool replies exposed usable selectors to the model. + +The first harness run stopped after ten calls with “Add did not clear the text input.” This was a harness assertion error: Makepad exposes the placeholder as display text, while the editable `value` was empty. The corrected assertion requires the actual TextInput value. Version 2 retained the same admitted bundle digest; no app-source change was used to bypass the failure. + +The model's earlier turn stopped at its 32-call host budget and explicitly left persistence, scrolling, Chinese glyphs and some visual checks unverified. The independent device run and manual review below supply those additional observations; the model's self-report alone is not the acceptance result. + +## Manual visual review + +Portrait app captures showed readable text, correct long-title wrapping, reachable bottom rows, readable Chinese glyphs, good app contrast and no unintended app overlap. A separate Android-wide screenshot showed the keyboard open: the input, Add button and filters remained above it, and the list viewport shrank correctly. App-only texture captures cannot establish keyboard presentation by themselves. + +The UI is usable, with generous spacing that leaves only about one task row above the keyboard. Two shell observations are separate from the generated app: the floating four-dot overlay sits near the Done filter without obscuring its label, and white status-bar content has low contrast against the light background. This is a successful scoped review, not a claim of flawless UI. + +The harness's machine-readable status remains `functional_passed_visual_review_required`; the subsequent manual review recorded here completes the observed portrait visual checks. Rotation/landscape behavior was not established. + +## Retained review captures + +These unmodified screenshots are from the final post-review device run described above. + +![Installed Task Planner with Chinese and wrapped long titles](evidence/task-planner-titles.png) + +![Android keyboard with Task Planner input and Add still visible](evidence/task-planner-keyboard.png) + +## Review result + +Independent source and evidence reviews scored the implemented developer flow **4.6/5 (A−)**. Review found and fixed close acknowledgement preceding cleanup, and input followed by inspection returning an older cached frame. Both regression tests and the final device rerun passed. This score applies to the implemented offline developer flow, not the whole ADR. + +## Remaining limits + +Malformed saved JSON preservation and save-failure reporting were not fault-injected. This run does not claim a full-app developer-revocation fault test; earlier L0 denial/background tests cover a narrower path. The app remains a local `dev.studio.*` developer install, with no publisher signature or public catalog admission. + +The `mod.studio` toolbox adapter, editable flow templates, image generation/comparison and rich in-screen asset routes remain unimplemented. See [ADR 0006](../adr/0006-app-studio-on-the-phone.md) for the implemented boundary and intended later work. diff --git a/docs/studio/task-planner-brief.md b/docs/studio/task-planner-brief.md new file mode 100644 index 00000000..e7b68233 --- /dev/null +++ b/docs/studio/task-planner-brief.md @@ -0,0 +1,156 @@ +# Task Planner: a fresh App Studio acceptance project + +This is a design brief for the model authoring a new app. It contains no app +implementation. Passing a preview of an existing card does not satisfy it. +The acceptance run starts with an empty project in the test package, records +the model's tool calls and resulting bundle, and exercises that bundle through +Makepad's actual widget input path. + +## Brief to give the authoring model + +Create **Task Planner**, a small offline phone app for keeping a short personal +task list. Its app id is `dev.studio.task-planner`. Build a fresh contained Splash +app (`main.splash`), not an L0 glance card or a modified existing app example. +Use the installed Studio tool schemas exactly; do not invent tool names, +widget methods, syntax, host services or a successful tool result. + +Design a calm, readable portrait screen with a warm neutral background, dark +text, one blue accent and generous spacing. Put the title and remaining-task +count at the top, a text field and Add button underneath, then All / Active / +Done filters and a scrollable task list. Use real editable and tappable +widgets. Show completion with both a readable label/state and a visual change, +not color alone. Keep controls at least 44 logical pixels high. The layout +must fit the current device, keep long titles readable and remain usable with +the keyboard open. Draw a simple original checklist icon as local vector +artwork. No generated raster design, borrowed screenshots, network artwork or +existing A2 app/card/assets are needed. + +A new app opens with no tasks, `0 active`, All selected, and `No tasks yet`. +Typing a nonblank title and pressing Add creates one active task, clears the +field and updates the count. Blank input creates nothing and explains that a +title is required. Completion is reversible. All shows every task, Active +shows incomplete tasks, and Done shows completed tasks; a filter with no +matches shows a short empty-state message. Filters do not change task data. +Titles and completion survive closing and reopening the app, including a +restart of the test Home process. On reopen, select All. There is no login, +network request, AI feature in the resulting planner, or permission request. + +The only app capability is `storage`. Keep task data in the app's own storage +jail, separate from the Studio project's source and the agent conversation. +Persist a versioned JSON document in `tasks.json`: + +```json +{"schema":1,"next_id":1,"tasks":[]} +``` + +Each task is `{ "id": , "title": , "completed": + }`; increment `next_id` after insertion. The filter is session UI +state. Do not silently overwrite a malformed saved document: show a useful +storage error and preserve its bytes. Surface a failed save rather than +claiming that the change was persisted. Use only storage/error APIs supported +by the pinned runtime. + +Give the input, Add button and filter controls stable source names: +`task_title`, `add_task`, `filter_all`, `filter_active`, `filter_done`. +Name the scrollable list `task_list`, and status and validation text +`task_count` and `task_error`. Task titles +must appear as native widget text. Each task has a visible `Complete` or +`Reopen` control associated with that task; the test can resolve its current +native widget id from inspection. These names are test selectors, not a +requirement to invent dynamic ids or custom runtime APIs. + +Before implementation, write a short `DESIGN.md` outside `bundle/` stating the +screen layout, states, data ownership and supported runtime APIs. Then author +`bundle/manifest.json`, `bundle/main.splash` and `bundle/assets/icon.svg` from +scratch. Include other metadata only when the real local bundle checker +requires it. Do not invent publisher identity, signatures, public submission, +or platform claims. Keep source notes, logs, runtime state and test evidence +outside the bundle. + +The local developer checker may compute integrity and stamp its own immutable +snapshot; it must leave the authored source unchanged. A developer check is +not publisher signing. Check the bundle, open it in Studio, inspect its real widget tree and image, +and drive its controls through Studio's actual input tool. Inspect after each +state change. `studio.inspect` returns a compact page in `snapshot.widgets`; +if `next_offset` is an integer, inspect the same instance with that `offset` +until it is null. Use the exact returned `selector` for `widget_id`; truncated +`text` or `value` is explicitly flagged and is not the complete field. +`snapshot_path` names the complete JSON inspection artifact relative to your +conversation workspace, and `path` names its PNG. The functional harness reads +the bounded full artifact so pagination cannot hide missing controls or +failed layout checks. The model should use selector pages for interaction; +reading a large artifact through a text tool is still subject to that tool's +output limit. Use `view_image` on the PNG for visual review. +Repair source errors or UX failures, recheck the new revision, +and reopen that revision before testing it. Preview state and installed app +state are separate; a preview does not prove installed-app persistence. +Leave final installation to the acceptance harness after your preview checks; +close the preview and return the final checked digest. The harness installs +into the developer test profile and exercises persistence through the same +Studio tools. Do not prepopulate installed task data. Never replace the +installed production Home or Bridge. + +Report which checks ran and the final immutable bundle revision. Report +unavailable tools, errors and unfinished steps directly. A returned filename +or `settled: true` alone does not prove a readable or functional app. Leave +publication unsigned and local; the requested result is a working developer +app, not a public App Hub submission. + +## Acceptance sequence + +The machine-readable contract is +[`task-planner-acceptance.json`](../../tools/studio-flow/task-planner-acceptance.json). +Run it against the model's final bundle, not a separately hand-authored +replacement. Preserve the generation transcript and source revision with the +receipt; the functional harness cannot prove authorship by itself. + +1. Inspect the empty screen and capture it. Check that the input, Add and three + filters are visible, enabled where appropriate, and at least 44 logical + pixels high. Blank Add must leave zero tasks and show validation. +2. Through native text input and button clicks, add `Buy oats`, `Call Sam`, + and `Read ten pages`. Verify three task titles and `3 active`. +3. Complete `Call Sam`; verify `2 active`, All still contains all three, + Active contains only Buy oats and Read ten pages, and Done only Call Sam. +4. Reopen Call Sam from Done; Done becomes empty and the count returns to + three. Return to All, then complete Call Sam again for persistence testing. +5. Close and reopen the preview: its disposable storage must reset to the + empty state. Developer-install the checked revision and open it by app id. + Repeat the three-task sequence in the installed app, then close/reopen it + and force-stop/relaunch the separate test Home package. Verify all three + titles, Call Sam's completed state and `2 active` after each installed-app + restart. Installed storage must not inherit preview test data. +6. Through native text input, add `测试中文任务：买燕麦`, reopen the installed + app and verify the exact title and `3 active`. Capture the typed, added + and reopened states for glyph review: correct widget text does not prove + correct font rendering. Then add a long title, inspect wrapping and scroll + to the final item. Verify + typing remains possible with the keyboard open. Review actual native + captures for clipping, overlap, contrast and top-to-bottom orientation. +7. In an isolated copy of the app data only, exercise malformed saved JSON and + a save failure if the test API can create that condition. Preserve existing + data and mark any unsupported fault-injection check unverified. + +The overall result is incomplete if model authoring, actual input behavior, +installed-app persistence or visual review is missing. A developer bundle +check is not a full App Hub publishing approval. + +## Existing flow and runtime boundaries + +The procedure follows Design Flow's +[text-brief flow](https://github.com/OctoSense-org/OctoScript-App-Design-Flow/blob/0e59346e810ed694702b1df48f4283dc8104358c/flows/script-app/FLOW.md) +and [script API](https://github.com/OctoSense-org/OctoScript-App-Design-Flow/blob/0e59346e810ed694702b1df48f4283dc8104358c/docs/SCRIPT-API.md). +That revision pins the same OctoScript-Makepad release as this shell's +`native-runtime.lock.json`. Use the pinned implementation when a historical +API description disagrees with source. + +Desktop tests use an owned hidden `card-host --remote` and real `/snap`, +`/click`, `/t` and `/g` operations. The current Android build has no HTTP +remote server. Its Studio integration must call Makepad's underlying widget +inspection and event dispatch on the UI thread; desktop captures do not count +as phone evidence. The earlier L0 `studio.render` probe verifies offscreen +rendering only. + +The existing toolbox library runs declared `mod.research` methods. It does +not yet provide a `mod.studio` authoring adapter. This brief and test contract +are inputs to the Studio flow, not a claim that an executable OctoScript +workflow template has already been installed. diff --git a/phone/README.md b/phone/README.md index 2a3d5b54..d65718fd 100644 --- a/phone/README.md +++ b/phone/README.md @@ -84,21 +84,25 @@ signer, kept outside the checkout. Use the pinned packager, not upstream's: it carries this app's Java activity ([docs/build-tool.md](docs/build-tool.md)). Signing, receipts and the ROM variant: [rom/docs/home-build.md](../rom/docs/home-build.md). -**Package name.** Home's application ID is `dev.makepad.octosense`, in both -the standalone and the ROM variant. A phone running the OctoSense ROM already -has that ID, signed with the platform key, so a development build cannot -replace it. To install a test build beside it, call the packager with another -package name, from `phone/`: - -```sh -../.sources/makepad/target/release/cargo-makepad makepad android \ - --sdk-path=/path/to/makepad-android \ - --package-name=dev.makepad.octosense.scriptapps \ - build -p octosense-home --release -``` - -`run` in place of `build` also installs and starts it; address it with its own -name, for example `adb shell am start -n dev.makepad.octosense.scriptapps/.MakepadApp`. +**Developer options and a test package.** `--development` selects signing +only. Add `--dev-mode` to compile Settings' developer options; this preserves +`--release` optimization and does not turn developer mode on. Normal builds +omit the feature. The build receipt records `development`, `dev_mode` and +`home_package`; ROM staging refuses developer-enabled artifacts. + +Home normally uses `dev.makepad.octosense`. To build a test Home beside the +installed app, add `--package-name dev.makepad.octosense.studio` to the +standalone command above. The default output becomes +`rom/out/home/standalone-dev-mode/dev.makepad.octosense.studio/` when both options +are used. Install only that test Home APK; the accompanying Bridge retains its +existing package ID and must not replace the device's Bridge. The separate +Home does not receive privileged ROM services without satisfying their existing +package/signature checks. + +The new build-plan and receipt paths are unit tested and dry-run checked. +APK and device validation remain separate: [App Studio milestone 1](../docs/adr/0006-app-studio-on-the-phone.md) +is in progress; the full phone authoring loop is not yet available. +The [studio device probe](../tools/studio-device-probe.py) checks denial, light/dark PNG output and background cancellation in that separate package. It needs a `--dev-mode` APK built with `MAKEPAD_FORCE_DEBUGGABLE=1` so `run-as` can provision test fixtures. The probe restores developer settings and returns to Home; it neither installs APKs nor tests model image delivery. Resolve startup permission prompts before running the probe; rendering does not need location access, which can remain denied. Device execution remains **unverified** until its evidence is recorded. **OpenHarmony:** `python3 rom/scripts/build-home-ohos.py --deveco-home ... --packager ... --signing-config ...` builds a normal OpenHarmony app with an diff --git a/phone/README.zh-CN.md b/phone/README.zh-CN.md index 71c25141..c6f21fc8 100644 --- a/phone/README.zh-CN.md +++ b/phone/README.zh-CN.md @@ -48,16 +48,12 @@ rom/scripts/build-home.sh --variant standalone --development \ 需要一个 `cargo-makepad` 的 Android SDK/NDK 目录（`cargo-makepad makepad android --sdk-path= install-toolchain`）、带 platform 35 和 build-tools 35.0.0 的 Android SDK、完整的 JDK 17+ 以及 Gradle 8.11.1；脚本不会安装其中任何一项。加上 `--dry-run` 可打印构建计划；发布版请把 `--development` 换成指向现有签名者的 `--sign-key` 和 `--sign-cert`，签名文件放在检出目录之外。请使用锁定版本的打包工具，而不是上游的：它带有本应用的 Java activity（[docs/build-tool.md（英文）](docs/build-tool.md)）。签名、回执和 ROM 变体见 [rom/docs/home-build.md（英文）](../rom/docs/home-build.md)。 -**包名。** Home 的应用 ID 是 `dev.makepad.octosense`，独立版和 ROM 版相同。运行 OctoSense ROM 的手机上已经有这个 ID，并由平台密钥签名，因此开发版无法替换它。要在旁边安装测试版，请在 `phone/` 中以另一个包名直接调用打包工具： +**开发者选项与测试包。** `--development` 只选择签名方式。另加 `--dev-mode` 才会编译设置中的开发者选项；仍使用 `--release` 优化，也不会自动开启开发者模式。默认构建不包含该 feature。构建回执记录 `development`、`dev_mode` 和 `home_package`，ROM 暂存会拒绝包含开发者模式的产物。 -```sh -../.sources/makepad/target/release/cargo-makepad makepad android \ - --sdk-path=/path/to/makepad-android \ - --package-name=dev.makepad.octosense.scriptapps \ - build -p octosense-home --release -``` +Home 默认使用 `dev.makepad.octosense`。要在已安装的应用旁构建测试 Home，请给上面的独立版命令加上 `--package-name dev.makepad.octosense.studio`。同时使用两个选项时，默认输出为 `rom/out/home/standalone-dev-mode/dev.makepad.octosense.studio/`。只安装测试 Home APK；配套 Bridge 保持原包名，不能替换设备上已安装的 Bridge。独立测试 Home 仍须满足现有包名与签名检查，才可访问 ROM 的特权服务。 -把 `build` 换成 `run` 会同时安装并启动它；用它自己的名字访问，例如 `adb shell am start -n dev.makepad.octosense.scriptapps/.MakepadApp`。 +新增构建计划和回执路径已通过单元测试与 dry-run 检查；APK 和设备验证另行进行。[App Studio 里程碑 1](../docs/adr/0006-app-studio-on-the-phone.md) 正在实现，完整的手机应用创作循环尚不可用。 +[Studio 设备探针](../tools/studio-device-probe.py) 在该独立测试包中检查权限拒绝、浅色/深色 PNG 输出和切入后台后的取消。它需要使用 `MAKEPAD_FORCE_DEBUGGABLE=1` 构建的 `--dev-mode` APK，让 `run-as` 能写入测试夹具。探针会恢复开发者设置并返回主屏；它不安装 APK，也不测试模型能否收到图片。运行前须先处理首次启动的权限提示；渲染不需要定位权限，可以保持拒绝。在记录设备证据前，这些设备检查仍属**未验证**。 **OpenHarmony：** `python3 rom/scripts/build-home-ohos.py --deveco-home ... --packager ... --signing-config ...` 使用现有的 DevEco 签名配置构建普通的 OpenHarmony 应用（[rom/docs/home-build.md（英文）](../rom/docs/home-build.md#openharmony-home)）。**iOS 模拟器：** 在 `phone/` 中运行 `../.sources/makepad/target/release/cargo-makepad makepad apple ios --org=dev.makepad --app=octosense run-sim -p octosense-home --features mobile-only`。两者都不在 CI 中构建。 diff --git a/rom/README.md b/rom/README.md index 83f36fcc..374bf692 100644 --- a/rom/README.md +++ b/rom/README.md @@ -66,6 +66,8 @@ python3 rom/scripts/stage-home.py # verify the receipt, copy the rom/scripts/stage-forks.sh /path/to/lineage-tree # apply vendor/octosense and stage the Quickstep and SystemUI forks ``` +**App Studio development builds.** Home’s build wrapper accepts `--dev-mode` independently of the `--development` signing option. Standalone tests can use `--package-name dev.makepad.octosense.studio`; only Home is renamed, so keep the installed Bridge. Developer-enabled receipts cannot be staged or published as ROM releases. See the [build contract](docs/home-build.md#android-builds) and [in-progress milestone](../docs/adr/0006-app-studio-on-the-phone.md). Build-plan tests do not establish device operation. + Then, on the host, `scripts/run-rom-rootfs.sh` enters the chroot (under `OCTOSENSE_BUILD_ROOT`, default `/home/ubuntu/octosense-adr0001`) and runs `build-rom.sh preflight`, `bacon` (the full signed build) or `module `. diff --git a/rom/README.zh-CN.md b/rom/README.zh-CN.md index 97d84a89..b884d68a 100644 --- a/rom/README.zh-CN.md +++ b/rom/README.zh-CN.md @@ -44,6 +44,8 @@ python3 rom/scripts/stage-home.py # verify the receipt, copy the rom/scripts/stage-forks.sh /path/to/lineage-tree # apply vendor/octosense and stage the Quickstep and SystemUI forks ``` +**App Studio 开发构建。** Home 构建脚本支持 `--dev-mode`，与只控制签名的 `--development` 独立。独立测试可使用 `--package-name dev.makepad.octosense.studio`；只改变 Home 包名，保留设备上的 Bridge。包含开发者模式的回执不能进入 ROM 暂存或发布。参见[构建约定](docs/home-build.md#android-builds)与[进行中的里程碑](../docs/adr/0006-app-studio-on-the-phone.md)。构建计划测试不代表设备运行已验证。 + 然后在主机上，`scripts/run-rom-rootfs.sh` 进入 chroot（位于 `OCTOSENSE_BUILD_ROOT` 下，默认 `/home/ubuntu/octosense-adr0001`），并运行 `build-rom.sh preflight`、`bacon`（完整签名构建）或 `module `。它要求 `build-rom.sh` 位于构建根目录下的 `exports/build-rom.sh`，并由 systemd 启动；该 unit 和主机配置不在本仓库中。`scripts/make-keys.sh` 一次性生成签名密钥；`scripts/release.sh ` 从 `~/.config/octosense/build.env` 中指定的主机取回构建完成的镜像。签名、构建回执和 ROM 版本：[docs/home-build.md（英文）](docs/home-build.md)。 刷写方式： diff --git a/rom/docs/home-build.md b/rom/docs/home-build.md index 3a22f118..52a0d646 100644 --- a/rom/docs/home-build.md +++ b/rom/docs/home-build.md @@ -121,8 +121,28 @@ Each APK build exports `OctoSenseHome.apk`, `OctoSenseBridge.apk` and `build.jso under `out/home/standalone/` or `out/home/rom/`. Home and Bridge are signed together and their certificate digests must match. The receipt records source revision/dirty state, runtime/patch/native-app inputs, APK hashes and certificate -digests. Existing application IDs, signature guards and platform imports are -unchanged. The build does not create or migrate signing keys. +digests. The default application IDs, signature guards and platform imports are +unchanged; standalone Home can use the test identity described below. The build does not create or migrate signing keys. + +**Developer options and a separate test Home.** `--development` selects only +the signing keystore. `--dev-mode` separately compiles the existing Home +`dev-mode` feature, enabling Settings to offer developer options; it does not +activate them or change release optimization. Default builds omit that feature. +Both values are recorded in `build.json`. Developer-enabled builds default to +`out/home/-dev-mode/`, and ROM staging/publication refuses them. + +For testing beside the installed Home, add +`--package-name dev.makepad.octosense.studio` to a standalone build. The wrapper +forwards this identity to the packager and verifies the resulting APK identity. +The default output then has an additional `dev.makepad.octosense.studio/` +subdirectory. Only Home is renamed: the Bridge APK keeps its existing ID and +must not replace the phone's installed Bridge. A test Home does not acquire +ROM services merely by changing its package name; their signature and package +guards remain in force. A ROM build cannot override its Home identity. + +These options have unit and dry-run coverage. An APK build and assigned-device +validation are separate checks; [ADR 0006](../../docs/adr/0006-app-studio-on-the-phone.md) +tracks the in-progress studio milestone and its remaining device acceptance. **The octos kernel.** Every APK carries Home's octos kernel as `lib/arm64-v8a/liboctos.so` (an Android app may exec only from its native diff --git a/rom/scripts/build-home.py b/rom/scripts/build-home.py index fdaea63c..d83a143c 100644 --- a/rom/scripts/build-home.py +++ b/rom/scripts/build-home.py @@ -24,6 +24,7 @@ # framework checkouts (.sources/, tools/setup.py) are at its root. REPO = ROOT.parent HOME = REPO / "phone" +HOME_PACKAGE = "dev.makepad.octosense" def _load_kernel_tool(): @@ -59,6 +60,8 @@ def arguments(argv=None): p.add_argument("--sign-key", type=Path, help="Existing signer PKCS#8 key; never generated by this command") p.add_argument("--sign-cert", type=Path, help="Existing signer X.509 certificate") p.add_argument("--development", action="store_true", help="Standalone only: sign both APKs with Makepad's development keystore") + p.add_argument("--dev-mode", action="store_true", help="Compile Home with developer options; independent of signing and disabled unless explicitly enabled") + p.add_argument("--package-name", default=HOME_PACKAGE, help="Standalone Home package ID for side-by-side testing; does not rename the Bridge APK") p.add_argument("--version-code", default="auto") p.add_argument("--output", type=Path) p.add_argument("--offline", action="store_true") @@ -73,13 +76,20 @@ def arguments(argv=None): p.error("--development is only for standalone builds without release keys") if not args.development and not args.sign_key: p.error("Supply the existing signer with --sign-key/--sign-cert, or use --development for standalone testing") + if not re.fullmatch(r"[A-Za-z][A-Za-z0-9_]*(?:\.[A-Za-z][A-Za-z0-9_]*)+", args.package_name): + p.error("--package-name must be a dotted Android application ID") + if args.variant == "rom" and args.package_name != HOME_PACKAGE: + p.error("--package-name overrides are only for standalone Home test builds") if args.version_code != "auto" and (not args.version_code.isdigit() or not 0 < int(args.version_code) < 2100000000): p.error("--version-code must be auto or a positive Android version code") for name in ("sdk", "android_sdk", "gradle_home", "java_home", "packager", "sign_key", "sign_cert", "output", "octos_kernel"): if getattr(args, name): setattr(args, name, Path(getattr(args, name)).resolve()) args.java_home = args.java_home or args.sdk / "openjdk" - args.output = args.output or ROOT / "out/home" / args.variant + output = ROOT / "out/home" / (args.variant + ("-dev-mode" if args.dev_mode else "")) + if args.package_name != HOME_PACKAGE: + output /= args.package_name + args.output = args.output or output return args @@ -103,6 +113,10 @@ def build_plan(args): gradle = [str(HOME / "android/gradlew"), "--no-daemon", ":contracts:exportHomeContracts", ":system-bridge:assembleRelease"] packager = args.packager or sources / "makepad/target/release/cargo-makepad" android = [str(packager), "makepad", "android", f"--sdk-path={args.sdk}", "--abi=aarch64", f"--version-code={args.version_code}", "--no-sign", "build", "-p", "octosense-home", "--release", "--locked"] + if args.dev_mode: + android += ["--features", "dev-mode"] + if args.package_name != HOME_PACKAGE: + android.insert(android.index("build"), f"--package-name={args.package_name}") if args.offline: cargo.append("--offline") gradle.append("--offline") @@ -152,12 +166,21 @@ def certificate_digest(apksigner, apk, env): return match[1] +def home_version(badging, expected_package): + """Refuse an APK with the wrong identity before recording the build.""" + package = re.search(r"^package: name='([^']+)'.*versionCode='([0-9]+)'", badging, re.MULTILINE) + if not package or package[1] != expected_package: + raise RuntimeError(f"Home APK identity does not match requested package {expected_package}") + return int(package[2]) + + def main(argv=None): args = arguments(argv) plan = build_plan(args) kernel = kernel_plan(args)[1] if args.dry_run: print(json.dumps({"variant": args.variant, "development": args.development, + "dev_mode": args.dev_mode, "home_package": args.package_name, "output": str(args.output), "octos_kernel": str(kernel) if kernel else None, "android_env": {"MAKEPAD_ANDROID_EXTRA_LIBS": extra_libs(kernel)}, @@ -229,8 +252,9 @@ def main(argv=None): raise RuntimeError("Home and Bridge must have the same signing identity") aapt = args.android_sdk / "build-tools/35.0.0/aapt2" badging = subprocess.check_output([str(aapt), "dump", "badging", str(args.output / "OctoSenseHome.apk")], env=env, text=True) - version = re.search(r"versionCode='([0-9]+)'", badging).group(1) + version = home_version(badging, args.package_name) receipt = {"schema_version": 1, "variant": args.variant, "development": args.development, + "dev_mode": args.dev_mode, "home_package": args.package_name, "home_version_code": int(version), "source_revision": subprocess.check_output(["git", "-C", str(REPO), "rev-parse", "HEAD"], text=True).strip(), "source_dirty": bool(subprocess.check_output(["git", "-C", str(REPO), "status", "--porcelain"], text=True)), diff --git a/rom/scripts/stage-home.py b/rom/scripts/stage-home.py index ec66e1a5..8e88545a 100644 --- a/rom/scripts/stage-home.py +++ b/rom/scripts/stage-home.py @@ -34,6 +34,10 @@ def verify(build): receipt = json.loads((build / "build.json").read_text()) if receipt.get("schema_version") != 1 or receipt.get("variant") != "rom" or receipt.get("development") is not False: raise ValueError("ROM staging requires a ROM build signed with the existing platform certificate") + if receipt.get("dev_mode", False) is not False: + raise ValueError("ROM staging refuses Home builds with developer mode compiled in") + if receipt.get("home_package", "dev.makepad.octosense") != "dev.makepad.octosense": + raise ValueError("ROM staging refuses a Home test package") certificates = set() for name in (HOME_APK, BRIDGE_APK): item = receipt["artifacts"][name] diff --git a/rom/tests/test_home_build.py b/rom/tests/test_home_build.py index a8e668f3..77901e29 100644 --- a/rom/tests/test_home_build.py +++ b/rom/tests/test_home_build.py @@ -54,6 +54,51 @@ def test_outputs_are_separate_and_builds_do_not_deploy(self): self.assertFalse(any("adb" in c or "fastboot" in c for c in commands)) self.assertFalse(any("OctoSense-mobile" in arg for c in commands for arg in c)) + def test_developer_options_are_separate_from_signing_and_keep_release_optimization(self): + for variant, signer in (("standalone", ["--development"]), + ("rom", ["--sign-key", "/keys/platform.pk8", "--sign-cert", "/keys/platform.x509.pem"])): + with self.subTest(variant=variant): + ordinary = self.args("--variant", variant, *signer) + developer = self.args("--variant", variant, *signer, "--dev-mode") + normal_plan, dev_plan = build.build_plan(ordinary), build.build_plan(developer) + self.assertNotIn("dev-mode", normal_plan[-1][1]) + self.assertEqual(dev_plan[:-1], normal_plan[:-1], "kernel, Bridge and packager are unchanged") + self.assertEqual(dev_plan[-1][1], normal_plan[-1][1] + ["--features", "dev-mode"]) + self.assertIn("--release", dev_plan[-1][1]) + self.assertNotEqual(ordinary.output, developer.output) + + def test_test_package_only_changes_home_and_is_recorded_in_dry_run(self): + package = "dev.makepad.octosense.studio" + argv = ["--variant", "standalone", "--development", "--dev-mode", "--package-name", package, + "--sdk", "/sdk with spaces", "--android-sdk", "/android", "--no-octos-kernel", "--dry-run"] + with contextlib.redirect_stdout(io.StringIO()) as output: + build.main(argv) + plan = json.loads(output.getvalue()) + self.assertTrue(plan["development"]) + self.assertTrue(plan["dev_mode"]) + self.assertEqual(plan["home_package"], package) + self.assertFalse(plan["installs_or_flashes"]) + self.assertTrue(plan["output"].endswith("standalone-dev-mode/" + package)) + commands = [step["argv"] for step in plan["steps"]] + self.assertIn("--package-name=" + package, commands[-1]) + self.assertLess(commands[-1].index("--package-name=" + package), commands[-1].index("build")) + self.assertFalse(any("--package-name=" + package in c for c in commands[:-1])) + + def test_bad_package_names_and_rom_overrides_are_refused(self): + for package in ("bare", "dev.test/escape", "dev.test..app", "dev.test.$value", "dev.1app"): + with self.subTest(package=package), contextlib.redirect_stderr(io.StringIO()), self.assertRaises(SystemExit): + self.args("--variant", "standalone", "--development", "--package-name", package) + with contextlib.redirect_stderr(io.StringIO()), self.assertRaises(SystemExit): + self.args("--variant", "rom", "--sign-key", "/keys/platform.pk8", "--sign-cert", "/keys/platform.x509.pem", + "--package-name", "dev.makepad.octosense.studio") + + def test_receipt_requires_the_requested_apk_identity(self): + badging = "package: name='dev.makepad.octosense.studio' versionCode='42' versionName='1'\n" + self.assertEqual(build.home_version(badging, "dev.makepad.octosense.studio"), 42) + for text in (badging, "no package in this APK"): + with self.assertRaises(RuntimeError): + build.home_version(text, "dev.makepad.octosense") + def test_existing_packager_skips_tool_compilation(self): args = self.args("--variant", "standalone", "--development", "--packager", "/tools/cargo-makepad", "--no-octos-kernel") plan = build.build_plan(args) @@ -143,6 +188,18 @@ def test_rejects_standalone_on_the_rom_channel(self): with self.assertRaises(ValueError): stage.verify(self.directory) + def test_rom_channel_rejects_developer_options_even_with_platform_signing(self): + self.receipt["dev_mode"] = True + self.write_receipt() + with self.assertRaisesRegex(ValueError, "developer mode"): + stage.verify(self.directory) + + def test_rom_channel_rejects_a_home_test_identity(self): + self.receipt["home_package"] = "dev.makepad.octosense.studio" + self.write_receipt() + with self.assertRaisesRegex(ValueError, "test package"): + stage.verify(self.directory) + def test_rejects_an_apk_replaced_after_signing(self): self.write_receipt() (self.directory / "OctoSenseHome.apk").write_bytes(b"another APK") diff --git a/tools/studio-device-probe.py b/tools/studio-device-probe.py new file mode 100644 index 00000000..cae9aaab --- /dev/null +++ b/tools/studio-device-probe.py @@ -0,0 +1,370 @@ +#!/usr/bin/env python3 +"""Exercise the developer-only studio renderer in an already installed test APK. + +Requires a --dev-mode build with MAKEPAD_FORCE_DEBUGGABLE=1. Never installs, +flashes, clears app data, changes the default Home, or accesses another package. +The supplied device must be unlocked and app startup permission prompts resolved. +Rendering needs no location grant; location can stay denied. This probes the real executor and GPU; +it does not test model calls, image delivery to a model, or ROM capture. +""" +import argparse +import hashlib +import json +from pathlib import Path, PurePosixPath +import re +import shlex +import struct +import subprocess +import sys +import threading +import time +import uuid + +ROOT = Path(__file__).resolve().parents[1] +DEFAULT_PACKAGE = "dev.makepad.octosense.studio" +FIXTURE = ROOT / "apps/calendar/host-service/resources/agenda.card" +PNG_LIMIT = 5 * 1024 * 1024 + + +class ProbeError(RuntimeError): + pass + + +def test_package(value): + # Keep this utility incapable of selecting Home, Bridge, or an unrelated app. + if not re.fullmatch(r"dev\.makepad\.octosense\.studio(?:\.[A-Za-z][A-Za-z0-9_]*)*", value): + raise argparse.ArgumentTypeError("use dev.makepad.octosense.studio or a child test package") + return value + + +def png_metadata(data): + if len(data) < 33 or len(data) > PNG_LIMIT or data[:8] != b"\x89PNG\r\n\x1a\n": + raise ProbeError("renderer output is not a bounded PNG") + if data[12:16] != b"IHDR": + raise ProbeError("PNG has no leading IHDR") + width, height = struct.unpack(">II", data[16:24]) + if not width or not height: + raise ProbeError("PNG has empty dimensions") + return {"width": width, "height": height, "bytes": len(data), + "sha256": hashlib.sha256(data).hexdigest()} + + +def output_name(value): + if not isinstance(value, str) or not re.fullmatch(r"\.studio-[0-9a-f-]+\.png", value): + raise ProbeError("renderer returned an unexpected output path") + return value + + +def display_metrics(size_text, density_text): + """The probe targets a portrait phone window; respect adb display overrides.""" + def effective(text, pattern): + values = dict(re.findall(pattern, text)) + value = values.get("Override", values.get("Physical")) + if value is None: + raise ProbeError("wm did not report usable display metrics") + return value + size = effective(size_text, r"(Physical|Override) size:\s*(\d+x\d+)") + density = int(effective(density_text, r"(Physical|Override) density:\s*(\d+)")) + width, height = map(int, size.split("x")) + scale = density / 160.0 + if not 0.5 <= scale <= 4 or width > height or width / scale - 40 < 240: + raise ProbeError("probe requires a portrait phone display with at least 240 logical pixels of card width") + return {"screen_width_px": width, "screen_height_px": height, "density_dpi": density, + "layout_scale": scale, "expected_card_width_px": width - 40 * scale, + "min_card_height_px": 72 * scale, "max_card_height_px": 440 * scale, + "width_tolerance_px": 4, "assumption": "portrait full-window layout; no Makepad DPI override"} + + +def check_png_geometry(png, metrics): + if abs(png["width"] - metrics["expected_card_width_px"]) > metrics["width_tolerance_px"]: + raise ProbeError(f"PNG width {png['width']} differs from display-derived glance width " + f"{metrics['expected_card_width_px']} (tolerance 4px)") + if not metrics["min_card_height_px"] - 1 <= png["height"] <= metrics["max_card_height_px"] + 1: + raise ProbeError("PNG height is outside the 72..440 logical-pixel glance bounds at display density") + + +def runtime_profile(lines, package): + """Use the app mount namespace's canonical path, never adb's substitute.""" + matches = [re.search(r"studio-test-profile: home=(\S+) canonical=(\S+) build=(\S+)", line) + for line in lines if "studio-test-profile:" in line] + if len(matches) != 1 or matches[0] is None: + raise ProbeError("expected one complete studio-test-profile diagnostic") + home, canonical, build = matches[0].groups() + allowed = {f"/data/data/{package}/files/.octosense", f"/data/user/0/{package}/files/.octosense"} + if home not in allowed or canonical not in allowed: + raise ProbeError("runtime profile path is outside this test package") + if build != "Development": + raise ProbeError("studio probe requires a Development build (compile with --dev-mode)") + return {"home": home, "canonical": canonical, "build": build} + + +class Device: + def __init__(self, adb, serial, package): + self.prefix = [adb, "-s", serial] + self.package = package + + def call(self, *args, data=None, check=True, timeout=15): + result = subprocess.run(self.prefix + list(args), input=data, capture_output=True, timeout=timeout) + if check and result.returncode: + raise ProbeError(f"adb {args[0]} failed: {result.stderr.decode(errors='replace').strip()}") + return result + + def shell(self, *args, **kwargs): + return self.call("shell", shlex.join(args), **kwargs) + + def private(self, script, **kwargs): + # Shell v2 preserves stdin EOF, binary stdout, stderr, and remote exit + # status. exec-out loses the status and can leave stdin writers waiting. + return self.call("shell", "-T", shlex.join(["run-as", self.package, "sh", "-c", script]), **kwargs) + + def read(self, path, limit=8192): + quoted = shlex.quote(path) + result = self.private(f"if [ ! -e {quoted} ]; then exit 44; fi; head -c {limit + 1} {quoted}", check=False) + if result.returncode == 44: + return None + if result.returncode: + raise ProbeError("private read failed: " + result.stderr.decode(errors="replace").strip()) + if len(result.stdout) > limit: + raise ProbeError(f"private result exceeds {limit} bytes") + return result.stdout + + def write(self, path, data): + self.private(f"umask 077; cat > {shlex.quote(path)}", data=data) + + def stop(self): + self.shell("am", "force-stop", self.package) + + def launch(self, spec): + config = json.dumps({"test_actions": [f"studio-render:{spec}"]}, separators=(",", ":")) + result = self.shell("am", "start", "-n", f"{self.package}/.MakepadApp", + "--es", "makepad.APP_CONFIG", config) + if b"Error:" in result.stdout or b"Error:" in result.stderr: + raise ProbeError(result.stdout.decode(errors="replace")) + deadline = time.monotonic() + 10 + while time.monotonic() < deadline: + result = self.shell("pidof", self.package, check=False) + pids = result.stdout.decode().strip().split() + if len(pids) == 1 and pids[0].isdigit(): + return pids[0] + time.sleep(0.05) + raise ProbeError("test activity did not start") + + +class AppLog: + """Stream this process's logs; never clear the device-wide log buffer.""" + def __init__(self, device, pid, background): + self.lines = [] + self.background_sent = False + self.background_error = None + self.denied = threading.Event() + self.process = subprocess.Popen(device.prefix + ["logcat", "--pid=" + pid, "-v", "raw"], + stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True) + def collect(): + for line in self.process.stdout: + # Store only probe diagnostics, not unrelated app/provider logs. + if "studio-test-" in line or "studio-render-geometry:" in line or "studio test:" in line: + self.lines.append(line.rstrip()) + if "studio test action requires developer mode" in line: + self.denied.set() + if background and not self.background_sent and "studio-test-started:" in line: + self.background_sent = True + try: + device.shell("input", "keyevent", "KEYCODE_HOME") + except Exception as error: + self.background_error = str(error) + self.thread = threading.Thread(target=collect, daemon=True) + self.thread.start() + + def close(self): + self.process.terminate() + try: + self.process.wait(timeout=3) + except subprocess.TimeoutExpired: + self.process.kill() + self.process.wait() + self.thread.join(timeout=16) + self.process.stdout.close() + + +def run_case(device, root, name, destination, timeout, metrics): + directory = f"{root}/{name}" + device.stop() + device.private(f"mkdir -m 700 -p {shlex.quote(directory)}") + device.write(directory + "/card.l0", FIXTURE.read_bytes()) + data = {"day": {"title": "Studio device probe", "as_of": "FIXTURE", + "pick1_title": "1. Top row", "pick1_body": "09:00", + "pick2_title": "2. Middle row", "pick2_body": "12:00", + "pick3_title": "3. Bottom row", "pick3_body": "17:00", + "summary": "End of fixture — no account data"}} + device.write(directory + "/data.json", json.dumps(data).encode()) + spec = {"workspace": directory, "source_path": "card.l0", "data_path": "data.json", "dark": name == "dark"} + device.write(directory + "/spec.json", json.dumps(spec).encode()) + pid = device.launch(directory + "/spec.json") + logs = AppLog(device, pid, name == "background") + record = {"case": name, "status": "failed", "pid": pid, "workspace": directory} + try: + deadline = time.monotonic() + timeout + result = None + while time.monotonic() < deadline: + raw = device.read(directory + "/studio-result.json") + if raw: + try: + result = json.loads(raw) + break + except json.JSONDecodeError: + pass # Writer may be between write and fsync. + if name == "denied" and logs.denied.is_set(): + break + time.sleep(0.05) + if name == "denied": + if not logs.denied.is_set() or device.read(directory + "/studio-result.json") is not None: + raise ProbeError("expected developer-mode denial without a result file") + record.update(status="passed", denial="developer mode required") + else: + if not isinstance(result, dict): + raise ProbeError("no complete studio-result.json before timeout") + record["reply"] = result + if name == "background": + record["home_sent_after_submit"] = logs.background_sent + if logs.background_error: + raise ProbeError(logs.background_error) + if result.get("ok") is True: + record["status"] = "inconclusive" + record["reason"] = "render completed before background cancellation; rerun required" + else: + error = result.get("error", {}) + if not logs.background_sent or error.get("kind") != "studio_render_failed" or not any( + word in error.get("message", "") for word in ("not_foreground", "studio_cancelled", "cancelled") + ): + raise ProbeError("expected cancellation after renderer submission") + record["status"] = "passed" + else: + if result.get("ok") is not True: + raise ProbeError(f"renderer failed: {result.get('error')}") + rendered = result.get("data", {}) + if rendered.get("settled") is not True: + raise ProbeError("renderer did not settle") + png = device.read(directory + "/" + output_name(rendered.get("path")), PNG_LIMIT) + if png is None: + raise ProbeError("renderer PNG is missing") + metadata = png_metadata(png) + if (metadata["width"], metadata["height"]) != (rendered.get("width"), rendered.get("height")): + raise ProbeError("reply dimensions differ from PNG header") + (destination / f"{name}.png").write_bytes(png) + record.update(png=metadata, expected_geometry=metrics) + check_png_geometry(metadata, metrics) + record["status"] = "passed" + if name in ("denied", "background") and record["status"] == "passed": + files = device.private(f"find {shlex.quote(directory)} -name '*.png' -print").stdout + if files.strip(): + raise ProbeError("denied/cancelled render left a PNG behind") + except Exception as error: + record.update(status="failed", reason=str(error)) + finally: + logs.close() + try: + record["profile"] = runtime_profile(logs.lines, device.package) + except ProbeError as error: + record.update(status="failed", reason=str(error)) + (destination / f"{name}.log").write_text("\n".join(logs.lines) + "\n") + (destination / f"{name}.json").write_text(json.dumps(record, indent=2) + "\n") + return record + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--serial", required=True, help="explicit adb serial of the authorized unlocked test device") + parser.add_argument("--package", type=test_package, default=DEFAULT_PACKAGE) + parser.add_argument("--adb", default="adb") + parser.add_argument("--output", type=Path, required=True, help="new local evidence directory") + parser.add_argument("--timeout", type=float, default=35, help="seconds per case (minimum 30)") + parser.add_argument("--wake", action="store_true", help="wake screen and dismiss a noncredential keyguard; never unlocks credentials") + parser.add_argument("--dry-run", action="store_true", help="print plan without contacting adb") + args = parser.parse_args() + if args.timeout < 30: + parser.error("--timeout must be at least 30 seconds") + plan = {"serial": args.serial, "package": args.package, "cases": ["denied", "light", "dark", "background"], + "fixture": str(FIXTURE.relative_to(ROOT)), "output": str(args.output), + "installs_or_flashes": False, "model_image_delivery": "not tested"} + if args.dry_run: + print(json.dumps(plan, indent=2)) + return 0 + args.output.mkdir(parents=True, exist_ok=False) + evidence = {**plan, "started_unix": int(time.time()), "cases": [], "status": "failed"} + device = Device(args.adb, args.serial, args.package) + saved = {} + modified = False + try: + if device.call("get-state").stdout.strip() != b"device": + raise ProbeError("selected device is unavailable") + # Verify run-as before changing anything. Optimized APKs need the explicit + # MAKEPAD_FORCE_DEBUGGABLE packaging opt-in, independent of dev-mode. + app_root = device.private("pwd").stdout.decode().strip() + if not PurePosixPath(app_root).is_absolute(): + raise ProbeError("run-as did not return an absolute app directory") + device.stop() + device.private("umask 077; mkdir -p files/.octosense files/studio-fixture") + home = device.private("cd files/.octosense && pwd -P").stdout.decode().strip() + for name in ("developer-profile", "dev-mode.json"): + path = f"{home}/{name}" + saved[path] = device.read(path, 64 * 1024) + modified = True + device.private("rm -f " + shlex.join(list(saved))) + evidence["device"] = {key: device.shell("getprop", prop).stdout.decode().strip() for key, prop in ( + ("model", "ro.product.model"), ("fingerprint", "ro.build.fingerprint"))} + permissions = device.shell("dumpsys", "package", args.package).stdout.decode(errors="replace") + evidence["startup_location_permissions"] = [line.strip() for line in permissions.splitlines() + if re.search(r"android\.permission\.ACCESS_(?:FINE|COARSE)_LOCATION: granted=", line)] + if args.wake: + device.shell("input", "keyevent", "KEYCODE_WAKEUP") + device.shell("wm", "dismiss-keyguard", check=False) + metrics = display_metrics(device.shell("wm", "size").stdout.decode(), + device.shell("wm", "density").stdout.decode()) + evidence["display"] = metrics + evidence["visual_inspection"] = "required: verify readable fixture text, top-to-bottom order, and light/dark appearance" + root = app_root + "/files/studio-fixture/" + uuid.uuid4().hex + evidence["fixture_workspace"] = root + evidence["fixture_sha256"] = hashlib.sha256(FIXTURE.read_bytes()).hexdigest() + denied = run_case(device, root, "denied", args.output, args.timeout, metrics) + evidence["cases"].append(denied) + if denied["status"] != "passed": + raise ProbeError("pre-grant denial/profile check failed; developer mode was not provisioned") + profile_id = denied["profile"]["canonical"] + evidence["runtime_profile"] = denied["profile"] + device.stop() + device.write(home + "/developer-profile", b"studio-device-probe\n") + device.write(home + "/dev-mode.json", json.dumps({"scope": "all", "origin": "settings", + "since": int(time.time()), "profile_id": profile_id}).encode()) + for name in ("light", "dark", "background"): + evidence["cases"].append(run_case(device, root, name, args.output, args.timeout, metrics)) + cases = evidence["cases"] + if all(case["status"] == "passed" for case in cases): + if cases[1]["png"]["sha256"] == cases[2]["png"]["sha256"]: + raise ProbeError("light and dark renders are identical") + evidence["status"] = "passed" + except Exception as error: + evidence["error"] = str(error) + finally: + if modified: + try: + device.stop() + for path, data in saved.items(): + if data is None: + device.private("rm -f " + shlex.quote(path)) + else: + device.write(path, data) + evidence["developer_state_restored"] = True + except Exception as error: + evidence.update(status="failed", restore_error=str(error)) + try: + device.shell("input", "keyevent", "KEYCODE_HOME") + except Exception as error: + evidence.update(status="failed", home_error=str(error)) + evidence["finished_unix"] = int(time.time()) + (args.output / "evidence.json").write_text(json.dumps(evidence, indent=2) + "\n") + print(json.dumps({"status": evidence["status"], "evidence": str(args.output / "evidence.json")}, indent=2)) + return 0 if evidence["status"] == "passed" else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/studio-flow-device-test.py b/tools/studio-flow-device-test.py new file mode 100644 index 00000000..af1c22ec --- /dev/null +++ b/tools/studio-flow-device-test.py @@ -0,0 +1,480 @@ +#!/usr/bin/env python3 +"""Exercise a real model-authored Task Planner through the Studio instrument. + +Only the separate developer test package is in scope. This harness never +writes application source or changes task storage to manufacture success. +""" +import argparse +import hashlib +import importlib.util +import json +from pathlib import Path +import re +import time +import uuid + +ROOT = Path(__file__).resolve().parents[1] +CONTRACT = ROOT / "tools/studio-flow/task-planner-acceptance.json" +_spec = importlib.util.spec_from_file_location("studio_device_probe", ROOT / "tools/studio-device-probe.py") +_probe = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(_probe) +ProbeError = _probe.ProbeError +Device = _probe.Device + + +def widgets(snapshot): + if not isinstance(snapshot, dict) or not isinstance(snapshot.get("widgets"), list): + raise ProbeError("inspect result needs a native widgets array") + return [node for node in snapshot["widgets"] if isinstance(node, dict) and node.get("visible") is True] + + +def node_text(node): + return str(node.get("text") or node.get("value") or "").strip() + + +def input_value(node): + """Read editable content, never the placeholder exposed as native text.""" + if "TextInput" not in str(node.get("type", "")) or not isinstance(node.get("value"), str): + raise ProbeError("selected TextInput must expose its actual string value") + return node["value"] + + +def bounds(node): + value = node.get("rect") + if not isinstance(value, list) or len(value) != 4 or not all(isinstance(n, (int, float)) for n in value): + raise ProbeError("native widget has no measured rectangle") + if value[2] <= 0 or value[3] <= 0: + raise ProbeError("native widget has empty bounds") + return value + + +def unique_node(snapshot, *, source_id=None, text=None, kind=None, enabled=True): + nodes = widgets(snapshot) + if source_id is not None: + nodes = [node for node in nodes if str(node.get("id", "")) == source_id + or str(node.get("id", "")).endswith(("/" + source_id, "." + source_id))] + if text is not None: + nodes = [node for node in nodes if node_text(node) == text] + if kind is not None: + nodes = [node for node in nodes if kind.lower() in str(node.get("type", "")).lower()] + if len(nodes) != 1: + raise ProbeError(f"selector must match one visible native widget, found {len(nodes)}: " + f"id={source_id!r} text={text!r} type={kind!r}") + node = nodes[0] + bounds(node) + if enabled and node.get("enabled") is not True: + raise ProbeError("selected native control is disabled") + return node + + +def task_action(snapshot, title, action): + title_node = unique_node(snapshot, text=title, enabled=False) + title_rect = bounds(title_node) + row_center = title_rect[1] + title_rect[3] / 2 + candidates = [] + for node in widgets(snapshot): + if node_text(node) != action or node.get("enabled") is not True: + continue + rect = bounds(node) + if rect[1] <= row_center <= rect[1] + rect[3]: + candidates.append(node) + if len(candidates) != 1: + raise ProbeError(f"cannot identify one {action} control beside task {title!r}") + return candidates[0] + + +def assert_tasks(snapshot, expected, all_titles): + shown = {node_text(node) for node in widgets(snapshot)} + actual = [title for title in all_titles if title in shown] + if set(actual) != set(expected): + raise ProbeError(f"visible tasks differ: expected {expected!r}, found {actual!r}") + + +def assert_count(snapshot, source_id, number): + node = unique_node(snapshot, source_id=source_id, enabled=False) + if not re.search(rf"\b{number}\s+active\b", node_text(node), re.IGNORECASE): + raise ProbeError(f"expected {number} active; native status is {node_text(node)!r}") + + +def verify_controls(snapshot, contract): + names = contract["selectors"] + checks = [] + for key in ("title_input", "add", "all", "active", "done"): + node = unique_node(snapshot, source_id=names[key]) + rect = bounds(node) + if rect[3] < contract["minimum_control_height_logical"]: + raise ProbeError(f"{names[key]} is only {rect[3]} logical pixels high") + checks.append({"selector": names[key], "widget_id": node["id"], "rect": rect}) + return checks + + +def safe_relative(value): + if not isinstance(value, str) or not value or value.startswith("/") or "\\" in value: + raise ProbeError("expected a relative workspace path") + if any(part in ("", ".", "..") for part in value.split("/")): + raise ProbeError("workspace path contains traversal or empty components") + return value + + +def full_inspection(device, workspace, instance, compact): + encoded = json.dumps(compact, ensure_ascii=False, separators=(",", ":")).encode("utf-8") + if len(encoded) > 3800: + raise ProbeError("inspect response exceeds the model-visible 3800-byte budget") + path = safe_relative(compact.get("snapshot_path")) + raw = device.read(workspace + "/" + path, 1024 * 1024) + if raw is None: + raise ProbeError("inspect returned a missing full snapshot artifact") + if len(raw) > 1024 * 1024: + raise ProbeError("full snapshot artifact exceeds 1MiB") + try: + full = json.loads(raw) + except (UnicodeDecodeError, json.JSONDecodeError) as error: + raise ProbeError("full snapshot artifact is not complete UTF-8 JSON") from error + if not isinstance(full, dict) or full.get("instance_id") != instance: + raise ProbeError("full snapshot artifact belongs to another instance") + for key in ("instance_id", "path", "width", "height", "settled"): + if key not in full or full[key] != compact.get(key): + raise ProbeError(f"full snapshot and compact response disagree on {key}") + widgets(full.get("snapshot")) + return full + + +class Studio: + def __init__(self, device, workspace, evidence_dir, timeout): + self.device, self.workspace, self.evidence_dir = device, workspace, evidence_dir + self.timeout = timeout + self.calls = [] + self.inspections = 0 + self.instance = None + self.snapshot = None + + def start(self): + import shlex + self.device.stop() + self.device.private("umask 077; mkdir -p " + shlex.join([ + self.workspace + "/requests", self.workspace + "/responses"])) + config_path = self.workspace + "/studio-flow-config.json" + self.device.write(config_path, json.dumps({"workspace": self.workspace}).encode()) + config = json.dumps({"test_actions": ["studio-flow:" + config_path]}, separators=(",", ":")) + launch = self.device.shell("am", "start", "-n", self.device.package + "/.MakepadApp", + "--es", "makepad.APP_CONFIG", config) + if b"Error:" in launch.stdout or b"Error:" in launch.stderr: + raise ProbeError("studio flow activity failed to start") + self.instance = None + self.snapshot = None + + def call(self, name, args): + import shlex + request_id = str(uuid.uuid4()) + request = {"name": name, "args": args} + payload = json.dumps(request).encode() + if len(payload) > 16 * 1024: + raise ProbeError("test request exceeds 16KiB") + temporary = self.workspace + "/requests/" + request_id + ".pending" + destination = self.workspace + "/requests/" + request_id + ".json" + self.device.write(temporary, payload) + self.device.private("mv " + shlex.join([temporary, destination])) + started = time.monotonic() + response = None + while time.monotonic() - started < self.timeout: + raw = self.device.read(self.workspace + "/responses/" + request_id + ".json", 1024 * 1024) + if raw: + try: + response = json.loads(raw) + break + except json.JSONDecodeError: + pass + time.sleep(0.1) + record = {"request_id": request_id, "request": request, "reply": response, + "elapsed_seconds": round(time.monotonic() - started, 3)} + self.calls.append(record) + (self.evidence_dir / "tool-calls.json").write_text(json.dumps(self.calls, indent=2) + "\n") + if not isinstance(response, dict): + raise ProbeError(f"{name}: no complete response before timeout") + if response.get("ok") is not True: + raise ProbeError(f"{name}: {response.get('error')}") + result = response.get("data") + if not isinstance(result, dict): + raise ProbeError(f"{name}: expected object result") + return result + + def open(self, **args): + result = self.call("studio.open", args) + instance = result.get("instance_id") + if not isinstance(instance, str) or not instance: + raise ProbeError("studio.open returned no instance_id") + self.instance = instance + return result + + def close(self): + if self.instance is not None: + self.call("studio.close", {"instance_id": self.instance}) + self.instance = None + self.snapshot = None + + def inspect(self, label): + compact = self.call("studio.inspect", {"instance_id": self.instance}) + result = full_inspection(self.device, self.workspace, self.instance, compact) + snapshot = result["snapshot"] + self.snapshot = snapshot + self.inspections += 1 + stem = f"{self.inspections:03d}-{label}" + (self.evidence_dir / f"{stem}.json").write_text(json.dumps(result, indent=2) + "\n") + (self.evidence_dir / f"{stem}.compact.json").write_text(json.dumps(compact, indent=2) + "\n") + path = safe_relative(result.get("path")) + png = self.device.read(self.workspace + "/" + path, _probe.PNG_LIMIT) + if png is None: + raise ProbeError("inspect returned a missing PNG") + metadata = _probe.png_metadata(png) + (self.evidence_dir / f"{stem}.png").write_bytes(png) + if metadata["width"] < 240 or metadata["height"] < 240: + raise ProbeError("full-app inspection PNG is implausibly small") + if (metadata["width"], metadata["height"]) != (result["width"], result["height"]): + raise ProbeError("inspection PNG dimensions disagree with its full snapshot") + checks = snapshot.get("checks") + if not isinstance(checks, dict) or checks.get("pass") is not True: + raise ProbeError(f"native layout checks did not pass: {checks}") + return snapshot + + def input(self, node, action="tap", text=None): + args = {"instance_id": self.instance, "widget_id": node.get("selector", node["id"]), "action": action} + if text is not None: + args["text"] = text + return self.call("studio.input", args) + + def tap(self, source_id, label): + self.input(unique_node(self.snapshot, source_id=source_id)) + return self.inspect(label) + + def add(self, title, selectors, label): + field = unique_node(self.snapshot, source_id=selectors["title_input"]) + self.input(field, "text", title) + self.inspect(label + "-typed") + return self.tap(selectors["add"], label + "-added") + + +def exercise(studio, contract, label): + names = contract["selectors"] + titles = contract["fixture_titles"] + snap = studio.inspect(label + "-empty") + controls = verify_controls(snap, contract) + assert_tasks(snap, [], titles) + assert_count(snap, names["count"], 0) + unique_node(snap, text="No tasks yet", enabled=False) + snap = studio.tap(names["add"], label + "-blank") + assert_tasks(snap, [], titles) + error = unique_node(snap, source_id=names["validation"], enabled=False) + if not node_text(error): + raise ProbeError("blank Add did not explain validation") + for index, title in enumerate(titles): + snap = studio.add(title, names, label + f"-task-{index + 1}") + assert_tasks(snap, titles[:index + 1], titles) + assert_count(snap, names["count"], index + 1) + if input_value(unique_node(snap, source_id=names["title_input"])) != "": + raise ProbeError("Add did not clear the text input") + studio.input(task_action(snap, contract["completed_title"], "Complete")) + snap = studio.inspect(label + "-completed") + assert_count(snap, names["count"], 2) + for filter_name in ("active", "done", "all"): + snap = studio.tap(names[filter_name], label + "-" + filter_name) + assert_tasks(snap, contract["expected_before_restart"][filter_name], titles) + snap = studio.tap(names["done"], label + "-done-before-reopen") + studio.input(task_action(snap, contract["completed_title"], "Reopen")) + snap = studio.inspect(label + "-task-reopened") + assert_tasks(snap, [], titles) + assert_count(snap, names["count"], 3) + snap = studio.tap(names["all"], label + "-all-before-persist") + studio.input(task_action(snap, contract["completed_title"], "Complete")) + snap = studio.inspect(label + "-persist-state") + assert_count(snap, names["count"], 2) + return {"controls": controls, "state": contract["expected_before_restart"]} + + +def assert_persisted(studio, contract, label): + snap = studio.inspect(label + "-all") + names = contract["selectors"] + assert_count(snap, names["count"], 2) + assert_tasks(snap, contract["fixture_titles"], contract["fixture_titles"]) + snap = studio.tap(names["done"], label + "-done") + assert_tasks(snap, [contract["completed_title"]], contract["fixture_titles"]) + studio.tap(names["all"], label + "-all-restored") + + +def exercise_multilingual(studio, contract): + title = contract["multilingual_title"] + names = contract["selectors"] + snapshot = studio.add(title, names, "multilingual") + unique_node(snapshot, text=title, enabled=False) + assert_count(snapshot, names["count"], 3) + studio.close() + studio.open(app_id=contract["app_id"]) + snapshot = studio.inspect("multilingual-reopened") + unique_node(snapshot, text=title, enabled=False) + assert_count(snapshot, names["count"], 3) + return {"status": "passed", "title": title, + "verified": "exact native text after input and installed-app reopen", + "glyph_review": "required; exact widget text alone does not prove readable glyphs"} + + +def keyboard_diagnostics(device): + """Record only visibility/ownership; app PNGs exclude the Android IME.""" + result = device.shell("dumpsys", "input_method", check=False) + text = result.stdout.decode(errors="replace") + shown = re.findall(r"\b(?:mInputShown|mIsInputViewShown)=(true|false)", text) + owners = re.findall(r"\bpackageName=([A-Za-z0-9_.]+)", text) + visible = bool(shown) and all(value == "true" for value in shown) + owned = device.package in owners + return {"status": "observed" if result.returncode == 0 and visible and owned else "unverified", + "input_shown": visible if shown else None, + "test_package_reported": owned, + "capture_scope": "app texture only; Android keyboard pixels are not captured", + "visual_keyboard_review": "not established by app PNG"} + + +def generation_receipt(path): + raw = path.read_bytes() + data = json.loads(raw) + if data.get("authorship") != "model" or not isinstance(data.get("model"), str) or not data["model"].strip(): + raise ProbeError("generation receipt must identify model authorship and model") + for field in ("bundle_digest", "transcript_sha256"): + if not isinstance(data.get(field), str) or not re.fullmatch(r"[0-9a-f]{64}", data[field]): + raise ProbeError(f"generation receipt needs a 64-character lowercase {field}") + return {key: data[key] for key in ("authorship", "model", "bundle_digest", "transcript_sha256")} | { + "receipt_sha256": hashlib.sha256(raw).hexdigest()} + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--serial", required=True) + parser.add_argument("--package", type=_probe.test_package, default=_probe.DEFAULT_PACKAGE) + parser.add_argument("--adb", default="adb") + parser.add_argument("--workspace", required=True, help="absolute authoring workspace inside the test package's files directory") + parser.add_argument("--bundle-path", required=True, help="model-authored bundle, relative to workspace") + parser.add_argument("--generation-receipt", type=Path, required=True) + parser.add_argument("--output", type=Path, required=True, help="new local evidence directory") + parser.add_argument("--timeout", type=float, default=40) + parser.add_argument("--dry-run", action="store_true") + args = parser.parse_args() + if args.timeout < 30: + parser.error("--timeout must be at least 30 seconds") + try: + safe_relative(args.bundle_path) + allowed = [f"/data/user/0/{args.package}/files/", f"/data/data/{args.package}/files/"] + if not any(args.workspace.startswith(prefix) for prefix in allowed): + raise ProbeError("workspace must belong to the studio test package") + safe_relative(args.workspace[1:]) + contract = json.loads(CONTRACT.read_text()) + generation = generation_receipt(args.generation_receipt) + except (OSError, ValueError, ProbeError) as error: + parser.error(str(error)) + plan = {"serial": args.serial, "package": args.package, "workspace": args.workspace, + "bundle_path": args.bundle_path, "app_id": contract["app_id"], "generation": generation, + "contract_sha256": hashlib.sha256(CONTRACT.read_bytes()).hexdigest(), + "direct_source_or_app_storage_writes": False} + if args.dry_run: + print(json.dumps(plan, indent=2)) + return 0 + args.output.mkdir(parents=True, exist_ok=False) + evidence = {**plan, "status": "failed", "started_unix": int(time.time()), "checks": {}, + "visual_review": {"status": "required", "items": ["readability", "clipping", "overlap", "contrast", "orientation", "multilingual glyphs", "keyboard"]}, + "fault_checks": {name: "not run; requires isolated fault injection" for name in contract["fault_checks"]}} + device = Device(args.adb, args.serial, args.package) + studio = Studio(device, args.workspace, args.output, args.timeout) + try: + if device.call("get-state").stdout.strip() != b"device": + raise ProbeError("authorized device unavailable") + state = device.read("files/.octosense/dev-mode.json", 65536) + if not state or json.loads(state).get("scope") != "all": + raise ProbeError("configure developer mode in this test profile before running the flow") + studio.start() + checked = studio.call("studio.bundle_check", {"bundle_path": args.bundle_path}) + evidence["admission"] = checked + if checked.get("digest") != generation["bundle_digest"] or checked.get("app_id") != contract["app_id"]: + raise ProbeError("checked app identity/digest does not match the model generation receipt") + if checked.get("source_modified") is not False or checked.get("publisher_signed") is not False: + raise ProbeError("expected source-preserving unsigned developer preparation") + evidence["checks"]["model_bundle_digest_binding"] = "passed; authorship evidence requires transcript review" + studio.open(bundle_path=args.bundle_path) + evidence["checks"]["preview_interactions"] = exercise(studio, contract, "preview") + studio.close() + studio.open(bundle_path=args.bundle_path) + snap = studio.inspect("preview-reset") + assert_tasks(snap, [], contract["fixture_titles"]) + assert_count(snap, contract["selectors"]["count"], 0) + evidence["checks"]["preview_storage_discarded"] = "passed" + studio.close() + installed = studio.call("studio.install", {"bundle_path": args.bundle_path}) + if installed.get("digest") != checked["digest"] or installed.get("installed") is not True: + raise ProbeError("developer install did not preserve checked bundle identity") + evidence["installation"] = installed + studio.open(app_id=contract["app_id"]) + evidence["checks"]["installed_interactions_and_separate_storage"] = exercise(studio, contract, "installed") + studio.close() + studio.open(app_id=contract["app_id"]) + assert_persisted(studio, contract, "installed-reopen") + evidence["checks"]["installed_close_reopen"] = "passed" + # An abrupt process stop while open proves persistence is app-owned, + # rather than an in-memory preview or a graceful-close-only save. + studio.start() + studio.open(app_id=contract["app_id"]) + assert_persisted(studio, contract, "process-restart") + evidence["checks"]["installed_process_restart"] = "passed" + names = contract["selectors"] + evidence["checks"]["multilingual_native_input_and_restart"] = exercise_multilingual(studio, contract) + studio.add(contract["long_title"], names, "long-title") + # More rows than can fit at the required minimum height makes scroll + # acceptance meaningful; sending an event to a short list proves little. + list_node = unique_node(studio.snapshot, source_id=names["list"]) + required_rows = int(bounds(list_node)[3] / contract["minimum_control_height_logical"]) + 3 + extra_count = max(1, required_rows - 5) + if extra_count > 20: + raise ProbeError("unexpectedly large list viewport; inspect display scale") + extra_titles = [f"Scroll check {index + 1:02d}" for index in range(extra_count)] + for index, title in enumerate(extra_titles): + studio.add(title, names, f"scroll-row-{index + 1:02d}") + def scroll(delta, label): + node = unique_node(studio.snapshot, source_id=names["list"]) + studio.call("studio.input", {"instance_id": studio.instance, + "widget_id": node.get("selector", node["id"]), + "action": "scroll", "delta_y": delta}) + return studio.inspect(label) + top = scroll(-2000, "scroll-top") + unique_node(top, text=contract["fixture_titles"][0], enabled=False) + bottom = scroll(2000, "scroll-bottom") + unique_node(bottom, text=extra_titles[-1], enabled=False) + if any(node_text(node) == contract["fixture_titles"][0] for node in widgets(bottom)): + raise ProbeError("scroll did not move the first row out of the viewport") + assert_count(bottom, names["count"], 4 + extra_count) + evidence["checks"]["native_scroll"] = {"status": "passed", "extra_rows": extra_count, + "first_row_at_top": contract["fixture_titles"][0], "last_row_at_bottom": extra_titles[-1]} + # Typing targets the native editor. Its app-only capture cannot contain + # the OS keyboard, so visibility/owner evidence is explicitly separate. + studio.input(unique_node(bottom, source_id=names["title_input"]), "text", "Keyboard layout check") + studio.inspect("keyboard-layout") + evidence["keyboard"] = keyboard_diagnostics(device) + evidence["checks"]["long_title"] = "native input exercised; review captures for wrapping and clipping" + studio.close() + evidence["status"] = "functional_passed_visual_review_required" + except Exception as error: + evidence["error"] = str(error) + finally: + try: + studio.close() + except Exception as error: + evidence["close_error"] = str(error) + evidence["status"] = "failed" + try: + device.stop() + device.shell("input", "keyevent", "KEYCODE_HOME") + except Exception as error: + evidence["cleanup_error"] = str(error) + evidence["status"] = "failed" + evidence["tool_call_count"] = len(studio.calls) + evidence["finished_unix"] = int(time.time()) + (args.output / "evidence.json").write_text(json.dumps(evidence, indent=2) + "\n") + print(json.dumps({"status": evidence["status"], "evidence": str(args.output / "evidence.json")}, indent=2)) + return 0 if evidence["status"] == "functional_passed_visual_review_required" else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/studio-flow/task-planner-acceptance.json b/tools/studio-flow/task-planner-acceptance.json new file mode 100644 index 00000000..ac7fa549 --- /dev/null +++ b/tools/studio-flow/task-planner-acceptance.json @@ -0,0 +1,70 @@ +{ + "schema": 1, + "app_id": "dev.studio.task-planner", + "app_name": "Task Planner", + "source_requirement": "fresh model-authored bundle; preserve generation transcript and final digest", + "storage_file": "tasks.json", + "capabilities": [ + "storage" + ], + "selectors": { + "title_input": "task_title", + "add": "add_task", + "all": "filter_all", + "active": "filter_active", + "done": "filter_done", + "count": "task_count", + "validation": "task_error", + "list": "task_list" + }, + "minimum_control_height_logical": 44, + "fixture_titles": [ + "Buy oats", + "Call Sam", + "Read ten pages" + ], + "long_title": "Plan a quiet weekend walk and remember to bring water for everyone", + "completed_title": "Call Sam", + "expected_before_restart": { + "active_count": 2, + "all": [ + "Buy oats", + "Call Sam", + "Read ten pages" + ], + "active": [ + "Buy oats", + "Read ten pages" + ], + "done": [ + "Call Sam" + ] + }, + "required_checks": [ + "model_authored_new_bundle", + "real_bundle_admission", + "native_empty_state", + "blank_title_validation", + "add_three_via_native_input", + "complete_via_native_input", + "all_active_done_filters", + "reopen_completed_task", + "preview_close_reopen_resets_disposable_data", + "developer_install", + "installed_storage_is_separate_from_preview", + "installed_app_process_restart_persistence", + "multilingual_native_input_and_restart", + "long_title_scroll_and_keyboard", + "real_native_capture_visual_review" + ], + "fault_checks": [ + "malformed_saved_json_preserved", + "save_failure_reported" + ], + "excluded_claims": [ + "public_store_approval", + "publisher_signing", + "model_image_delivery_from_filename_alone" + ], + "multilingual_title": "测试中文任务：买燕麦" +} diff --git a/tools/studio-model-evidence.py b/tools/studio-model-evidence.py new file mode 100644 index 00000000..45b9b407 --- /dev/null +++ b/tools/studio-model-evidence.py @@ -0,0 +1,205 @@ +#!/usr/bin/env python3 +"""Extract tool inputs/results and committed assistant finals from local octos evidence. + +Accepts UI-ledger snapshot JSON, arrays of ledger entries, ledger JSONL, and +session-message JSONL. Reasoning, streamed deltas, user prompts and provider +metadata are excluded by an allowlist. Credential-like fields are redacted. +A retained ring can be incomplete: this tool records that limitation and +never treats a requested view_image call as proof of image delivery. +""" +import argparse +import hashlib +import json +from pathlib import Path +import re + +SESSION = "_main:api:octosense#system" +SENSITIVE = {"api_key", "apikey", "access_token", "refresh_token", "authorization", + "proxy_authorization", "password", "secret", "credentials", "token"} + + +def clean(value): + if isinstance(value, dict): + return {key: "[redacted]" if key.lower().replace("-", "_") in SENSITIVE else clean(item) + for key, item in value.items() + if "reasoning" not in key.lower() and key.lower() not in ("thinking", "thought_signature")} + if isinstance(value, list): + return [clean(item) for item in value] + if not isinstance(value, str): + return value + text = re.sub(r"(?i)\bBearer\s+[^\s\"']+", "Bearer [redacted]", value) + text = re.sub(r"\bsk-[A-Za-z0-9_-]{16,}\b", "[redacted-api-key]", text) + text = re.sub(r'(?i)(["\'](?:api_key|access_token|refresh_token|password|authorization|secret)["\']\s*:\s*)["\'][^"\']*["\']', + r'\1"[redacted]"', text) + text = re.sub(r"(?i)([?&](?:api_key|access_token|token|key)=)[^&\s]+", r"\1[redacted]", text) + return text + + +def load_records(path): + raw = path.read_bytes() + text = raw.decode("utf-8") + try: + root = json.loads(text) + if isinstance(root, list): + return root, raw + if isinstance(root, dict) and isinstance(root.get("entries"), list): + return root["entries"], raw + return [root], raw + except json.JSONDecodeError: + records = [] + for index, line in enumerate(text.splitlines()): + if not line.strip(): + continue + try: + records.append(json.loads(line)) + except json.JSONDecodeError as error: + raise ValueError(f"{path.name}: invalid JSONL line {index + 1}; copy after writer flush") from error + return records, raw + + +def parse_output(text): + if not isinstance(text, str): + return text if isinstance(text, dict) else None + try: + return json.loads(text) + except json.JSONDecodeError: + return None + + +def extract(records, session=SESSION): + calls, persisted, terminals, finals, errors = {}, {}, {}, [], [] + def call_for(turn, call_id): + key = (turn or "unknown", call_id) + return calls.setdefault(key, {"turn_id": turn, "tool_call_id": call_id}) + for record in records: + if not isinstance(record, dict): + continue + event = record.get("event", {}) + if isinstance(event, dict) and event.get("session_id") == session: + turn = event.get("turn_id") + kind = event.get("kind") + if kind in ("tool_started", "tool_completed"): + item = call_for(turn, event.get("tool_call_id")) + item["name"] = event.get("tool_name") + if kind == "tool_started": + item.update(arguments=clean(event.get("arguments")), input_complete=True, start_seq=record.get("seq")) + else: + item.update(success=event.get("success"), output=clean(event.get("output_preview")), + output_is_preview=True, end_seq=record.get("seq")) + elif kind == "turn_completed": + identity = event.get("session_result") or {} + terminals[turn] = identity.get("message_id") + elif kind == "turn_error": + errors.append({"turn_id": turn, "code": event.get("code"), "message": clean(event.get("message"))}) + partial = event.get("partial_result") or {} + identity = partial.get("session_result") or {} + if identity.get("message_id"): + terminals[turn] = identity["message_id"] + elif kind == "envelope_v2": + envelope = event.get("envelope", {}) + turn = envelope.get("turn_id", turn) + payload = envelope.get("payload", {}) + data = payload.get("data", {}) + payload_type = payload.get("type") + if payload_type == "assistant_persisted": + message_id = (data.get("meta") or {}).get("message_id") + if message_id: + persisted[message_id] = {"turn_id": turn, "message_id": message_id, "text": clean(data.get("text", ""))} + elif payload_type == "tool_start": + item = call_for(turn, data.get("tool_call_id")) + item.setdefault("name", data.get("name")) + if "arguments" not in item: + item.update(arguments=clean(data.get("arguments_preview")), input_complete=False) + elif payload_type == "tool_end": + item = call_for(turn, data.get("tool_call_id")) + item.setdefault("success", data.get("status") == "complete") + item.setdefault("output", clean(data.get("output_preview"))) + item.setdefault("output_is_preview", True) + continue + # Session JSONL stores Message directly; trust only the selected input + # file's scope. Metadata, prompts and reasoning_content are never copied. + role = record.get("role") + turn = record.get("thread_id") + if role == "assistant": + for tool in record.get("tool_calls") or []: + item = call_for(turn, tool.get("id")) + item.update(name=tool.get("name"), arguments=clean(tool.get("arguments")), input_complete=True) + # No terminal identity in legacy Message JSONL: omit its prose. + elif role == "tool" and record.get("tool_call_id"): + item = call_for(turn, record["tool_call_id"]) + item.update(output=clean(record.get("content")), output_is_preview=False) + for turn, message_id in terminals.items(): + if message_id in persisted: + finals.append({**persisted[message_id], "terminal_verified": True, "source": "ledger_terminal_message_id"}) + result = list(calls.values()) + for item in result: + if item.get("name") == "view_image": + output = parse_output(item.get("output")) + shown = output.get("shown_to_model") if isinstance(output, dict) else None + item["image_delivery"] = "reported_shown_to_model" if shown is True else "reported_not_shown" if shown is False else "unverified" + return {"tool_calls": result, "assistant_finals": finals, "turn_errors": errors} + + +def merge_extractions(parts): + calls, finals, errors = {}, {}, {} + for part in parts: + for call in part.get("tool_calls", []): + key = (call.get("turn_id"), call.get("tool_call_id")) + old = calls.setdefault(key, {}) + # Never replace complete input/output with a later ring preview. + incoming = dict(call) + if old.get("input_complete") and not incoming.get("input_complete"): + incoming.pop("arguments", None) + incoming.pop("input_complete", None) + if old.get("output_is_preview") is False and incoming.get("output_is_preview"): + incoming.pop("output", None) + incoming.pop("output_is_preview", None) + old.update(incoming) + for final in part.get("assistant_finals", []): + key = (final.get("turn_id"), final.get("message_id"), final.get("text")) + finals[key] = final + for error in part.get("turn_errors", []): + errors[(error.get("turn_id"), error.get("code"), error.get("message"))] = error + return {"tool_calls": list(calls.values()), "assistant_finals": list(finals.values()), "turn_errors": list(errors.values())} + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--input", type=Path, action="append", required=True) + parser.add_argument("--output", type=Path, required=True) + parser.add_argument("--merge", type=Path, help="previous sanitized output, preserving tool events evicted from a ledger ring") + parser.add_argument("--session", default=SESSION) + args = parser.parse_args() + sources, parts, all_records = [], [], [] + if args.merge: + previous = json.loads(args.merge.read_text()) + parts.append(previous) + sources.extend(previous.get("sources", [])) + for path in args.input: + records, raw = load_records(path) + all_records.extend(records) + sources.append({"name": path.name, "sha256": hashlib.sha256(raw).hexdigest()}) + parts.append(extract(all_records, args.session)) + output = {"schema": 1, "session": args.session, "sources": sources, **merge_extractions(parts), + "limitations": ["Ledger rings and output previews may omit events or truncate results.", + "Legacy assistant prose without a terminal identity is omitted.", + "shown_to_model is runtime delivery evidence, not a visual quality judgment.", + "Reasoning, user prompts and provider metadata are excluded; credential-like fields are redacted."]} + calls = output["tool_calls"] + for call in calls: + if call.get("name") == "view_image": + payload = parse_output(call.get("output")) + shown = payload.get("shown_to_model") if isinstance(payload, dict) else None + call["image_delivery"] = "reported_shown_to_model" if shown is True else "reported_not_shown" if shown is False else "unverified" + output["summary"] = {"tool_calls": len(calls), "assistant_finals": len(output["assistant_finals"]), + "turn_errors": len(output["turn_errors"]), + "authoring_calls_observed": sum(c.get("name") in ("write_file", "edit_file", "apply_patch") for c in calls), + "view_image_calls": sum(c.get("name") == "view_image" for c in calls), + "images_reported_shown": sum(c.get("image_delivery") == "reported_shown_to_model" for c in calls)} + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps(clean(output), indent=2, ensure_ascii=False) + "\n") + print(json.dumps(output["summary"], indent=2)) + + +if __name__ == "__main__": + main()