Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/apps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ on:
- 'native-runtime.lock.json'
- 'runtime-patches.lock.json'
- 'tools/setup.py'
- 'tools/sync-app-interface.py'
- '.github/workflows/apps.yml'
pull_request:
paths:
Expand All @@ -26,6 +27,7 @@ on:
- 'native-runtime.lock.json'
- 'runtime-patches.lock.json'
- 'tools/setup.py'
- 'tools/sync-app-interface.py'
- '.github/workflows/apps.yml'
workflow_dispatch:

Expand Down Expand Up @@ -61,6 +63,8 @@ jobs:
# though most protocol crates do not build Makepad. The OAuth host
# adapter also checks App Hub's Makepad-linked service integration.
- run: python3 tools/setup.py
- name: Shared system app interface
run: python3 tools/sync-app-interface.py --check
- uses: Swatinem/rust-cache@v2
with:
shared-key: services
Expand Down
8 changes: 4 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ octosense-reference = { path = "apps/reference" }
makepad-sheets = { git = "https://github.com/OctoSense-org/makepad.git", rev = "68d1f4ecc111daa90c50530e77df3245f05fc2cf", default-features = false }
makepad-terminal = { git = "https://github.com/OctoSense-org/makepad.git", rev = "68d1f4ecc111daa90c50530e77df3245f05fc2cf" }
octosense-appcard = { path = "apps/appcard/module" }
octosense-app-hub-app = { git = "https://github.com/OctoSense-org/OctoSense-App-Hub", rev = "a52d928b0c14d2822c09f6ccd18f44bf7b885221" }
octosense-app-hub-app = { git = "https://github.com/OctoSense-org/OctoSense-App-Hub", rev = "f762267864fc3d68631b46924899cff324f7d157" }
makepad-calculator = { git = "https://github.com/OctoSense-org/makepad.git", rev = "68d1f4ecc111daa90c50530e77df3245f05fc2cf", default-features = false }
makepad-clock = { git = "https://github.com/OctoSense-org/makepad.git", rev = "68d1f4ecc111daa90c50530e77df3245f05fc2cf", default-features = false }
makepad-notes = { git = "https://github.com/OctoSense-org/makepad.git", rev = "68d1f4ecc111daa90c50530e77df3245f05fc2cf", default-features = false }
Expand Down Expand Up @@ -106,12 +106,12 @@ octoscript-makepad = { git = "https://github.com/OctoSense-org/Octoscript-Makepa
octoscript-node = { git = "https://github.com/OctoSense-org/Octoscript-Makepad.git", rev = "aa80f72c509c767faf04822b939d44b2f34fbc81" }
# --- App Hub's host service registry the Mail and llm services register
# into; the same revision as App Hub's app crate (a native app, above).
octosense-appstore = { git = "https://github.com/OctoSense-org/OctoSense-App-Hub", rev = "a52d928b0c14d2822c09f6ccd18f44bf7b885221" }
octosense-appstore = { git = "https://github.com/OctoSense-org/OctoSense-App-Hub", rev = "f762267864fc3d68631b46924899cff324f7d157" }
# The app policy (resolution, isolate settings, the Splash adapter) and the
# Hub client the Card runner resolves an app's policy with; the shell's
# glance tiles run cards under it (crates/shell/src/glance_card.rs).
octosense-app-policy = { git = "https://github.com/OctoSense-org/OctoSense-App-Hub", rev = "a52d928b0c14d2822c09f6ccd18f44bf7b885221" }
octosense-app-hub = { git = "https://github.com/OctoSense-org/OctoSense-App-Hub", rev = "a52d928b0c14d2822c09f6ccd18f44bf7b885221" }
octosense-app-policy = { git = "https://github.com/OctoSense-org/OctoSense-App-Hub", rev = "f762267864fc3d68631b46924899cff324f7d157" }
octosense-app-hub = { git = "https://github.com/OctoSense-org/OctoSense-App-Hub", rev = "f762267864fc3d68631b46924899cff324f7d157" }
# The app contract (ADR 0005), from crates.io by version: App Hub, the shell
# and Rinx all depend on `octosense-app-contract = "1"`, so Cargo links ONE
# 1.x for the whole build (Cargo.lock pins the exact version OctoSense ships)
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ OctoSense is an agent shell that runs on top of an ordinary operating system. On

This repository holds the shell, its services, the system apps, and the three products built from them. Most native apps come from other repositories: the OctoSense fork of Makepad, App Hub and Rinx ([What it depends on](#what-it-depends-on)).

The script apps share [responsive, theme-aware interface styles](apps/README.md#shared-appearance)
across desktop and phone. [Local native UX checks](tools/app-ux/README.md) cover
saved state and actual interaction; phone-size previews remain separate from device acceptance.

| Product | What it is | Where |
| --- | --- | --- |
| **OctoSense desktop** | The shell as one Makepad window on macOS (Windows and Linux untested) | [`desktop/`](desktop/README.md) |
Expand Down
4 changes: 4 additions & 0 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ OctoSense 是运行在普通操作系统之上的 Agent Shell。从屏幕上看

本仓库存放 Shell、Shell 服务、系统应用,以及由它们构建的三个产品。大多数原生应用来自其他仓库:OctoSense 的 Makepad fork、App Hub 和 Rinx(见[依赖](#依赖))。

脚本应用在桌面和手机端共用[自适应主题与布局](apps/README.zh-CN.md#共用外观)。
[本机原生 UX 检查](tools/app-ux/README.zh-CN.md) 验证真实交互和保存状态;
手机尺寸预览与真机验收分别记录。

| 产品 | 是什么 | 位置 |
| --- | --- | --- |
| **OctoSense 桌面端** | 在 macOS 上作为一个 Makepad 窗口运行的 Shell(Windows 和 Linux 未经测试) | [`desktop/`](desktop/README.zh-CN.md) |
Expand Down
45 changes: 41 additions & 4 deletions apps/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,37 @@ OctoScript-App-Design-Flow:

## The apps

### Shared appearance

News, Photos, Mail, Calendar, Maps, AI providers and YouTube share the interface
in [`interface.splash`](interface.splash): theme-aware page/card/field surfaces,
readable secondary text, 44-point actions and 48-point inputs. Desktop content
has a maximum width; narrow windows keep the same actions. Camera and media
viewers retain a dark viewing surface. App Hub uses the same spacing and action
hierarchy in its native module. Expanded Glance cards and in-card chat use the
shared shell on both desktop and phone.

When a keyboard shrinks a focused hosted app, the shell reveals the active
editor through the app’s own scrolling container and keeps fixed actions in
place. Maps switches its tiles with the host appearance. Camera controls use
a dark scrim to remain readable over a bright preview; Android waits for the
first-use permission answer before preparing that preview. A permission dialog
pauses only a playing video; it cannot turn an unprepared camera into a fake
playing state when the app resumes. The preview uses its resolved on-screen
size for aspect-preserving fill, including `Fill` layouts and resized windows;
a rotated camera frame is cropped at the edges instead of stretched. AI providers’
Android QR scanner chooses a preview/YUV size advertised by the camera before
opening it; the preview keeps that aspect ratio, and callbacks from an earlier
scanner session cannot close a newly opened one.

After editing the interface, run `python3 tools/sync-app-interface.py` from the
repository root. It embeds the exact prelude in each contained bundle;
`python3 tools/sync-app-interface.py --check` detects drift in CI.
The prelude resolves the current host theme on restyle, without replacing
unsaved input. [Local native UX checks](../tools/app-ux/README.md) exercise
editing, persistence, appearance switching, browsing and search in isolated
profiles. Phone-sized desktop captures are not physical-device verification.

| App | Id | What it does | Capabilities (manifest) | Network hosts (manifest) | Host services |
| --- | --- | --- | --- | --- | --- |
| [News](news/bundle) | `os.news` | Hacker News, TechMeme and Google News feeds in tabs (Today, HN, TechMeme, Google, Saved), with a reader for stories | `storage`, `net`, `images`, `web`, `news`, `glance` | `hn.algolia.com`, `www.techmeme.com`, `news.google.com`, `api.gdeltproject.org`, `feeds.bbci.co.uk`, `feeds.npr.org`, `www.theguardian.com`, `feeds.arstechnica.com` | [`news`](news/host-service) |
Expand Down Expand Up @@ -93,10 +124,16 @@ reaches only the hosts the manifest lists.
runtime). Playback opens YouTube's mobile watch page, which autoplays muted
and shows its own "Open App" prompt. Search reads YouTube's results page and
depends on its layout.
- **Camera**: on the OnePlus 6 test run (2026-09-25) Camera captured a photo
and released the camera in the background, but the live preview drew pure
black; unresolved. Desktop builds have no camera and the Android emulator
refuses one, so capture is untested elsewhere.
- **Camera**: the Redmi Note 12 test package (2026-10-07) passed delayed
first-use permission, front/back switching, Photo/Video preview selection and
background resume. The stretched `Fill` preview is corrected; native GPU
circle measurements pass, while a clear-scene visual comparison on the phone
is still pending. The OnePlus 6 black preview reported on 2026-09-25 has not
been retested. Still capture was not exercised in this run; Android video
recording remains unimplemented.
- **AI providers camera import**: the Redmi Note 12 scanner opens with supported
preview buffers and passed cancel, system Back, background interruption and
reopening. QR decoding and importing a provider were not exercised in this run.
- **Photos Memories**: open **Memories → Create memories**, optionally entering
a theme such as “summer with family.” The host's `model.complete` uses the
provider configured in **Settings → AI providers** to curate up to three
Expand Down
33 changes: 30 additions & 3 deletions apps/README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,30 @@ OctoScript-App-Design-Flow 的 `AGENTS.md`,再读 `docs/QUICKSTART.md`),

## 应用一览

### 共用外观

新闻、相册、邮件、日历、地图、AI providers 和 YouTube 共用
[`interface.splash`](interface.splash):页面、卡片和输入框跟随主题,
辅助文字保持可读,操作按钮高 44 点,输入框高 48 点。桌面内容限制最大宽度,
窄窗口保留相同操作。相机和媒体查看器保留深色观看背景。应用大厅的原生模块
采用相同的间距和操作层级。展开的 Glance 卡片及卡片内聊天在桌面和手机端
使用同一套 Shell。

键盘缩小当前应用的可视区域时,Shell 通过应用自己的滚动区域显示当前输入框,
固定操作栏保留原位。地图图块随宿主切换明暗外观。相机按钮使用深色底板,
在明亮取景画面上仍可阅读;Android 等待首次权限弹窗的结果后才准备预览。
权限弹窗只能暂停正在播放的视频,不能将尚未初始化的相机在返回后误标为正在播放。
预览按布局完成后的实际显示尺寸等比填充,支持 `Fill` 和窗口缩放;
旋转后的相机画面裁切边缘,不再拉伸。AI providers 的 Android 扫码器
先选择相机声明支持的预览/YUV 输出尺寸,再打开相机;显示时保持比例,
旧扫码会话的回调不会关闭新打开的扫码器。

修改后在仓库根目录运行 `python3 tools/sync-app-interface.py`,将共用样式
原样嵌入各个独立应用包;CI 通过 `python3 tools/sync-app-interface.py --check`
检查是否同步。样式重载时读取当前宿主主题,同时保留未保存的输入。
[本机原生 UX 检查](../tools/app-ux/README.zh-CN.md) 在独立配置中验证编辑、
持久化、主题切换、浏览和搜索。手机尺寸的桌面截图不代表真机验证。

| 应用 | Id | 功能 | 权限(manifest) | 网络主机(manifest) | 宿主服务 |
| --- | --- | --- | --- | --- | --- |
| [News](news/bundle) | `os.news` | Hacker News、TechMeme 和 Google News 的订阅源,分标签页(Today、HN、TechMeme、Google、Saved),带文章阅读器 | `storage`、`net`、`images`、`web`、`news`、`glance` | `hn.algolia.com`、`www.techmeme.com`、`news.google.com`、`api.gdeltproject.org`、`feeds.bbci.co.uk`、`feeds.npr.org`、`www.theguardian.com`、`feeds.arstechnica.com` | [`news`](news/host-service) |
Expand All @@ -80,9 +104,12 @@ OctoScript-App-Design-Flow 的 `AGENTS.md`,再读 `docs/QUICKSTART.md`),
- **YouTube**:在 OnePlus 6 上测试(2026-09-27),搜索、结果、播放和播放记录都正常;
关闭播放器会结束页面(makepad#43,已在运行时中)。播放打开的是 YouTube 移动版观看页,
它会静音自动播放,并显示自己的“Open App”提示。搜索读取 YouTube 的搜索结果页,依赖其布局。
- **Camera**:在 OnePlus 6 测试中(2026-09-25),Camera 能拍照并在后台释放
相机,但实时预览是纯黑的,尚未解决。桌面构建没有相机,Android 模拟器拒绝
提供相机,因此其他环境下拍摄未经测试。
- **Camera**:Redmi Note 12 独立测试包(2026-10-07)已通过延迟首次授权、
前后镜头切换、Photo/Video 预览模式切换和后台恢复。`Fill` 预览拉伸已修正,
原生 GPU 的圆形尺寸对比通过;手机明亮场景的外观对比仍待验证。
2026-09-25 报告的 OnePlus 6 黑屏尚未复测。本轮未测试拍照;Android 录像仍未实现。
- **AI providers 相机导入**:Redmi Note 12 扫码器使用设备支持的预览尺寸,
已通过打开、取消、系统返回、后台中断和重新打开;本轮未测试二维码解码及服务商导入。
- **Photos 回忆**:打开 **Memories → Create memories**,可选输入“summer with family”等主题。
宿主的 `model.complete` 使用 **设置 → AI providers** 中配置的服务商,整理最多三个
带标题、简短叙述和有序幻灯片的故事。只发送相片目录的元数据(日期、地点、姓名、
Expand Down
Loading
Loading