glance: resolve sys.digest sources from host-held digests (ADR 0002 M4) by ymote · Pull Request #87 · OctoSense-org/OctoSense

ymote · 2026-09-28T02:03:56Z

Part of #63, milestone M4 in ADR 0002.

Dependencies merged. OctoSense-org/OctoScript#40 (sys.digest in the L0 catalog) merged as 30a21b78, and OctoSense-org/OctoScript-Makepad#50, which repins Octoscript to it, merged as c29cfb84. Both native-runtime.lock.json files and the root Cargo.toml pin Octoscript-Makepad c29cfb84 (octoscript-makepad, octoscript-node) and Octoscript 30a21b78, and so do the docs that name them.

Rebased onto main (2026-10-01). The conflicts were resolved against main's current design:

Cargo.toml and both native-runtime.lock.json files: main's pins (makepad e29a0eaa, App Hub, the contract crate and so on), with only the Octoscript and Octoscript-Makepad revisions moved. Cargo.lock needed no change, because both resolve to .sources/ paths. phone/docs/makepad-fork.md and rom/docs/home-build.md name the new revisions.
glance.rs: main's interactive cards (Interactive glance cards under the publishing app's policy [skip ci] #153) added script cards and GlanceCard::contained. sys.digest resolution applies only to source (L0) cards, since a script card carries its own values and binds no digest. Both fields are kept. Caller::Contained is now a struct variant, so the test uses Caller::granted.
glance_digest::digest_root now resolves under the host storage's apps root, using octosense_ai_host::toolbox_root. That is where main's system toolbox (G3 host tools, host_tools/toolbox.rs) writes its run results, so the glance service reads the same folder the toolbox writes. It is None until the host storage exists.

A glance card can now declare sys.digest(app:, id:). When the glance service realizes the card, it fills that source from the digests the host holds for the calling app, with provenance. The card no longer has to carry the findings as text.

Where digests come from (`crates/shell/src/glance_digest.rs`)

Files: system-toolbox run results at <app folder>/toolbox/runs/<template>/<run>.json (RunOptions::write_result, feat(toolbox): the system toolbox's workflow template library #82). The shell reads the file directly and has no crate dependency on the unmerged toolbox.
Host-owned app folder: the app folder the shell gives the toolbox is <apps root>/.host/toolbox/<app id>, outside the app's jail (<apps root>/<app id>). If the file lived in the jail, the app could forge a digest, citations included. When the toolbox is wired into the shell, it must be given this folder.
One output shape, whatever the template (news-digest data.digest, topic-brief data.brief, …):
- top level: status, topic, language, summary, retrieved_at, count
- points: rows {id, text, label, cite, citations: [source index]}
- sources: rows {id, n, title, source, url, published_at}
- retrieved_at is the newest provenance retrieved_at among the kept sources, falling back to started_at.
Provenance:
- A source is kept only when the run's host-kept provenance lists its id with the same http(s) URL.
- A point is kept only when it still cites a kept source. Its citations become indexes into sources, and cite holds the same list as 1-based text.
- A point with a URL in its text or label is dropped, and a summary with a URL is emptied. Links come only from sources[].url.
Caps:
- the file: 512 KiB
- summary: 800 chars
- points: 8, text 400 chars each
- sources: 8, title 200 chars each
- citations per point: 8
- template folders scanned: 64
- symlinked files are ignored
Absence is a state, not an error. A missing, expired, malformed, oversized or mismatched digest resolves to an empty record with $state .failed. status is one of missing, expired or failed, and the card renders its own "no digest yet" vocabulary. A mismatch means the file's app_id or run_id differs from the request.

Binding (`glance.rs`)

Own app only. app must be the publisher, which is the caller: its app id, or its launcher id (news for os.news). Any other value is refused at publish. The folder read is always the caller's, whatever the card says.
Ids. An id is [A-Za-z0-9_-]{1,64}, the run-id charset, so there's no path traversal. The checker enforces this for literals, and the resolver enforces it again for all ids. An id can also come from a state initial (id: state.run).
The publisher can't spoof it. The resolved value and its $status replace anything the publisher put in data.
Expiry. A digest is current for 48 h after its run's started_at. A card bound to a current digest expires no later than the digest does.
Resolution time. Digests are resolved at publish time. An agent that uses a fixed run id (glance) overwrites the same file on each run and republishes the card.

Retention

After a publish that binds digests, each of the app's template folders keeps its 8 newest results plus every run id that a live glance card of that app binds (GlanceCard::digests). Everything else is deleted. Nothing else in the shell deletes run results.

M4 card

crates/shell/resources/glance/news-brief.card binds sys.digest(app: "os.news", id: "glance"). It shows the topic, summary, points with their citation numbers, and the numbered sources. When the digest is missing it falls back to its own vocabulary.
OCTOSENSE_GLANCE_DEMO=digest publishes it. OCTOSENSE_GLANCE_DEMO=1 still publishes the old data-fed demo.

Tests (all pass locally)

Resolver tests: stable shape from a news-digest result (a fixture built from the toolbox's city-infrastructure case); topic-brief gives the same shape; missing, expired, malformed, other-app, traversal and oversized digests; links only from provenance; caps; failed runs; retention; RFC 3339 parsing.
Glance tests: the digest renders into the lowered body and forged data is ignored; expiry is capped by the digest; another app's app is refused; Maps gets its own empty folder; a missing or expired digest renders the absence text; publishing prunes old runs but keeps bound ones.
Commands, after the rebase, in a fresh worktree with python3 tools/setup.py --no-hub. They were re-run after the repin to c29cfb84, with .sources refreshed by setup.py --no-hub --update (plain setup refuses to move a checkout). All of these pass:
- python3 tools/setup.py --check --cargo and python3 tools/native_apps.py --check
- python3 -m unittest discover -s tools -p 'test_*.py'
- from desktop/: the three cargo check --locked -p octosense … builds and bash ../tools/check-shell-graph.sh -p octosense
- from phone/: the three cargo check --locked -p octosense-home … builds and bash ../tools/check-shell-graph.sh -p octosense-home
- from phone/: cargo test --locked --features mobile-apps -p octosense-shell -p octosense-home -p octosense-app-policy -p octosense-app-hub -p octosense-appcard -p octosense-ai-host (shell: 736 passed), plus the appcard (app-appcard) and host_tools (toolbox-peers) runs

Headless render (before the rebase; not repeated)

The desktop build ran with its window hidden (MAKEPAD_HIDE_WINDOWS=1, --remote, --test-action glance, OCTOSENSE_GLANCE_DEMO=digest, OCTOSENSE_APP_DATA pointing at a scratch apps root). Frames came from the remote /g route; I took no OS screenshots.

With a current glance.json: the log shows glance: os.news published digest and the panel lays the tile out at 328×260, the cap. It shows the topic, summary, three cited points (1, 2, 3) and the sources list, and the rest is clipped at the tile cap.
With no digest: the tile is 328×95 and reads "No digest yet. News will brief you after its next read."
No panics in either run.

Open

The Card runner (App Hub card-host) doesn't resolve sys.digest yet, so a bundle's own page.card that declares it renders the empty state.
Main's system toolbox now writes run results under the folder this reads. The News agent (M3) that publishes the bound card is still to come.
With a real model summary, the M4 card fits better. The fixture's fallback summary pushes the source list past the 260 pt tile cap.

🤖 Generated with Claude Code

… GPU timing, terminal shortcuts) Makepad main 39a9de88 adds a hidden map's idle frame loop fix (#79), Android HTTP without redirects (#75), the sheets font by crate (#80), Linux GL remote grabs (#93), Android Vulkan without a debug messenger (#92), the touch-down power hint (#83), GL GPU timing (#81), atlas text on phones (#82), the DrawText current-list fix (#89), terminal keyboard shortcuts (#76) and the terminal on OpenHarmony (#91), and OpenHarmony's keyboard, Studio, GL lifetimes and idle frames (#84, #85, #88, #87), through Octoscript-Makepad 62760863 (#70). makepad #90 is not in it. - root Cargo.toml: makepad and octoscript-makepad revisions together; native-apps.json (Sheets, Terminal), regenerated pins unchanged otherwise - both native-runtime.lock.json files - runtime-patches.lock.json: the settings patch is rebased onto 39a9de88 with the same added and removed lines; its touch-down line in MakepadActivity.java now sits next to #83's hint, and draw_text.rs context follows #87's cfg. The system-back, IME and two Windows patches reapply unchanged. The map, OpenHarmony GL-lifetime and DrawText patches are dropped: they are makepad #79, #88 and #89, now upstream. New tree 976bf55b. - Cargo.lock: unchanged - the three pinned-hash docs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… GPU timing, terminal shortcuts) Makepad main 39a9de88 adds a hidden map's idle frame loop fix (#79), Android HTTP without redirects (#75), the sheets font by crate (#80), Linux GL remote grabs (#93), Android Vulkan without a debug messenger (#92), the touch-down power hint (#83), GL GPU timing (#81), atlas text on phones (#82), the DrawText current-list fix (#89), terminal keyboard shortcuts (#76) and the terminal on OpenHarmony (#91), and OpenHarmony's keyboard, Studio, GL lifetimes and idle frames (#84, #85, #88, #87), through Octoscript-Makepad 62760863 (#70). makepad #90 is not in it. - root Cargo.toml: makepad and octoscript-makepad revisions together; native-apps.json (Sheets, Terminal), regenerated pins unchanged otherwise - both native-runtime.lock.json files - runtime-patches.lock.json: the settings patch is rebased onto 39a9de88 with the same added and removed lines; its touch-down line in MakepadActivity.java now sits next to #83's hint, and draw_text.rs context follows #87's cfg. The system-back, IME and two Windows patches reapply unchanged. The map, OpenHarmony GL-lifetime and DrawText patches are dropped: they are makepad #79, #88 and #89, now upstream. New tree 976bf55b. - Cargo.lock: unchanged - the pinned-hash docs under phone/docs; rom/docs/home-build.md follows in its own change, so this one does not trigger rom.yml (whose jobs local CI cannot run) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A glance card may declare `sys.digest(app:, id:)` (Octoscript #40); the glance service fills it in when it realizes the card, from the toolbox run results the host keeps for the calling app, instead of the card carrying the findings as text. - glance_digest.rs: reads `<apps root>/.host/toolbox/<app>/toolbox/runs/ <template>/<run>.json` (host-owned, outside the app's jail, so an app cannot forge a digest) and returns one template-independent record: status, topic, language, summary, retrieved_at, count, points {id, text, label, cite, citations: [source index]}, sources {id, n, title, source, url, published_at}. A source is kept only if the run's host provenance lists its id with the same http(s) URL; a point only if it still cites a kept source; model text with a URL is dropped. Caps on the file (512 KiB) and every field. Missing, expired (48 h after the run), malformed or failed digests resolve to an empty record with $state failed. - glance.rs: `app` must be the caller (its id or launcher id), never an argument's say-so; the resolved value and $status replace anything the publisher sent; a bound card expires no later than its digest; after a publish, each template folder keeps its 8 newest results plus every run a live card binds. - news-brief.card: the M4 News digest card bound to `sys.digest(app: "os.news", id: "glance")`; OCTOSENSE_GLANCE_DEMO=digest publishes it. - Octoscript #40 (sys.digest) is already in main's pins (Octoscript 5991dfae, Octoscript-Makepad 8f103d0c); rebased onto main, the PR's own repin commits are dropped. - Rebased onto main: composes with the in-card sys.chat (glance_chat.rs): digests resolve into the card's data first, then its chat is seeded and the card lowered; the card keeps both its digests and its L0 source. - glance_digest::digest_root reads under the host storage's apps root (octosense_ai_host::toolbox_root), where the toolbox writes its runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A card notification's second line (#290) falls back to the card's own summary in its data. For a card bound to `sys.digest`, that data is the host's digest (glance_digest.rs), so the line now reads the data the store keeps rather than the arguments the publisher sent, which could carry a summary the card itself refuses. A card without kept L0 data (a script card) reads its arguments as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

GlanceCard gained `digests` in this branch. The phone feed test's `card` helper in mobile_pages.rs, added on main after this branch forked, built a GlanceCard without the field, so the shell's lib tests stopped compiling once the branch was rebased onto main. The sample cards bind no digests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ymote · 2026-10-02T23:01:25Z

Local CI passed on 6106cc3 (tools/ci-local.sh --only all, 14m58s, Darwin arm64).

GitHub would run for this PR: desktop.yml, phone.yml, apps.yml. Merged on this local pass (macOS runner queue); GitHub CI runs on the merge commit on main.

Job	Step	Result	Time
ci-local / drift	The local mapping fits the workflows	PASS	-
desktop.yml / desktop	Prepare the pinned framework sources	PASS	0s
desktop.yml / desktop	Compile the desktop with the shared runtime	PASS	2m20s
desktop.yml / desktop	One copy of every shell source	PASS	0s
desktop.yml / desktop	Desktop tools and scripts	PASS	27s
desktop.yml / desktop	One Makepad, App Hub, octos and Rinx in the locked graph	PASS	4s
desktop.yml / desktop	The native apps' generated places match native-apps.json	PASS	0s
phone.yml / home	Prepare pinned sources and the reviewed runtime patch	PASS	1s
phone.yml / home	Compile Home and its bundled modules	PASS	1m48s
phone.yml / home	Verify the single runtime graph	PASS	2s
phone.yml / home	The native apps' generated places match native-apps.json	PASS	0s
phone.yml / home	The pinned octos revision	PASS	0s
phone.yml / home	Build octos (unless cached)	PASS	-
phone.yml / home	Build Home, App Hub admission and runtime policy tests	PASS	44s
phone.yml / home	Test Home, App Hub admission and runtime policy	PASS	4m09s
phone.yml / home	Two-lane scenario (real kernel, scripted model)	PASS	36s
apps.yml / services	python3 tools/setup.py	PASS	0s
apps.yml / services	Kernel service	PASS	8s
apps.yml / services	App peers	PASS	20s
apps.yml / services	AI providers config	PASS	1s
apps.yml / services	L0 in-card chat (sys.chat, host side)	PASS	2s
apps.yml / services	System toolbox (workflow templates)	PASS	1m15s
apps.yml / kernel-security	python3 tools/setup.py	PASS	1s
apps.yml / kernel-security	The pinned octos revision	PASS	0s
apps.yml / kernel-security	Build octos (unless cached)	PASS	-
apps.yml / kernel-security	Real-kernel tests	PASS	34s
apps.yml / apps	python3 tools/setup.py	PASS	1s
apps.yml / apps	Host services (Mail, AI providers, News)	PASS	21s
apps.yml / apps	Shell AI services (crates/ai-host)	PASS	12s
apps.yml / apps	AppCard runtime setup safeguards	PASS	0s
apps.yml / apps	AppCard clippy (own crates; shared runtime compiles)	PASS	1s
apps.yml / apps	AppCard transport and store	PASS	4s
apps.yml / apps	One Makepad, App Hub, octos and Rinx in the locked graph	PASS	3s
rom.yml / product	python3 -m unittest discover -s tests -v	PASS	1m14s
rom.yml / product	Check generated Agent Binder client	PASS	0s
rom.yml / product	bash -n scripts/stage-forks.sh scripts/publish-release.sh scripts/apply-to-tree.sh scripts/build-rom.sh	PASS	-
rom.yml / web-installer	npm ci --ignore-scripts --no-audit --no-fund	PASS	20s
rom.yml / web-installer	npm test	PASS	1s
rom.yml / web-installer	npx playwright install --with-deps chromium	PASS	1s
rom.yml / web-installer	npm run test:browser	PASS	5s
rom.yml / web-installer	python3 -m unittest discover -s tests -p test_installer_manifest.py -v	PASS	0s
rom.yml / web-installer	bash -n scripts/release.sh	PASS	0s

Notes:

GitHub runs Python 3.12; this run used Python 3.9.6
apps.yml:services runs on ubuntu-latest in GitHub; here on macOS, so #[cfg(target_os = "linux")] code in it is not exercised (--linux-host runs it on Linux)
apps.yml:kernel-security runs on ubuntu-latest in GitHub; here on macOS, so #[cfg(target_os = "linux")] code in it is not exercised (--linux-host runs it on Linux)

ymote · 2026-10-02T23:01:41Z

The Linux-host steps also passed on this same head, 6106cc3. Run 1 started 2026-10-02T15:31:57 (tools/ci-local.sh --only all --linux-host): 36 steps passed. That includes apps services, kernel-security and linux-host / sandbox (Landlock, seccomp) on the Linux build host. Its only skips were the 8 rom.yml steps, which can't run on the Linux host, and this PR doesn't trigger rom.yml.

The merge evidence is run 2, a Mac-only --only all run with the rom toolchain: 42/42 passed. Its summary is here: #87 (comment)

Rebasing onto main needed one trivial compile fix, commit 6106cc3. The phone feed test's card helper in mobile_pages.rs, added on main after this branch forked, now sets digests: Vec::new(). On that head, the glance tests from phone/ (65) and cargo test --locked -p octosense-l0-chat (9) pass.

ymote force-pushed the feat/glance-digest-source branch from 01a6992 to 7f69f61 Compare October 1, 2026 16:45

ymote marked this pull request as ready for review October 1, 2026 17:23

ymote force-pushed the feat/glance-digest-source branch from 8936e05 to f389f27 Compare October 2, 2026 03:46

ymote mentioned this pull request Oct 2, 2026

Repin the runtime to makepad 39a9de88 (OpenHarmony idle and input, GL GPU timing, terminal shortcuts) #289

Merged

ymote force-pushed the feat/glance-digest-source branch 2 times, most recently from c985173 to ae86163 Compare October 2, 2026 18:59

ymote and others added 3 commits October 2, 2026 15:28

ymote force-pushed the feat/glance-digest-source branch from ae86163 to 6106cc3 Compare October 2, 2026 22:31

ymote merged commit 9a5c4fa into main Oct 2, 2026
3 of 5 checks passed

This was referenced Oct 5, 2026

docs: bring docs/ up to date with the code and rewrite the guides to the README's standard #333

Merged

Octoscript apps cannot play model.audio or model.video results: wire AudioPlayer, document Video #403

Open

Navigation Menu

Repository navigation

glance: resolve sys.digest sources from host-held digests (ADR 0002 M4) - #87

glance: resolve sys.digest sources from host-held digests (ADR 0002 M4)#87
ymote merged 3 commits into
mainfrom
feat/glance-digest-source

ymote commented Sep 28, 2026 •

edited

Loading

Uh oh!

ymote commented Oct 2, 2026

Uh oh!

Uh oh!

ymote commented Oct 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ymote commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Where digests come from (crates/shell/src/glance_digest.rs)

Binding (glance.rs)

Retention

M4 card

Tests (all pass locally)

Headless render (before the rebase; not repeated)

Open

Uh oh!

ymote commented Oct 2, 2026

Uh oh!

Uh oh!

ymote commented Oct 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ymote commented Sep 28, 2026 •

edited

Loading

Where digests come from (`crates/shell/src/glance_digest.rs`)

Binding (`glance.rs`)