pdfcraft: update to upstream main, robustness fixes by ymote · Pull Request #502 · OctoSense-org/OctoSense

ymote · 2026-10-11T17:57:29Z

Why

The pdfcraft engine (ADR 0013) was 318 commits behind upstream. Upstream's robustness work covers CMap parsing, JBIG2 dictionaries, decompression bombs, mesh shadings, dash patterns, PDF functions, image resampling, lopdf object streams, hayro nesting, JavaScript util.printf, and content past 256 MiB. This PR brings that work in, as its own change. Font bundling and Chinese editing are out of scope.

What

pdfcraft pins. All ten pdfcraft pins in the root Cargo.toml move to ymote/pdfcraft 51f09eea. That includes #501's pdfcraft-edit and the [patch.crates-io] block for the engine's PDF stack (hayro, hayro-interpret, hayro-jbig2, hayro-syntax, lopdf). The revision is two fork merges:

Update to upstream main (f22b5d42): robustness fixes ymote/pdfcraft#2 (ef202258) merges storytold/pdfcraft main at f22b5d42, keeping history. Upstream already has the fork's own two fixes, the comment dates and the page-label bounds.
Bound the total size of page-label prefixes (the fork's 4 MiB cap, on upstream's code) ymote/pdfcraft#3 (51f09eea) puts back a bound that upstream's code is looser on (below). It is offered upstream as M2: bound the total size of page-label prefixes storytold/pdfcraft#1146.

vectorcraft pins. The four vectorcraft pins move to ymote/vectorcraft 2cd3a1ca (ymote/vectorcraft#1). Here is why:

The engine's hayro-interpret, patched in for the whole workspace, has a third InterpreterWarning, ContentTruncated: page content cut at its safety budget.
vectorcraft-pdf matched the two crates.io variants exhaustively, so the shell with craft-engines no longer compiled (E0004).
vectorcraft#1 adds a wildcard arm; vectorcraft's own steps are green.
Only the vector skill's two revision lines change; its 679 commands are the same.
designcraft and hayro-write, the other users of the patched stack, compile unchanged.

The engine's hardened JBIG2 decoder is now the one built. On main, Cargo.lock lists the hayro-jbig2 patch under [[patch.unused]]. Here is why:

hayro-syntax asks for ^0.3.0.
crates.io's 0.3.1 outranks the engine's vendored 0.3.0, so Cargo builds crates.io's copy.
That copy has none of the engine's limits: 2^28 pixels per bitmap, 2^26 symbol pixels per dictionary, and a bound on decoding steps that don't advance (upstream: one changed byte in a symbol dictionary hung the renderer).

The fix and its guard:

The lock now pins hayro-jbig2 to the patch (cargo update -p hayro-jbig2@0.3.1 --precise 0.3.0). That also drops second copies of fearless_simd and hayro-ccitt.
New test tests/skill.rs the_engines_hardened_pdf_stack_is_the_one_built fails if any of the five patches goes unused or isn't at the engine's revision.
On the lock Cargo first writes for these pins, the test fails with "Cargo.lock lists hayro-jbig2's [patch] as unused".

The page-label bound: looser, then restored.

Before: the fork's 221ffad0 capped the decoded page-label prefixes of a document at 4 MiB in total.
Looser: upstream's code, which replaced it in deps: point makepad at OctoSense-org/makepad #2, has no such total. Each range decodes its own prefix, ranges may share one prefix object, and the only bound left is the 200,000-entry tree limit. A small file could hold about 200 MiB of copies of one 1,024-byte prefix while it opens.
Restored: Bound the total size of page-label prefixes (the fork's 4 MiB cap, on upstream's code) ymote/pdfcraft#3 puts the 4 MiB total back in both readers (inspection and Organize), in upstream's structure, with tests: 5,000 ranges sharing a 1,000-byte prefix are refused, and exactly 4 MiB is still read.
The pdf service's a_hostile_page_label_cannot_kill_the_process passes on every revision.

Cargo.lock (1,875 → 1,882 packages):

every pdfcraft crate moves 0.4.0 → 0.6.0;
new: pdfcraft-geom and pdfcraft-platform, and seven crates.io crates for pdfcraft-sign's new signature algorithms (sha3, keccak, sponge-cursor, ripemd, p521, bp256, bp384);
vectorcraft changes only its revision;
no second copy of any crate the workspace already had (the JBIG2 fix removes two).

New engine commands and their classes

Each class was read from the command's implementation in apps/pdf/host-service/skill/safety-rules.json, and the skill is regenerated.

Command	Class	Why
`command_run`, `command_batch`	code	They run other commands. `command_run` calls the tool behind any registry command id (79 tools, among them `doc_open`, `doc_save`, `sign_document`, `js_run` and `action_run`) with the caller's `params`. `command_batch` runs up to 1,000 of them.
`printer_options`	device	Reads the queue's CUPS PPD under `/etc/cups/ppd` and spawns `lpoptions`.
`doc_create_multiple`	file	Reads up to 1,000 files (PDFs, images, text), then writes `out`, or one PDF per file into `out_dir`.
`fill_sign_add`	safe → file	With the new `path`, it reads an image signature (PNG or JPEG, at most 4 MiB and 4 MP) from a file.
`bookmark_from_structure`	safe	Bookmarks from the document's tagged headings.
`comment_image_preview`	safe	Returns an image signature's picture, or its page rendered without it (at most 600 dpi), as PNG bytes.
`doc_inspect`	safe	`doc_info` for one or every open document.
`fill_sign_date_format`	safe	Sets or reads two session-memory fields (the date pattern and name language). Nothing is saved.
`object_list`, `object_move`	safe	List a page's editable objects, and move them as one undoable edit.
`page_enhance_camera`	safe	Renders the page, warps the given or detected corners upright, and replaces the page's picture. Pixels only, no camera.
`render_preview`	safe	A PNG at most 4,096 pixels a side, never above 72 dpi.

The 144 commands: safe 90, file 33, code 14, device 3, host 4. doc_save's rule now names write_atomic's new staging file, .<name>.<16 hex>.pdfcraft-tmp. It is created exclusively, so a planted file or link at that name is never written through. ADR 0013's count of generated ids is now 4,569 (EN and zh-CN).

fill_sign_add and the pdf service. pdf.fill_sign still calls it with reviewed arguments only.

The service builds {page, type, at}, plus text for text and initials.
kind is never signature.
args::only refuses path, format, language and any other key before engine work.
New test doc_tests::fill_sign_never_reads_a_file: the engine places an image from path when called directly; through pdf.fill_sign every kind refuses those keys, and the document is untouched.
review.rs's safety notes say fill_sign_add is file, and why no file is read.

What PDF Tools or the pdf service could notice

PDF Tools, end to end on the new engine. I ran it on the build host's headless weston, with the recipe in apps/pdftools/tests/evidence/linux-20261011/README.md. The build was cargo build --locked -p octosense --no-default-features --features app-hub,craft-engines, at the runtime's 64 ms script budget. Neither shell log has a script error, an overrun or a panic.

ui.py --only shell, light then dark: passed.
The --edit-sample Edit run, light and dark, then Apply anyway and Apply at once: passed. It repeats the steps of edit-20261011.
No visible change. Every frame matches main's evidence pixel for pixel once cut to the app's area:
- 18 light and 8 dark shell frames from linux-20261011, drawn with pdfcraft d801407;
- the 8 Edit frames from edit-20261011, drawn with 615ff308.
The only differences, neither from the engine:
- the comment times, which are each run's clock;
- the Edit panel's font box, "Times Roman" for "Times-Roman". PDF Tools: honest Edit on real-world PDFs (pdf.edit_check, plain refusals, a browser-printed test page) #501's bundle changed that (font_label) after linux-20261011 was taken.
Edit said the same, word for word, as on 615ff308 in all four states PDF Tools: honest Edit on real-world PDFs (pdf.edit_check, plain refusals, a browser-printed test page) #501 recorded (the title piece and the Chinese piece, light and dark). That covers the five-pieces note, "Unnamed font · 27.8 pt", the Helvetica and room warnings, and the Chinese refusal.
Text: word gaps follow position on the line (Text extraction: word gaps follow position on the line, not drawing order (#1062) storytold/pdfcraft#1074). Text turned at an angle reads along its own baseline (M2: text turned at any angle reads as lines along its own baseline storytold/pdfcraft#735). pdf.text, pdf.lines and pdf.find can differ slightly on such files.
Edit: a line split across content-stream pieces can be edited (Edit text: lines split across content stream pieces are listed and edited (fixes #155) storytold/pdfcraft#628).
Comments: a reply to a reply is listed (fix(comments): keep a reply to a reply in the comment listing (#806) storytold/pdfcraft#839).
Forms:
- fields that exist only as page widgets join the form (Forms: fields listed only as page widgets join the form storytold/pdfcraft#649);
- filled appearances use the field's existing simple font (Forms: encode appearances with the existing simple font storytold/pdfcraft#1044);
- radio and check boxes take their /Opt export values (Forms: radio groups and check boxes accept their /Opt export values storytold/pdfcraft#532).
Merge: a page repeated in one selection gets its own annotations (Extract/Combine: a page repeated in one selection gets its own annotations (#818) storytold/pdfcraft#852).
Fill & Sign: the date is unchanged (m/d/yyyy). Typed signatures stay upright on rotated pages (Typed and drawn signatures stay upright on rotated pages (fixes #299) storytold/pdfcraft#562).
JBIG2: a hostile image is refused instead of decoded without bounds.
Page labels: a tree whose decoded prefixes pass 4 MiB shows physical page numbers with one warning, as before the sync.
Not affected: no existing pdf-service test needed a change. PDF Tools: honest Edit on real-world PDFs (pdf.edit_check, plain refusals, a browser-printed test page) #501's edit_check and refusal tests pass unchanged; they check exact fonts, boxes, piece counts and refusal words.

Tests

On the Linux build host, Rust 1.99.0, on main cf06ac5 plus this branch. The tested tree's 12 changed files are byte for byte this head's.

Command	Result
`OCTOSENSE_SKILL_REGEN=1 cargo test --locked -p octosense-pdf-service --test skill`	regenerated (revision lines); 2 passed
`cargo test --locked -p octosense-pdf-service`	72 passed, 2 ignored by design (one prints timings, one is a cold-start scenario another test runs)
`cargo test --locked -p octosense-engine-area`	24 passed
`cargo clippy --locked -p octosense-pdf-service --all-targets -- -D warnings`	pass
`OCTOSENSE_SKILL_REGEN=1 cargo test --locked -p octosense-vector-service --test skill`	regenerated (revision lines); 1 passed
`cargo test --locked -p octosense-vector-service`	38 passed
`cargo clippy --locked -p octosense-vector-service --all-targets -- -D warnings`	pass
`cargo test --locked -p octosense-design-service`	17 passed
from `phone/`: `cargo test --locked --features mobile-apps,craft-engines -p octosense-shell host_tools`	126 passed, among them `every_granted_engine_tool_answers_within_its_declared_result`
`cargo check --locked -p octosense`	pass
`cargo build --locked -p octosense --no-default-features --features app-hub,craft-engines`	pass (the journeys' shell)
the JBIG2 guard on the lock Cargo first writes for these pins	fails as it should: "Cargo.lock lists hayro-jbig2's [patch] as unused"

The pdf-service run includes these tests:

PDF Tools: honest Edit on real-world PDFs (pdf.edit_check, plain refusals, a browser-printed test page) #501's a_printed_title_reads_as_pieces_in_subset_fonts, an_edit_check_says_when_the_font_or_the_room_changes, an_edit_check_foretells_the_edit, an_edit_the_engine_refuses_is_refused_in_plain_words and an_edit_check_on_a_plain_pdf_finds_nothing_to_say;
a_hostile_comment_date_is_read_safely, a_comment_date_cut_inside_a_character_still_opens and a_hostile_page_label_cannot_kill_the_process;
a_forms_own_scripts_never_run, work_goes_on_after_each_save and fill_sign_never_reads_a_file.

On the Mac, these pass:

python3 tools/setup.py --check --cargo;
python3 tools/native_apps.py --check;
bash tools/check-shell-graph.sh -p octosense, and from phone/, bash ../tools/check-shell-graph.sh -p octosense-home.

Not run:

ci-local and GitHub CI (the coordinator gates this);
the full macOS desktop build;
Windows.

rom paths: this PR touches no rom/, phone/android/, phone/resources/android/, phone/tests/ or tools/kernel-artifact.py path, so rom.yml doesn't run.

🤖 Generated with Claude Code

Moves the pdfcraft engine and its patched PDF stack from 615ff308 to 51f09eea: ymote/pdfcraft's merge of storytold/pdfcraft main (f22b5d42, 318 commits, ymote/pdfcraft#2) plus the fork's 4 MiB cap on the total of decoded page-label prefixes (ymote/pdfcraft#3), which upstream's code had dropped. All ten pdfcraft pins in the root Cargo.toml move, #501's pdfcraft-edit included. - The lock pins hayro-jbig2 to the engine's hardened copy. Its patch had been unused: crates.io's 0.3.1 outranked the vendored 0.3.0, so JBIG2 images were decoded without the engine's bounds on bitmap and dictionary pixels and on decoding steps that never advance. That also drops second copies of fearless_simd and hayro-ccitt. tests/skill.rs now fails if any of the five patches goes unused or isn't at the engine's revision. - vectorcraft moves to 2cd3a1ca (ymote/vectorcraft#1), whose PDF import takes the warning the engine's hayro-interpret adds (ContentTruncated). The patch reaches vectorcraft-pdf too, which otherwise stops compiling. Only the vector skill's revision lines change. - The pdf skill is regenerated: twelve new engine commands, each classified from its implementation in skill/safety-rules.json, and fill_sign_add is file now (its new `path` reads an image signature). ADR 0013's count of generated ids follows (4,569). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

fill_sign_add now reads an image signature from the file its `path` names. pdf.fill_sign builds the command's arguments itself, and args::only refuses `path`, `format` and `language` before any engine work. doc_tests::fill_sign_never_reads_a_file proves both sides: the engine places the image when called directly, while through the service every kind refuses those keys and the document stays untouched. review.rs's safety notes say fill_sign_add is file, and why it still runs as a plain document edit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ymote · 2026-10-11T20:12:02Z

Local CI passed on 0533955 (tools/ci-local.sh --only all --linux-host --offload, 15m49s, Darwin arm64; the ubuntu jobs and the macOS jobs' portable steps (their macOS steps ran here) on the Linux build host, Linux x86_64).

GitHub would run for this PR: desktop.yml, phone.yml, apps.yml. Merged on this local pass (macOS runner queue); GitHub CI runs on the merge commit on main.

Job	Step	Result	Time
ci-local / drift	The local mapping fits the workflows	PASS	-
desktop.yml / desktop	Prepare the pinned framework sources	PASS	1s
desktop.yml / desktop	Compile the desktop with the shared runtime	PASS	5m13s
desktop.yml / desktop	Desktop tools and scripts	PASS	18s
desktop.yml / desktop (linux)	Prepare the pinned framework sources	PASS	0s
desktop.yml / desktop (linux)	One copy of every shell source	PASS	-
desktop.yml / desktop (linux)	In-app updater and installer handoff	PASS	1m37s
desktop.yml / desktop (linux)	One Makepad, App Hub, octos and Rinx in the locked graph	PASS	2s
desktop.yml / desktop (linux)	The native apps' generated places match native-apps.json	PASS	0s
desktop.yml / native-host-api	Prepare the pinned framework sources	PASS	1s
desktop.yml / native-host-api	Build the native acceptance host and pinned Hub CLI	PASS	1m26s
desktop.yml / native-host-api	Signed app tool calls a real OS permission API	PASS	3s
phone.yml / home	Prepare pinned sources and the reviewed runtime patch	PASS	1s
phone.yml / home	Validate launcher icons and Apple asset catalogs	PASS	4s
phone.yml / home	Compile Home and its bundled modules	PASS	4m00s
phone.yml / home (linux)	Prepare pinned sources and the reviewed runtime patch	PASS	0s
phone.yml / home (linux)	Verify the single runtime graph	PASS	2s
phone.yml / home (linux)	The native apps' generated places match native-apps.json	PASS	0s
phone.yml / home (linux)	The pinned octos revision	PASS	0s
phone.yml / home (linux)	Build octos (unless cached)	PASS	-
phone.yml / home (linux)	Build Home, App Hub admission and runtime policy tests	PASS	3m19s
phone.yml / home (linux)	Test Home, App Hub admission and runtime policy	PASS	52s
phone.yml / home (linux)	Two-lane scenario (real kernel, scripted model)	PASS	28s
phone.yml / home (linux)	Build opt-in AppCard tests	PASS	2m48s
phone.yml / home (linux)	Test opt-in AppCard	PASS	1s
phone.yml / home (linux)	Build craft-engine relay and system-chat tests	PASS	4m04s
phone.yml / home (linux)	Test craft-engine relay and system chat	PASS	4s
phone.yml / home (linux)	Build linked Terminal tests	PASS	2m28s
phone.yml / home (linux)	Test linked Terminal	PASS	1s
phone.yml / home (linux)	Build Wasm and runtime policy tests	PASS	1m16s
phone.yml / home (linux)	Test Wasm and runtime policy	PASS	11s
apps.yml / services (linux)	python3 tools/setup.py	PASS	0s
apps.yml / services (linux)	Shared system app interface	PASS	0s
apps.yml / services (linux)	Kernel service	PASS	18s
apps.yml / services (linux)	OctoSense release updater	PASS	6s
apps.yml / services (linux)	App peers	PASS	29s
apps.yml / services (linux)	AI providers config	PASS	6s
apps.yml / services (linux)	Connected provider protocols and Linux host adapter	PASS	46s
apps.yml / services (linux)	L0 in-card chat (sys.chat, host side)	PASS	10s
apps.yml / services (linux)	System toolbox (workflow templates)	PASS	1m37s
apps.yml / kernel-security (linux)	python3 tools/setup.py	PASS	0s
apps.yml / kernel-security (linux)	The pinned octos revision	PASS	0s
apps.yml / kernel-security (linux)	Build octos (unless cached)	PASS	-
apps.yml / kernel-security (linux)	Real-kernel tests	PASS	41s
apps.yml / apps (linux)	python3 tools/setup.py	PASS	0s
apps.yml / apps (linux)	Host services (Mail, AI providers, News)	PASS	2m05s
apps.yml / apps (linux)	Connected accounts and Rinx editor	PASS	1m40s
apps.yml / apps (linux)	Shell AI services (crates/ai-host)	PASS	2m59s
apps.yml / apps (linux)	AppCard runtime setup safeguards	PASS	0s
apps.yml / apps (linux)	AppCard clippy (own crates; shared runtime compiles)	PASS	28s
apps.yml / apps (linux)	AppCard transport and store	PASS	7s
apps.yml / apps (linux)	One Makepad, App Hub, octos and Rinx in the locked graph	PASS	3s
rom.yml / product (linux)	python3 -m unittest discover -s tests -v	SKIPPED (!)	-
rom.yml / product (linux)	Check generated Agent Binder client	SKIPPED (!)	-
rom.yml / product (linux)	bash -n scripts/stage-forks.sh scripts/publish-release.sh scripts/apply-to-tree.sh scripts/build-rom.sh	PASS	-
rom.yml / web-installer (linux)	npm ci --ignore-scripts --no-audit --no-fund	SKIPPED (!)	-
rom.yml / web-installer (linux)	npm test	SKIPPED (!)	-
rom.yml / web-installer (linux)	npx playwright install --with-deps chromium	SKIPPED (!)	-
rom.yml / web-installer (linux)	npm run test:browser	SKIPPED (!)	-
rom.yml / web-installer (linux)	python3 -m unittest discover -s tests -p test_installer_manifest.py -v	SKIPPED (!)	-
rom.yml / web-installer (linux)	bash -n scripts/release.sh	SKIPPED (!)	-
linux-host / sandbox (linux)	python3 tools/setup.py	PASS	0s
linux-host / sandbox (linux)	Process sandbox on Linux (Landlock, seccomp)	PASS	1m50s

Skipped (these did NOT pass; (!) marks a skip that blocks tools/ci-local-merge.sh):

rom.yml / product (linux): python3 -m unittest discover -s tests -v: javac -version does not work here: install a JDK (17 or newer) and put its bin/ on PATH or set JAVA_HOME
rom.yml / product (linux): Check generated Agent Binder client: needs build-tools/35.0.0/aidl, platforms/android-35/framework.aidl in the Android SDK (no Android SDK found): install the SDK packages 'build-tools;35.0.0' and 'platforms;android-35' (sdkmanager) and set ANDROID_HOME
rom.yml / web-installer (linux): npm ci --ignore-scripts --no-audit --no-fund: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH
rom.yml / web-installer (linux): npm test: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH
rom.yml / web-installer (linux): npx playwright install --with-deps chromium: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH
rom.yml / web-installer (linux): npm run test:browser: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH
rom.yml / web-installer (linux): python3 -m unittest discover -s tests -p test_installer_manifest.py -v: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH
rom.yml / web-installer (linux): bash -n scripts/release.sh: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH

Notes:

GitHub runs Python 3.12; this run used Python 3.9.6
GitHub runs Python 3.12; the Linux host used Python 3.14.4

Native GitHub acceptance passed on the same head: embedded-browser.yml.

Brings #500 up to main (72 commits, including the pdfcraft repin #502, the deck service #503 and the WebAssembly deadline fix #505) so it can be gated on its exact head. No conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ymote mentioned this pull request Oct 11, 2026

PDF import: take the warnings a patched hayro-interpret adds ymote/vectorcraft#1

Merged

ymote force-pushed the deps/pdfcraft-upstream branch from 0a82c78 to 0c7c661 Compare October 11, 2026 19:41

ymote marked this pull request as ready for review October 11, 2026 19:41

ymote and others added 2 commits October 11, 2026 12:54

ymote force-pushed the deps/pdfcraft-upstream branch from 0c7c661 to 0533955 Compare October 11, 2026 19:54

ymote merged commit 32dcfbb into main Oct 11, 2026
4 of 9 checks passed

ymote deleted the deps/pdfcraft-upstream branch October 11, 2026 20:12

ymote mentioned this pull request Oct 11, 2026

deck service v2: open documents for Quick Deck v2 #503

Merged

Navigation Menu

Repository navigation

pdfcraft: update to upstream main, robustness fixes - #502

pdfcraft: update to upstream main, robustness fixes#502
ymote merged 2 commits into
mainfrom
deps/pdfcraft-upstream

ymote commented Oct 11, 2026 •

edited

Loading

Uh oh!

ymote commented Oct 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ymote commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

New engine commands and their classes

What PDF Tools or the pdf service could notice

Tests

Uh oh!

ymote commented Oct 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ymote commented Oct 11, 2026 •

edited

Loading