Skip to content

feat(pdf): PDF object layer — M6 6.1 - #44

Merged
aenawi merged 1 commit into
mainfrom
aenawi/pdf-object-layer
Sep 7, 2026
Merged

aenawi merged 1 commit into
mainfrom
aenawi/pdf-object-layer

Conversation

@aenawi

@aenawi aenawi commented Sep 7, 2026

Copy link
Copy Markdown
Owner

What

mirsam-pdf — the object layer of PLAN §6.1. The package layer of a format that is not a package: mirsam-ooxml opens a ZIP and asks for a part by name, this crate opens a file and asks for an object by number, from the body, from inside a compressed object stream, or from a scan when the cross-reference table has gone stale.

Both spellings of the cross-reference, /Prev chains, hybrid-reference files, object streams, the general-purpose stream filters with the PNG and TIFF predictors, the page tree with the four attributes §7.7.3.4 inherits down it, and content streams decoded and joined.

Nothing is wired into the CLI. mirsam audit still turns a .pdf away, READABLE is unchanged, and no PDF joins tests/fixtures/. Text extraction is §6.2, and a binary that accepted a .pdf before it could read one would be making a claim this item has not earned. When the DocumentReader lands it arrives as a diff on a real document, the way quarterly-page.html records the read-only refusal now.

Why

The parser is written here and the decompressor is taken

§6.1 asked for this as an ADR rather than a Cargo.toml line, and ADR 0010 is the record. What settled it was not the size of ISO 32000-1 but which half of it mirsam asks for: the object graph and the content streams are §7.2 to §7.8, and everything from §8 on is the rendering this project has said it will never do.

The deciding cost was the second one. A library's object model becomes the adapter's vocabulary, and with it every answer to is a dangling reference null, is a wrong /Length readable, is a stale startxref fatal. Those are exactly the honesty decisions §6.2 turns on, and inheriting them from a dependency would have been inheriting them unexamined.

flate2 is taken for FlateDecode, held to the zlib-rs backend zip already resolves — the same rule ttf-parser is pinned under. The lockfile diff is one workspace member and no third-party crate.

What the file says about itself is checked, not believed

/Length may be an indirect reference — the value sits in an object the cross-reference table has not been parsed yet to find — and plenty of writers get the direct form wrong by a byte or two. So it is a hint, verified against endstream: taken when the bytes it names are followed by one, and the data runs to the next endstream otherwise. One rule covers indirect, absent and simply wrong, and no caller had to be handed a resolver that cannot exist yet.

Newest wins, and it is the walk that enforces it

Every insertion down the /Prev chain is first-wins, so an object an update redefined is found in the section that redefined it, and a free entry in a newer section hides one an older section still names — which is what a deletion is. The trailers merge the same way, which is why a /Root a signature moved is the one used.

A hybrid file's /XRefStm is read before the classic table beside it, because that table deliberately understates what the file holds (§7.5.8.4). A reader that took it as the whole truth would draw a blank page and report nothing wrong with it. The fixture proves it: the ToUnicode CMap in hybrid.pdf is reachable only through the stream.

A stale startxref is recovered from, not refused

Offsets go stale — a file concatenated by a script, truncated by a transfer or edited by hand — and every viewer opens it anyway by scanning for obj. So does this. A document reported unreadable while a reader opens it is the wrong half of standing rule 4. Reconstruction says so out loud rather than hiding it: the cross-reference table and the page tree each report when they had to be rebuilt.

The refusals are the design

  • Encryption. Refused at Pdf::open with the algorithm read out of the /Encrypt dictionary that §7.6.1 keeps in the clear — AES-256 (/Standard security handler, V5 R6) — and a handler this tool does not know is named rather than guessed at. Nothing decrypts, including the very common permissions-only wrapper with an empty user password: a tool that opened those is one nobody could run on a file they were given. The alternative was extracting nothing and reporting a clean file.
  • Rasterising. DCTDecode, JPXDecode, CCITTFaxDecode and JBIG2Decode are named, never decoded. A stream behind one is not text this crate failed to read, it is text that is not in the file — so a scanned page reaches unread as 5 0 R — /DCTDecode, a filter this tool does not decode. ADR 0009 reaching the format that needs it most, and note_unread is open so §6.2's own unread sources (a font with no ToUnicode map among them) land in one list rather than two.
  • Writing. No DocumentWriter here or planned. A broken Arabic PDF is rebuilt from its source document.

Verification

make verify clean — check-version, fmt-check, lint (-D warnings), test, doc (RUSTDOCFLAGS=-D warnings) and build. No existing test changed; the golden corpus is untouched because no reported output moved.

Seventy new tests over eight fixtures, one per generator family — classic table, cross-reference stream with object streams, hybrid, incremental update, stale startxref, AES-256, a distiller's ASCII85Decode/FlateDecode chain beside ASCIIHexDecode/RunLengthDecode/LZWDecode, and a scanned page. Written byte by byte by scripts/make-pdf-fixture.py (make pdfs), for the reason the OOXML generators avoid python-pptx: a fixture built by the library under test proves only that the library agrees with itself.

They sit in crates/mirsam-pdf/tests/pdfs/ rather than tests/fixtures/ on purpose — the golden corpus records what the binary does to a document, and the binary does not read a .pdf yet.

Every prefix and every single-byte corruption of all eight comes back as a document or as an error, and never as a panic. That is what the explicit bounds on nesting depth, reference-chain length, /Prev chain length and page-tree depth are there for: mirsam is run on files people were sent, which is the threat model a document tool has whether or not it says so.

Docs

PLAN.md §6.1 → [x] and M6 → [~] with the reasoning and acceptance; ARCHITECTURE.md gains the crate in the layout; CHANGELOG.md an Unreleased entry; AGENTS.md a note on why these fixtures are not corpus documents and what changes when the reader lands; Makefile a pdfs target, also called from fixtures.

The package layer of a format that is not a package. `mirsam-ooxml` opens
a ZIP and asks for a part by name; `mirsam-pdf` opens a file and asks for
an object by number — from the body, from inside a compressed object
stream, or from a scan when the cross-reference table has gone stale.

Both spellings of the cross-reference, `/Prev` chains, hybrid-reference
files, object streams, the general-purpose stream filters, the page tree
with the four attributes it inherits, and content streams decoded and
joined. Nothing is wired into the CLI: `mirsam audit` still turns a
`.pdf` away, because text extraction is §6.2 and a binary that accepted
one before it could read it would be making a claim this item has not
earned.

The parser is written here and the decompressor is taken, which §6.1
asked for as an ADR rather than a `Cargo.toml` line. What settled it was
not the size of ISO 32000-1 but which half of it mirsam asks for: the
object graph and the content streams are §7.2 to §7.8, and everything
from §8 on is the rendering this project has said it will never do. The
deciding cost was the second one — a library's object model becomes the
adapter's vocabulary, and with it every answer to *is a dangling
reference null, is a wrong `/Length` readable, is a stale `startxref`
fatal*. Those are exactly the honesty decisions §6.2 turns on, and
inheriting them from a dependency would have been inheriting them
unexamined. `flate2` is taken for `FlateDecode`, held to the `zlib-rs`
backend `zip` already resolves, so the lockfile gains one workspace
member and no third-party crate. See ADR 0010.

What the file says about itself is checked rather than believed. A
`/Length` may be an indirect reference — the value is in an object the
table has not been parsed yet to find — and plenty of writers get the
direct form wrong by a byte or two, so it is a hint, verified against
`endstream`. One rule covers indirect, absent and simply wrong, and no
caller had to be handed a resolver that cannot exist yet.

Newest wins, and it is the walk that enforces it. Every insertion is
first-wins down the `/Prev` chain, so an object an update redefined is
found in the section that redefined it, and a free entry in a newer
section *hides* one an older section still names. The trailers merge the
same way, which is why a `/Root` a signature moved is the one used. A
hybrid file's `/XRefStm` is read before the classic table beside it,
because that table deliberately understates what the file holds — a
reader that took it as the whole truth would draw a blank page and
report nothing wrong with it.

A stale `startxref` is recovered from rather than refused. Offsets go
stale, and every viewer opens such a file anyway by scanning for `obj`:
a document reported unreadable while a reader opens it is the wrong half
of standing rule 4. `Reconstruction` says so out loud instead of hiding
it.

The refusals are the design. An encrypted document is refused at
`Pdf::open` with the algorithm read out of the `/Encrypt` dictionary
§7.6.1 keeps in the clear — `AES-256 (/Standard security handler, V5
R6)`, and a handler this tool does not know is named rather than guessed
at. Nothing decrypts, including the permissions-only wrapper with an
empty user password: a tool that opened those is one nobody could run on
a file they were given. The four image codecs are named, never decoded,
because a stream behind `DCTDecode` is not text this crate failed to
read — it is text that is not in the file. That is ADR 0009 reaching the
format that needs it most, and `note_unread` is open so §6.2's own
unread sources reach one list rather than two.

Seventy tests over eight fixtures, one per generator family, written
byte by byte by `scripts/make-pdf-fixture.py` — a fixture built by the
library under test proves only that the library agrees with itself. They
sit beside the crate rather than in `tests/fixtures/`, because the
golden corpus is what the *binary* does to a document and the binary
does not read a `.pdf` yet. Every prefix and every single-byte
corruption of all eight comes back as a document or as an error and
never as a panic, which is what the bounds on nesting depth,
reference-chain length, `/Prev` chain length and page-tree depth are
for: this tool is run on files people were sent.
@aenawi
aenawi merged commit 9e5cab8 into main Sep 7, 2026
7 checks passed
@aenawi
aenawi deleted the aenawi/pdf-object-layer branch September 7, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant