Repository navigation
feat(pdf): PDF object layer — M6 6.1 - #44
Merged
Merged
Conversation
The package layer of a format that is not a package. `mirsam-ooxml` opens a ZIP and asks for a part by name; `mirsam-pdf` opens a file and asks for an object by number — from the body, from inside a compressed object stream, or from a scan when the cross-reference table has gone stale. Both spellings of the cross-reference, `/Prev` chains, hybrid-reference files, object streams, the general-purpose stream filters, the page tree with the four attributes it inherits, and content streams decoded and joined. Nothing is wired into the CLI: `mirsam audit` still turns a `.pdf` away, because text extraction is §6.2 and a binary that accepted one before it could read it would be making a claim this item has not earned. The parser is written here and the decompressor is taken, which §6.1 asked for as an ADR rather than a `Cargo.toml` line. What settled it was not the size of ISO 32000-1 but which half of it mirsam asks for: the object graph and the content streams are §7.2 to §7.8, and everything from §8 on is the rendering this project has said it will never do. The deciding cost was the second one — a library's object model becomes the adapter's vocabulary, and with it every answer to *is a dangling reference null, is a wrong `/Length` readable, is a stale `startxref` fatal*. Those are exactly the honesty decisions §6.2 turns on, and inheriting them from a dependency would have been inheriting them unexamined. `flate2` is taken for `FlateDecode`, held to the `zlib-rs` backend `zip` already resolves, so the lockfile gains one workspace member and no third-party crate. See ADR 0010. What the file says about itself is checked rather than believed. A `/Length` may be an indirect reference — the value is in an object the table has not been parsed yet to find — and plenty of writers get the direct form wrong by a byte or two, so it is a hint, verified against `endstream`. One rule covers indirect, absent and simply wrong, and no caller had to be handed a resolver that cannot exist yet. Newest wins, and it is the walk that enforces it. Every insertion is first-wins down the `/Prev` chain, so an object an update redefined is found in the section that redefined it, and a free entry in a newer section *hides* one an older section still names. The trailers merge the same way, which is why a `/Root` a signature moved is the one used. A hybrid file's `/XRefStm` is read before the classic table beside it, because that table deliberately understates what the file holds — a reader that took it as the whole truth would draw a blank page and report nothing wrong with it. A stale `startxref` is recovered from rather than refused. Offsets go stale, and every viewer opens such a file anyway by scanning for `obj`: a document reported unreadable while a reader opens it is the wrong half of standing rule 4. `Reconstruction` says so out loud instead of hiding it. The refusals are the design. An encrypted document is refused at `Pdf::open` with the algorithm read out of the `/Encrypt` dictionary §7.6.1 keeps in the clear — `AES-256 (/Standard security handler, V5 R6)`, and a handler this tool does not know is named rather than guessed at. Nothing decrypts, including the permissions-only wrapper with an empty user password: a tool that opened those is one nobody could run on a file they were given. The four image codecs are named, never decoded, because a stream behind `DCTDecode` is not text this crate failed to read — it is text that is not in the file. That is ADR 0009 reaching the format that needs it most, and `note_unread` is open so §6.2's own unread sources reach one list rather than two. Seventy tests over eight fixtures, one per generator family, written byte by byte by `scripts/make-pdf-fixture.py` — a fixture built by the library under test proves only that the library agrees with itself. They sit beside the crate rather than in `tests/fixtures/`, because the golden corpus is what the *binary* does to a document and the binary does not read a `.pdf` yet. Every prefix and every single-byte corruption of all eight comes back as a document or as an error and never as a panic, which is what the bounds on nesting depth, reference-chain length, `/Prev` chain length and page-tree depth are for: this tool is run on files people were sent.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
mirsam-pdf— the object layer of PLAN §6.1. The package layer of a format that is not a package:mirsam-ooxmlopens a ZIP and asks for a part by name, this crate opens a file and asks for an object by number, from the body, from inside a compressed object stream, or from a scan when the cross-reference table has gone stale.Both spellings of the cross-reference,
/Prevchains, hybrid-reference files, object streams, the general-purpose stream filters with the PNG and TIFF predictors, the page tree with the four attributes §7.7.3.4 inherits down it, and content streams decoded and joined.Nothing is wired into the CLI.
mirsam auditstill turns a.pdfaway,READABLEis unchanged, and no PDF joinstests/fixtures/. Text extraction is §6.2, and a binary that accepted a.pdfbefore it could read one would be making a claim this item has not earned. When theDocumentReaderlands it arrives as a diff on a real document, the wayquarterly-page.htmlrecords the read-only refusal now.Why
The parser is written here and the decompressor is taken
§6.1 asked for this as an ADR rather than a
Cargo.tomlline, and ADR 0010 is the record. What settled it was not the size of ISO 32000-1 but which half of it mirsam asks for: the object graph and the content streams are §7.2 to §7.8, and everything from §8 on is the rendering this project has said it will never do.The deciding cost was the second one. A library's object model becomes the adapter's vocabulary, and with it every answer to is a dangling reference null, is a wrong
/Lengthreadable, is a stalestartxreffatal. Those are exactly the honesty decisions §6.2 turns on, and inheriting them from a dependency would have been inheriting them unexamined.flate2is taken forFlateDecode, held to thezlib-rsbackendzipalready resolves — the same rulettf-parseris pinned under. The lockfile diff is one workspace member and no third-party crate.What the file says about itself is checked, not believed
/Lengthmay be an indirect reference — the value sits in an object the cross-reference table has not been parsed yet to find — and plenty of writers get the direct form wrong by a byte or two. So it is a hint, verified againstendstream: taken when the bytes it names are followed by one, and the data runs to the nextendstreamotherwise. One rule covers indirect, absent and simply wrong, and no caller had to be handed a resolver that cannot exist yet.Newest wins, and it is the walk that enforces it
Every insertion down the
/Prevchain is first-wins, so an object an update redefined is found in the section that redefined it, and a free entry in a newer section hides one an older section still names — which is what a deletion is. The trailers merge the same way, which is why a/Roota signature moved is the one used.A hybrid file's
/XRefStmis read before the classic table beside it, because that table deliberately understates what the file holds (§7.5.8.4). A reader that took it as the whole truth would draw a blank page and report nothing wrong with it. The fixture proves it: theToUnicodeCMap inhybrid.pdfis reachable only through the stream.A stale
startxrefis recovered from, not refusedOffsets go stale — a file concatenated by a script, truncated by a transfer or edited by hand — and every viewer opens it anyway by scanning for
obj. So does this. A document reported unreadable while a reader opens it is the wrong half of standing rule 4.Reconstructionsays so out loud rather than hiding it: the cross-reference table and the page tree each report when they had to be rebuilt.The refusals are the design
Pdf::openwith the algorithm read out of the/Encryptdictionary that §7.6.1 keeps in the clear —AES-256 (/Standard security handler, V5 R6)— and a handler this tool does not know is named rather than guessed at. Nothing decrypts, including the very common permissions-only wrapper with an empty user password: a tool that opened those is one nobody could run on a file they were given. The alternative was extracting nothing and reporting a clean file.DCTDecode,JPXDecode,CCITTFaxDecodeandJBIG2Decodeare named, never decoded. A stream behind one is not text this crate failed to read, it is text that is not in the file — so a scanned page reachesunreadas5 0 R — /DCTDecode, a filter this tool does not decode. ADR 0009 reaching the format that needs it most, andnote_unreadis open so §6.2's own unread sources (a font with noToUnicodemap among them) land in one list rather than two.DocumentWriterhere or planned. A broken Arabic PDF is rebuilt from its source document.Verification
make verifyclean —check-version,fmt-check,lint(-D warnings),test,doc(RUSTDOCFLAGS=-D warnings) andbuild. No existing test changed; the golden corpus is untouched because no reported output moved.Seventy new tests over eight fixtures, one per generator family — classic table, cross-reference stream with object streams, hybrid, incremental update, stale
startxref, AES-256, a distiller'sASCII85Decode/FlateDecodechain besideASCIIHexDecode/RunLengthDecode/LZWDecode, and a scanned page. Written byte by byte byscripts/make-pdf-fixture.py(make pdfs), for the reason the OOXML generators avoidpython-pptx: a fixture built by the library under test proves only that the library agrees with itself.They sit in
crates/mirsam-pdf/tests/pdfs/rather thantests/fixtures/on purpose — the golden corpus records what the binary does to a document, and the binary does not read a.pdfyet.Every prefix and every single-byte corruption of all eight comes back as a document or as an error, and never as a panic. That is what the explicit bounds on nesting depth, reference-chain length,
/Prevchain length and page-tree depth are there for: mirsam is run on files people were sent, which is the threat model a document tool has whether or not it says so.Docs
PLAN.md§6.1 →[x]and M6 →[~]with the reasoning and acceptance;ARCHITECTURE.mdgains the crate in the layout;CHANGELOG.mdan Unreleased entry;AGENTS.mda note on why these fixtures are not corpus documents and what changes when the reader lands;Makefileapdfstarget, also called fromfixtures.