Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 39 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ binary: no Python, no Node, no Office, no runtime of any kind on the target
machine.

`audit` reads `.pptx`, `.docx`, `.xlsx` and `.html`/`.htm`. `repair` writes
`.pptx` and `.xlsx`: the other two are refused as readable formats without a
`.pptx`, `.docx` and `.xlsx`: `.html` is refused as a readable format without a
writer, which is a usage error (exit `2`) and not a claim that the document was
not understood.

Expand Down Expand Up @@ -176,6 +176,41 @@ through untouched, and mechanically so: `xl/workbook.xml` is not a part a
repair plan can name, and a cell repair edits `@s` or the content of a `<v>`
and nothing beside it.

### Repairing a Word document, where two things are refused

A `.docx` repair writes `w:pPr/w:bidi`, `w:pPr/w:jc`, `w:rPr/w:lang@w:bidi`,
`w:rPr/w:rFonts@w:cs`, `w:pPr/w:numPr` and `w:tblPr/w:bidiVisual`, and every
one of them lands in its schema-sequence position — WordprocessingML's
property elements are `xsd:sequence`, so a correct element written in the
wrong place is a document Word offers to repair.

Three things about it are worth knowing before reading a repair report.

- **`w:jc` is written relative, so no direction is resolved to write one.**
The PowerPoint writer has to know the direction a paragraph inherits before
it can put a `Start` down, because `a:pPr/@algn` names physical edges. Word's
does not: `start` is the start edge whichever way the paragraph runs. The
same fact is why **a physical edge is refused** — `SetAlignment(Left)` and
`SetAlignment(Right)` appear in `repairs.skipped`, never as a repair that
failed, because Word has no spelling for "the left of the page whatever the
direction". No Word unit ever produces one, so this is a refusal you will not
see in practice; it is stated so the writer cannot start inventing one.
- **`--convert-bullets` needs a list the document already defines.** Word has
no per-paragraph bullet: a list is a `w:numPr` pointing into the numbering
part. The repair points at a bulleted definition the document already
carries, preferring one that draws the marker the author typed, and
otherwise the fix is **skipped**. A repair cannot add the numbering part
itself — that is a part, a content-type override and a relationship, none of
which is an edit to the paragraph the finding named.
- **`--font` clears `@w:cstheme` on the runs it fills.** `@w:cstheme` is what
Word renders and `@w:cs` the value it caches beside it, so writing the
typeface into the cache and leaving a theme reference standing would change
what the file says and not what a reader sees.

A paragraph inside a text box is a unit of its own and is repaired as one; the
paragraph anchoring the box is not repaired through it, and an `mc:Fallback`
is neither counted nor edited — the same two regions the reader steps over.

### HTML, where the direction is usually not in the document

An `.html` page states direction in three places and the adapter reads all of
Expand Down Expand Up @@ -380,7 +415,9 @@ share a stem: the report is named for it.

A report for a format the tool reads but cannot write holds the refusal
`repair` gave and the exit code it used, in place of a repair report — so the
day a writer lands, it shows up as a diff on a real document.
day a writer lands, it shows up as a diff on a real document. That is what the
two `.docx` reports did when M3's writer landed, and `.html` is what carries
the refusal now.

**The shaping fixtures are fonts, and they are generated too.** The five
under `crates/mirsam-core/tests/fonts/` are written byte by byte by
Expand Down
34 changes: 34 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,40 @@ Working towards byte-preserving `repair` for PPTX. See

### Added

- **The DOCX writer.** `mirsam repair report.docx fixed.docx` works, and
`repair` stops turning a Word document away as a readable format without a
writer. A repair writes `w:pPr/w:bidi`, `w:pPr/w:jc`, `w:rPr/w:lang@w:bidi`,
`w:rPr/w:rFonts@w:cs`, `w:pPr/w:numPr` and `w:tblPr/w:bidiVisual`, each
placed by rank against the schema sequence its parent is — WordprocessingML's
property elements are `xsd:sequence`, and a correct element in the wrong
position is a document Word offers to repair. `word.rs` is the third
vocabulary built on the same token-rewrite scaffold, and `token.rs` did not
have to learn a Word element name to carry it.

**The writer needs no inherited direction, unlike PowerPoint's.**
DrawingML's `algn` names physical edges, so lowering a `Start` there without
knowing the paragraph's direction reproduces the defect being repaired.
Word's `w:jc` is evaluated against the paragraph's own `w:bidi`, so `start`
is the start edge whichever way the paragraph runs — the inheritance pass is
absent from this adapter rather than skipped in it.

**Two repairs are refused, and both refusals are the format's.** Word has no
spelling for a *physical* edge, which is the refusal the conformance suite
already records for the reading side, so `SetAlignment(Left)` appears in
`repairs.skipped` rather than being lowered onto whichever relative edge
lands left today. And a typed bullet becomes a `w:numPr` pointing into the
numbering part, so `--convert-bullets` can only join a list the document
already defines — preferring one that draws the marker the author typed —
and is skipped otherwise. Filling `@w:cs` also clears `@w:cstheme` beside
it, because the theme reference is what Word renders and leaving it standing
would be a repair that changes what the file says and not what a reader
sees. See [`docs/PLAN.md`](docs/PLAN.md) §3.6.

The two corpus Word documents recorded the refusal this replaces, so the
change arrives as a repair report and a part-level diff on
`quarterly-review.docx` — whose `after` audit is now empty — rather than as
a test somebody remembered to update.

- **The XLSX adapter, and the second format `repair` can write.** `mirsam
audit book.xlsx` reads a workbook and `mirsam repair book.xlsx fixed.xlsx`
writes one. A cell is a paragraph and the worksheet around it is the
Expand Down
32 changes: 19 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,8 @@
finds right-to-left, bidirectional and typography defects in PowerPoint, Word,
Excel and HTML files, *proves* each one by resolving what the text will
actually look like when rendered, and repairs them without touching a byte it
was not asked to. PowerPoint and Excel are repaired; Word and HTML are
audit-only so far, and the PDF adapter is scheduled; see
[Roadmap](docs/ROADMAP.md).
was not asked to. PowerPoint, Word and Excel are repaired; HTML is audit-only
so far, and the PDF adapter is scheduled; see [Roadmap](docs/ROADMAP.md).

> **mirsam** (مَرْسَم) — an atelier; the room where things are properly drawn.
> From the root ر-س-م: to draw, to inscribe, to render.
Expand Down Expand Up @@ -154,23 +153,30 @@ so an agent can act on it without opening PowerPoint. See [`AGENTS.md`](AGENTS.m

**v0.1 “Steppe Eagle” — audit only, PPTX only.** Byte-preserving `repair` for
PPTX has landed on `main` and ships as v0.2. Three formats have landed since.
DOCX and HTML are **readers**: `mirsam audit report.docx` works, and `mirsam
audit page.html` reads `dir`, `lang` and the part of CSS that decides direction
— including a stylesheet the page links by a relative path. `repair` refuses
both and says why, because neither writer is built yet. A page linking a
stylesheet on a server is audited without it and the report names what it did
not read, because this tool makes no network calls.

HTML is a **reader**: `mirsam audit page.html` reads `dir`, `lang` and the part
of CSS that decides direction — including a stylesheet the page links by a
relative path. `repair` refuses it and says why, because its writer is not
built. A page linking a stylesheet on a server is audited without it and the
report names what it did not read, because this tool makes no network calls.

XLSX is the second format with a **writer**: `mirsam repair book.xlsx
fixed.xlsx` sets a sheet's `rightToLeft` and a cell's reading order and
alignment, and leaves every formula and defined name exactly as it found them
— a workbook's formatting is shared between its cells, so a repair appends a
format record and repoints the one cell rather than editing what forty cells
are using. A formula's cached result is not judged and is named in the report,
for the reason a stylesheet on a server is. The PDF adapter is specified and
scheduled; see [`docs/ROADMAP.md`](docs/ROADMAP.md). The tool reports what it
can actually verify and nothing more — a discipline inherited from its prior
art.
for the reason a stylesheet on a server is.

DOCX is the third: `mirsam repair report.docx fixed.docx` writes `w:bidi`,
`w:jc`, the complex-script language and font slots, a table's `w:bidiVisual`
and a real list in place of a typed bullet — each in the schema position Word
requires. Two repairs are refused rather than approximated, and both refusals
are the format's: Word has no way to state a *physical* edge, and a typed
bullet can only join a list the document already defines. The PDF adapter is
specified and scheduled; see [`docs/ROADMAP.md`](docs/ROADMAP.md). The tool
reports what it can actually verify and nothing more — a discipline inherited
from its prior art.

## Design

Expand Down
3 changes: 3 additions & 0 deletions crates/mirsam-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,9 @@ fn open_for_repair(path: &Path) -> Result<Box<dyn DocumentWriter>> {
"pptx" => Ok(Box::new(
PptxDocument::open(path).with_context(|| format!("opening {}", path.display()))?,
)),
"docx" => Ok(Box::new(
DocxDocument::open(path).with_context(|| format!("opening {}", path.display()))?,
)),
"xlsx" => Ok(Box::new(
XlsxDocument::open(path).with_context(|| format!("opening {}", path.display()))?,
)),
Expand Down
67 changes: 61 additions & 6 deletions crates/mirsam-cli/tests/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -766,21 +766,76 @@ fn a_correctly_marked_word_document_exits_zero() {

#[test]
fn repairing_a_readable_but_unwritable_format_says_so_rather_than_denying_it() {
// `docx` is not an unknown extension — `audit` reads it. Reporting it as
// `html` is not an unknown extension — `audit` reads it. Reporting it as
// one would be the tool contradicting itself between two commands, so the
// refusal names the real reason and stays a usage error.
let scratch = Scratch::new("docx-repair");
let input = docx(&scratch, "in.docx", "");
let out = run(&["repair", &input, &scratch.path("out.docx")]);
//
// This case was written against `.docx`, which is the format that used to
// answer it. Word gained a writer, so the claim moved to the format that
// still has none rather than being deleted with the adapter that outgrew
// it: what is under test is the refusal, not which format gives it.
let scratch = Scratch::new("html-repair");
let input = scratch.file(
"page.html",
"<p dir=\"rtl\">ارتفع الأداء بنسبة 25%</p>".as_bytes(),
);
let out = run(&["repair", &input, &scratch.path("out.html")]);

assert_eq!(out.code, exit::USAGE);
assert_eq!(out.code, exit::USAGE, "stderr:\n{}", out.stderr);
assert!(
out.stderr.contains("audited") && out.stderr.contains("not yet repaired"),
"stderr:\n{}",
out.stderr
);
// And nothing was written.
assert!(!Path::new(&scratch.path("out.docx")).exists());
assert!(!Path::new(&scratch.path("out.html")).exists());
}

#[test]
fn a_word_document_is_repaired_and_the_repaired_copy_audits_clean() {
// The other side of the case above: `repair` no longer turns a `.docx`
// away, and what it writes is a document the same binary reads back with
// the finding gone.
let scratch = Scratch::new("docx-repaired");
let input = docx(
&scratch,
"broken.docx",
r#"<w:pPr><w:bidi w:val="0"/></w:pPr>"#,
);
let output = scratch.path("fixed.docx");

let out = run(&["repair", &input, &output, "--format", "json"]);
assert_eq!(out.code, exit::OK, "stderr:\n{}", out.stderr);

let json = parse_json(&out);
assert_eq!(json["format"], "docx");
assert!(
!json["repairs"]["applied"].as_array().unwrap().is_empty(),
"{json:#?}"
);
assert!(
json["repairs"]["skipped"].as_array().unwrap().is_empty(),
"nothing in this document is beyond WordprocessingML:\n{json:#?}"
);
assert!(
rule_ids(&json["before"]["diagnostics"]).contains(&"direction-mismatch".to_string()),
"{json:#?}"
);
assert!(
!rule_ids(&json["after"]["diagnostics"]).contains(&"direction-mismatch".to_string()),
"{json:#?}"
);

// The input is never modified, and the direction landed on the paragraph.
let repaired = std::fs::read(&output).unwrap();
let mut archive = zip::ZipArchive::new(std::io::Cursor::new(repaired)).unwrap();
let mut document = String::new();
std::io::Read::read_to_string(
&mut archive.by_name("word/document.xml").unwrap(),
&mut document,
)
.unwrap();
assert!(document.contains(r#"<w:bidi w:val="1"/>"#), "{document}");
}

#[test]
Expand Down
12 changes: 8 additions & 4 deletions crates/mirsam-cli/tests/golden.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,16 @@
//!
//! ## A format the tool reads but cannot write
//!
//! `repair` refuses a `.docx`, and refuses it as a *readable format without a
//! `repair` refuses an `.html`, and refuses it as a *readable format without a
//! writer* rather than as an unknown extension. That refusal is recorded here
//! rather than worked around: the report holds the message and the exit code
//! it produced, so the day the Word writer lands, the corpus shows the change
//! as a diff on a real document instead of as a test somebody remembered to
//! update.
//! it produced, so the day the writer lands, the corpus shows the change as a
//! diff on a real document instead of as a test somebody remembered to update.
//!
//! It did exactly that for Word. The two `.docx` reports held that refusal
//! until M3's writer landed, and the change arrived as a repair report and a
//! part-level diff on `quarterly-review.docx` — which is the review this
//! paragraph exists to force.
//!
//! A diff is a change in behaviour. When it is intended, regenerate the
//! reports and commit them with the change that caused them, so the diff is
Expand Down
Loading
Loading