Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 52 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ shared code. See [`CREDITS.md`](CREDITS.md).
```bash
mirsam audit deck.pptx --format json # full diagnostic model
mirsam audit deck.pptx --strict # warnings block too
mirsam audit deck.pptx --fonts # also check the fonts installed here
mirsam repair deck.pptx fixed.pptx --format json # repaired copy, both audits
mirsam explain "<text>" # reproduce a defect with no document
mirsam rules --format json # every check and its id
Expand Down Expand Up @@ -92,6 +93,40 @@ agrees with the text, reported as an absent one where it contradicts it, with
`evidence.inherited_from` naming the style. The repair still writes to the
table, never to the style.

### The two font checks, which are off by default

`font-coverage` and `shaping-broken` are the only rules that ask a question
about **the machine running mirsam** rather than about the document, so they
run only when `--fonts` asks for them. Every report says which of the two
audits it is — `fonts NOT RUN` in the human output, `"fonts": {"checked":
false}` in JSON — and you must read that before concluding anything from their
silence. A check that did not run is `NOT RUN`, never a pass.

`--font-dir <DIR>` (repeatable) searches those directories instead of the
platform's, which is what makes the result reproducible: a fixed directory
gives a fixed answer on any machine.

**The claim runs one way only.** A font that is *here* and cannot draw the
text will not draw it anywhere — a `cmap` and a `GSUB` travel with the font —
which is what makes a finding worth reporting. A font that is here and draws
the text perfectly proves **nothing** about the reader's machine. Do not turn
silence from these two rules into "the Arabic will render".

They are two defects, not one, and the advice differs:

- `font-coverage` — the font has no glyph for some of the Arabic; it renders
as empty boxes. `evidence.offenders` is the exact list, by Unicode name.
An error when the font answers for *none* of the text, a warning when it
misses part of it.
- `shaping-broken` — the font has every letter and no shaping tables; the
Arabic renders as disconnected letters. Reported only on the aggregate —
no joins produced where at least four were required and the font had the
glyphs — because one letter drawn standalone is not a defect (invariant 7).

Neither is `fixable`. The repair for both is a different typeface, and which
one is an authoring decision; unlike `complex-font-missing`, which fills an
empty slot, these would be overwriting a font the author chose.

### Repairing

`repair <in> <out>` never modifies `<in>` and refuses `<out> == <in>` under
Expand Down Expand Up @@ -127,18 +162,25 @@ Read [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) first, then
[`docs/PLAN.md`](docs/PLAN.md) for the ordered work items.

**Adding a rule** — implement `Rule` in `crates/mirsam-core/src/rules/`,
register it in `Engine::with_options`. Nothing else changes. If its repair
register it in `Engine::build`. Nothing else changes. If its repair
needs a choice the text cannot make, the choice is a `RepairOptions` field,
not a CLI flag with logic behind it.
not a CLI flag with logic behind it. A rule that needs to know something about
the *machine* rather than the document takes a port instead, the way
`rules/font.rs` takes a `FontSource` — `RepairOptions` is for authoring
preferences, and putting a source there would make an audit's result depend on
which computer ran it without saying so anywhere.

**Adding a format** — new crate implementing `DocumentReader`; add
`DocumentWriter` only if the format can be faithfully edited in place.

**Resolving a font** — `mirsam-core` never opens one. `ports::FontSource`
takes a family name and answers with bytes, `mirsam-fonts` implements it over
the platform's font directories, and `shape` and `coverage` work on what comes
back. A source answering `None` means this machine has no such font, which is
a reportable state and not an error. Note which way the claim runs: a font
back; `Engine::with_fonts` is what arms the two rules over it, and
`Engine::with_options` leaves them registered and unrun. A source answering
`None` means this machine has no such font, which those rules treat as
silence: a fact about the computer, not a defect in the deck. Note which way
the claim runs: a font
that is *here* and cannot draw the text will not draw it anywhere, because a
`cmap` travels with the font — but a font that is here proves nothing about
the reader's machine, and no report may suggest otherwise.
Expand Down Expand Up @@ -169,7 +211,7 @@ A report for a format the tool reads but cannot write holds the refusal
`repair` gave and the exit code it used, in place of a repair report — so the
day a writer lands, it shows up as a diff on a real document.

**The shaping fixtures are fonts, and they are generated too.** The four
**The shaping fixtures are fonts, and they are generated too.** The five
under `crates/mirsam-core/tests/fonts/` are written byte by byte by
`scripts/make-shaping-fixture.py` — no `fontTools`, for the reason the
document generators avoid `python-pptx` — and regenerate with `make fonts`.
Expand All @@ -182,7 +224,11 @@ fixture: it claims the family `Mirsam Joining` a second time, sorts before
`joining.ttf` and shapes nothing, so a font source answering a family name
with whichever file it met first fails a test rather than a user's deck. That
is macOS's Arial in miniature — eleven files state that family, and only
`Arial.ttf` has any Arabic.
`Arial.ttf` has any Arabic. The fifth, `latin.ttf`, is the other defect
entirely: printable ASCII, no Arabic at all, no `GSUB`, which is Helvetica
under a complex-script slot. It is what `font-coverage` is proved against, and
shaping through it must stay silent — a letter a font has no glyph for says
nothing about whether the font can join.

**The Unicode name table is generated as well.** `charname.rs` comes from
`scripts/make-char-names.py`, which reads the UCD Python itself ships — no
Expand Down
36 changes: 36 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,42 @@ Working towards byte-preserving `repair` for PPTX. See

### Added

- **`font-coverage` and `shaping-broken`**, the two rules that turn 4.1's and
4.2's facts into findings. They are two defects, not one, and the advice
differs: a font with no glyph for the Arabic renders empty boxes, and a font
with every letter and no shaping tables renders a row of disconnected
letters. `font-coverage` names the exact characters that will not render —
`U+067E ARABIC LETTER PEH` — as an error when the font answers for none of
the text and a warning when it misses part of it. Neither is `fixable`: the
repair for both is a different typeface, and unlike `complex-font-missing`,
which fills an *empty* slot, these would be overwriting a font the author
chose.
- **`mirsam audit --fonts`**, and it is opt-in on purpose. These are the only
rules that ask about the machine rather than the document, and an audit that
resolved fonts without being asked would report differently on a laptop and
on a CI runner looking at one file. Every report now says which of the two
audits it is — `fonts NOT RUN` for a human, `"fonts": {"checked": false}` for
an agent — so their silence is never mistaken for a pass. `--font-dir <DIR>`
searches given directories instead of the platform's, which is what makes the
result reproducible. The claim runs one way only: a font that is *here* and
cannot draw the text will not draw it anywhere, and a font that is here and
draws it proves nothing about the reader's machine.
- The shaping threshold ADR 0008 left open is **four required joins**, counted
over letters the font actually has. A final form exists only because the
letter before it joined forwards, so every final has a companion initial or
medial — the dual-joining letters a font like Arial does shape. Two joins are
therefore one such letter, which is exactly the design choice the ADR forbids
concluding from, and is the whole of a two-letter word. Unmapped letters are
excluded so that a Latin-only font cannot produce a `shaping-broken` finding
beside its `font-coverage` one: the tool would be arguing with itself about
one paragraph.
- **`latin.ttf`**, a fifth generated fixture font: printable ASCII, no Arabic,
no `GSUB`. Helvetica under a complex-script slot, reduced to its principle,
so PLAN §4.3's acceptance is a committed test rather than a manual check.
Validated beyond the fixtures against macOS — `Helvetica` and `Comic Sans MS`
report every Arabic character of every paragraph, while `Times New Roman`,
`Geeza Pro`, `Mishafi`, `Al Nile`, `Baghdad` and `Arial` report nothing at
all.
- **Font coverage.** `mirsam-core::coverage` asks the question underneath
shaping — does the font have the letter at all? A font with no shaping
tables renders Arabic as disconnected letters; a font with no Arabic renders
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ prebuilt binaries for Linux, macOS and Windows, are milestone M7 — see
mirsam audit deck.pptx # inspect; exit 1 if blocking
mirsam audit deck.pptx --strict # warnings block too
mirsam audit deck.pptx --format json # for agents and CI
mirsam audit deck.pptx --fonts # also check the fonts installed here
mirsam repair deck.pptx fixed.pptx # write a repaired copy, then audit it
mirsam explain "<text>" # reproduce a defect without a document
mirsam rules # every check, with its id
Expand Down Expand Up @@ -118,6 +119,27 @@ written from `ar-SA`;
`--force` replaces an existing output. Repairing a repaired file is a no-op
that reproduces it byte for byte.

### Checking the fonts

`--fonts` adds two checks the rest of the tool cannot make from the file
alone. Each paragraph's complex-script typeface is resolved against the fonts
installed on this machine, and the Arabic is put through a real OpenType
shaper: `font-coverage` reports the exact characters the font has no glyph for
— they render as empty boxes — and `shaping-broken` reports a font that has
every letter and no shaping tables, which renders Arabic as a row of
disconnected letters. `--font-dir <DIR>` searches given directories instead of
the platform's, which is how you make the answer reproducible.

It is opt-in because it reads the machine rather than the document, and an
audit that resolved fonts unasked would report differently on two computers
looking at one file. Every report says which of the two audits it is —
`fonts NOT RUN`, or `"fonts": {"checked": false}` — so their silence is never
mistaken for a pass.

Read the result one way only. A font that is *here* and cannot draw the text
will not draw it anywhere; a font that is here and draws it perfectly proves
nothing about anyone else's machine.

### With an AI agent

`mirsam` is designed to be called by agents. `--format json` emits the full
Expand Down
1 change: 1 addition & 0 deletions crates/mirsam-cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ path = "src/main.rs"
[dependencies]
mirsam-core.workspace = true
mirsam-ooxml.workspace = true
mirsam-fonts.workspace = true
clap.workspace = true
serde_json.workspace = true
anyhow.workspace = true
Expand Down
77 changes: 73 additions & 4 deletions crates/mirsam-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,16 @@
mod render;

use anyhow::{Context, Result, bail};
use clap::{Parser, Subcommand, ValueEnum};
use clap::{Args, Parser, Subcommand, ValueEnum};
use mirsam_core::error::Error as CoreError;
use mirsam_core::rules::{DEFAULT_LOCALE, is_arabic_tag};
use mirsam_core::{DocumentReader, DocumentWriter, Engine, Repair, RepairOptions};
use mirsam_core::{DocumentReader, DocumentWriter, Engine, FontSource, Repair, RepairOptions};
use mirsam_fonts::SystemFonts;
use mirsam_ooxml::{DocxDocument, PptxDocument};
use std::fmt;
use std::path::{Path, PathBuf};
use std::process::ExitCode;
use std::sync::Arc;

/// Exit codes, stable across releases so CI can branch on them.
mod exit {
Expand Down Expand Up @@ -51,6 +53,8 @@ enum Command {
/// Treat warnings as blocking.
#[arg(long)]
strict: bool,
#[command(flatten)]
fonts: FontFlags,
#[arg(long, value_enum, default_value_t = Format::Text)]
format: Format,
},
Expand Down Expand Up @@ -89,6 +93,8 @@ enum Command {
/// Treat warnings remaining after the repair as blocking.
#[arg(long)]
strict: bool,
#[command(flatten)]
fonts: FontFlags,
#[arg(long, value_enum, default_value_t = Format::Text)]
format: Format,
},
Expand All @@ -113,6 +119,65 @@ enum Format {
Json,
}

/// Whether to shape the document's Arabic through the fonts this machine has.
///
/// Off by default, and that is a position rather than caution. Every other
/// check in this tool reads the document and would give the same answer on any
/// computer; `font-coverage` and `shaping-broken` read the *machine*, and an
/// audit whose result depended on which fonts happened to be installed on the
/// runner would be one nobody could reproduce. So the caller asks, and the
/// report says which of the two audits it is — a check that did not run is
/// `NOT RUN`, never an implied pass.
///
/// Note which way the claim runs when they are on. A font that is *here* and
/// cannot draw the text will not draw it anywhere, because a `cmap` and a
/// `GSUB` travel with the font. A font that is here and draws it perfectly
/// proves nothing about the reader's machine, and silence is not a promise
/// that the document will render.
#[derive(Args, Clone, Debug)]
struct FontFlags {
/// Resolve each paragraph's complex-script typeface against the fonts
/// installed here, and report Arabic the resolved font cannot draw or
/// cannot join. Machine-dependent: see --font-dir.
#[arg(long = "fonts")]
enabled: bool,
/// Search these directories instead of the platform's font directories.
/// Implies --fonts, and is what makes the check reproducible: a fixed
/// directory gives a fixed answer on any machine.
#[arg(long = "font-dir", value_name = "DIR")]
dirs: Vec<PathBuf>,
}

impl FontFlags {
fn requested(&self) -> bool {
self.enabled || !self.dirs.is_empty()
}

/// The source to arm the font rules with, or `None` to leave them unrun.
///
/// Built once per invocation and shared: `SystemFonts` indexes lazily and
/// keeps what it found, so a document of forty paragraphs walks the font
/// directories once rather than forty times.
fn source(&self) -> Option<Arc<dyn FontSource>> {
if !self.requested() {
return None;
}
Some(Arc::new(if self.dirs.is_empty() {
SystemFonts::new()
} else {
SystemFonts::in_dirs(self.dirs.clone())
}))
}
}

/// The engine, with the font rules armed only if the caller asked for them.
fn engine(options: &RepairOptions, fonts: &FontFlags) -> Engine {
match fonts.source() {
Some(source) => Engine::with_fonts(options, source),
None => Engine::with_options(options),
}
}

/// `--lang` must be a tag the `language-missing` rule accepts, or the repair
/// would write a value the re-audit immediately reports again.
fn arabic_tag(value: &str) -> std::result::Result<String, String> {
Expand Down Expand Up @@ -265,20 +330,22 @@ fn run() -> Result<u8> {
Command::Audit {
file,
strict,
fonts,
format,
} => {
let mut document = open(&file)?;
let units = document
.scan()
.with_context(|| format!("reading {}", file.display()))?;

let report = Engine::with_default_rules().audit(&units);
let report = engine(&RepairOptions::default(), &fonts).audit(&units);
let blocking = report.is_blocking(strict);
render::report(
&file,
document.format(),
&report,
strict,
fonts.requested(),
format == Format::Json,
);

Expand All @@ -294,6 +361,7 @@ fn run() -> Result<u8> {
align,
force,
strict,
fonts,
format,
} => {
// Refuse before reading anything. The writer refuses the first case
Expand All @@ -316,7 +384,7 @@ fn run() -> Result<u8> {
convert_bullets,
align,
};
let engine = Engine::with_options(&options);
let engine = engine(&options, &fonts);

let mut document = open_for_repair(&input)?;
let units = document
Expand Down Expand Up @@ -369,6 +437,7 @@ fn run() -> Result<u8> {
before: &before,
after: &after,
strict,
fonts_checked: fonts.requested(),
},
format == Format::Json,
);
Expand Down
Loading
Loading