[TranBaDat2607]

Fixes the out-of-memory kill when translating a long document. It hits Argos and LLM translation alike.

Cause

BabelDOC 0.5.24 builds a FontMapper in every pipeline stage: 6 per page on Argos, 7 on the LLM path, counted on real runs. Each one loads all 34 embedding fonts (266 MB) into pymupdf.Font objects.

Its constructor wraps bound methods in lru_cache, so every mapper and font sits in a reference cycle that only a full GC frees. Nothing calls gc.collect(), and full collections follow object counts, not bytes.

PDFusion runs one BabelDOC job per page, so on a 346-page book the sidecar reached ~12 GB and the kernel's OOM killer killed it 238 pages in.

Change

• processors/font_cache.py. install() replaces FontMapper.__init__ with a copy of BabelDOC's whose font loop takes each font from a process-wide cache, decorated once.
  • It only patches a babeldoc in VERIFIED_BABELDOC. Any other version logs a warning and keeps its own constructor.
  • routes/translation.py:_load_engine installs it.
• Comment cleanup (first commit). Deletes comments the code contradicts, among them the "~9 GB per sub-job" figure: BabelDOC's "Peak memory usage" line measures the whole process. It also removes the dead _MAX_PARALLEL_CHUNKS constant. Each claim is listed in the commit message.
• Docs. New section "Each font is loaded once per process" in docs/architecture/translation-pipeline.md, plus a CLAUDE.md invariant.

Tests

tests/test_font_cache.py has 17 tests, written test-first by a separate agent that never saw the implementation, and reviewed independently. They cover:

• sharing across mappers, stages and languages;
• one load per file;
• per-instance state;
• equivalence with BabelDOC's own constructor, per mapper and per font attribute;
• memos wrapped once;
• concurrent construction;
• failed loads not cached;
• fonts outliving their mappers;
• the version guard;
• the job-path wiring.

Mutation-checked against the real module: a lock-free cache and a load-then-setdefault cache each fail exactly the test meant for them. Full suite: 1425 passed, 8 skipped; ruff clean.

Measured

All on Linux, Argos and an echo stub, with no LLM calls.

Run	Before	After
Argos, 4 pages	0.26 → 6.5 GB, aborted	~1.7 GB after 8 pages
LLM path (echo stub), 6 pages in flight	>6.5 GB before the first page finished	—
Whole book, 301 text pages	OOM-killed at ~12 GB, page 238	23 min, peak 2.38 GB

• Output. Page 25 is identical before and after on both paths: every word, its position and its fonts. No PIPELINE_VERSION bump needed.
• Residual drift. Memory still rises ~1.7 MB per page over the book, most likely from the rolling-PDF rebuild and per-page source-font garbage. A thousands-page book will peak above 2.4 GB; not yet measured.

Not exercised

• the frozen PyInstaller build;
• Windows;
• a run through pnpm tauri dev.

Open questions

• babeldoc is pinned to a range (>=0.5.20,<0.6.0). The first new 0.5.x release turns CI red via test_the_installed_babeldoc_is_one_the_copy_was_checked_against, and a fresh build would ship unpatched. Pin babeldoc==0.5.24, or keep the tripwire?
• Expect merge conflicts with the unmerged fix/skip-textless-pages and fix/release-freed-memory branches, in processor.py, CLAUDE.md and the pipeline doc. On main, a blank page still fails the job ("contains no paragraphs"); the first of those branches fixes that.

🤖 Generated with Claude Code