perf(translate): remember BabelDOC font hash passes

docs/architecture/translation-pipeline.md

@@ -99,17 +99,55 @@ it into `babeldoc_chunk_error`.
 
 **What this does not fix, and why chunk size is a live question.** The layout model was never the largest per-chunk fixed cost.
 `FontMapper.__init__` sha3_256-verifies and loads **all 34 embedding fonts
-(~250 MB)** for `lang_out`, and nothing memoizes it — not `fontmap.py`, not
-`assets.py`. BabelDOC constructs one per stage: `ILCreater`, `ParagraphFinder`,
-`StylesAndFormulas`, `ILTranslator`, `Typesetting`, `PDFCreater`. Measured
-against the installed babeldoc on a warm cache: **1.02 s per `FontMapper`, ≈6 s
-per chunk**, none of which this cache touches. So "the reload is fixed" is not
-on its own a reason to shrink a chunk — shrinking one multiplies ~6 s by the
-extra chunks it creates, and on the Argos path (`_MAX_PARALLEL_CHUNKS_ARGOS = 2`)
-that can outweigh what the cache saves. `_PAGES_PER_CHUNK_ARGOS` is 1 today,
-like the LLM path; measure an end-to-end Argos run before trusting that, or
-before changing either constant. What a larger chunk buys is amortized
-per-chunk fixed overhead in general, not the ONNX reload specifically.
+(266 MB)** for `lang_out`, and BabelDOC memoizes neither — not `fontmap.py`, not
+`assets.py`. It constructs one per stage: `ILCreater`, `ParagraphFinder`,
+`StylesAndFormulas`, `ILTranslator` (a second inside `ILTranslatorLLMOnly`),
+`Typesetting`, `PDFCreater` (a second for scanned-page detection) — 7 per page
+on the Argos path, 8 on the LLM path. The hash is now remembered (next section);
+what is left is ~0.26 s per `FontMapper` of `pymupdf.Font` parsing, ≈2 s per
+chunk, which this cache does not touch either. So "the reload is fixed" is not
+on its own a reason to shrink a chunk — shrinking one multiplies that fixed cost
+by the extra chunks it creates, and on the Argos path
+(`_MAX_PARALLEL_CHUNKS_ARGOS = 2`) that can outweigh what the cache saves.
+`_PAGES_PER_CHUNK_ARGOS` is 1 today, like the LLM path; measure an end-to-end
+Argos run before trusting that, or before changing either constant. What a
+larger chunk buys is amortized per-chunk fixed overhead in general, not the ONNX
+reload specifically.
+
+## A font that passed its hash check is not hashed again
+
+A `FontMapper` took ~1.3 s, of which ~1.0 s was `assets.verify_file` hashing
+the 34 fonts (measured alone: 0.99 s; parsing them alone: 0.13 s; the ~0.26 s a
+mapper still costs with the hash remembered is that parse plus the rest of the
+constructor, under the contention of a real run). Traced over real one-page
+LLM-path runs on two different documents: **272 checks per page — 34 fonts × 8
+mappers (238 on Argos, with 7) — against the same 34 (file, expected hash)
+pairs**, none derived from the document; every pair passed,
+and every file was byte-identical before and after both runs. Only a process's
+first check can find anything (a missing or corrupt font, which BabelDOC then
+re-downloads); the rest re-hash unchanged bytes.
+
+`processors/font_verify_cache.py` replaces the module global
+`babeldoc.assets.assets.verify_file` with a memo — BabelDOC's asset functions
+call it by that global name, so that one attribute reaches every stage; re-check
+it when bumping babeldoc. Measured on a one-page run: **15.5 s → 7.9 s**, the
+`FontMapper` share 60% → 23%.
+
+- **Only a pass is remembered**, keyed on (path, expected hash) and tied to the
+  file's size and mtime as they stood both before and after the hash. A failure,
+  a font rewritten later (a re-download, the setup flow's offline restore) and a
+  font that changed mid-hash are all hashed again. Nothing persists across
+  processes. There is no single-flight: chunks that start together before the
+  warm thread has finished each hash the fonts once — extra work, never a wrong
+  answer.
+- **Installed in two places**, both idempotent: the `engine-warm` thread, and
+  `routes/translation.py:_load_engine` for a Translate click that beats it.
+- **The warm thread pre-checks the fonts on disk** (`preverify_cached_fonts`,
+  ~1 s), so the first page doesn't pay that round either. It calls `verify_file`
+  on existing files only, never `get_font_and_metadata`, which downloads — the
+  same rule that gates the layout-model load. It needs no `babeldoc_core_ready()`
+  gate for that reason, and has its own `try`, after the model, so a failure
+  never costs the larger warm-up.
 
 **Rolling-PDF pruning keeps two files, not one.** `cleanup_partial_artifacts`
 runs after every chunk now, not only on cancel, so a long run no longer piles

src/desktop_pdf_translator/api/routes/translation.py

@@ -81,9 +81,13 @@ def _load_engine() -> tuple[type, type]:
     beats that thread — or a warm-up that failed, which it does silently —
     lands here instead. Callers must run it off the event loop.
     """
+    from ...processors import font_verify_cache
     from ...processors.events import EventType
     from ...processors.processor import PDFProcessor
 
+    # Normally done by the warm thread too; a click that beats it would
+    # otherwise re-hash every font in every stage of the first job.
+    font_verify_cache.install()
     return EventType, PDFProcessor
 
 

src/desktop_pdf_translator/api/server.py

@@ -126,7 +126,8 @@ def _warm_translation_engine() -> None:
     only ever warms what is already on disk.
 
     Importing BabelDOC is still unconditional — it touches no assets, and it is
-    the larger half of the cold-start cost.
+    the larger half of the cold-start cost. So is the font pre-check that
+    follows (`_preverify_babeldoc_fonts`), which only reads.
 
     A Translate click can still beat this thread, so the job path imports
     BabelDOC in a thread of its own rather than assuming this one won the race
@@ -149,9 +150,27 @@ def _warm_translation_engine() -> None:
     except Exception as exc:  # noqa: BLE001
         logger.warning("Translation engine warm-up failed (non-fatal): %s", exc)
         return
+    _preverify_babeldoc_fonts()
     logger.info("Translation engine warm in %.1fs", time.perf_counter() - started)
 
 
+def _preverify_babeldoc_fonts() -> None:
+    """Hash BabelDOC's cached fonts once now, so the first page doesn't.
+
+    Every stage of every page re-checks them; `font_verify_cache` remembers the
+    passes, and this pays for the first round (~1 s) before the user clicks.
+    Ungated by `babeldoc_core_ready()`, unlike the layout model: it only reads
+    fonts already on disk and never downloads. Its own try, after the model,
+    so a failure here never costs the larger warm-up.
+    """
+    try:
+        from ..processors import font_verify_cache
+
+        font_verify_cache.preverify_cached_fonts()
+    except Exception as exc:  # noqa: BLE001
+        logger.warning("BabelDOC font pre-check failed (non-fatal): %s", exc)
+
+
 def _sweep_orphan_translate_dirs(max_age_seconds: int = 3600) -> int:
     """Remove `pdfusion-translate-*` dirs left behind by a prior sidecar that
     crashed or was killed before its next-job cleanup could fire.

src/desktop_pdf_translator/processors/font_verify_cache.py

@@ -0,0 +1,123 @@
+"""Remembers which of BabelDOC's cached assets have passed their hash check.
+
+BabelDOC builds a `FontMapper` in every pipeline stage — `ILCreater`,
+`ParagraphFinder`, `StylesAndFormulas`, `ILTranslator` (twice on the LLM path),
+`Typesetting`, `PDFCreater` (twice when scanned-page detection runs) — and each
+one runs `assets.verify_file` over every font of the target language: 34 files,
+266 MB of SHA3-256 for Vietnamese, ~1 s. Traced over two different documents on
+the LLM path, that is 272 checks per page (238 on Argos, which skips one
+mapper) against the same 34 (file, expected hash) pairs, none of them derived
+from the document, every file byte-identical before and after. Measured on a
+one-page run: 15.5 s → 7.9 s with the passes remembered.
+
+Only the first check in a process can find something new — a missing or corrupt
+font, which BabelDOC then re-downloads — so a pass is remembered and a failure
+never is. A pass is tied to the file's size and mtime as they were on both
+sides of the hash, so a font rewritten afterwards (a re-download, the setup
+flow restoring the offline zip) or while it was being hashed is checked again.
+
+Kept free of babeldoc imports at module scope, like `doc_layout_cache.py`: the
+boot warm thread imports this, and so does the suite.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import threading
+import time
+from pathlib import Path
+from typing import Callable, Optional
+
+logger = logging.getLogger(__name__)
+
+Verify = Callable[[Path, str], bool]
+
+# Marks a function this module produced, so `install()` can tell a memo it
+# already put in place from BabelDOC's own function — or from a test's fake,
+# which `monkeypatch` restores, taking the mark with it.
+_MEMO_MARK = "_pdfusion_memoized"
+
+_install_lock = threading.Lock()
+
+
+def _signature(path: Path) -> Optional[tuple[int, int]]:
+    try:
+        st = os.stat(path)
+    except OSError:
+        return None
+    return st.st_size, st.st_mtime_ns
+
+
+def memoize_verify(verify: Verify) -> Verify:
+    """Wrap `verify` so a (file, expected hash) that already passed, and has not
+    changed since, is answered without hashing it again."""
+    passed: dict[tuple[str, str], tuple[int, int]] = {}
+    lock = threading.Lock()
+
+    def memoized(path: Path, sha3_256: str) -> bool:
+        key = (os.fspath(path), sha3_256)
+        before = _signature(path)
+        if before is not None:
+            with lock:
+                if passed.get(key) == before:
+                    return True
+        ok = verify(path, sha3_256)
+        # Stat again after hashing: a pass on bytes that changed underneath it
+        # vouches for a file nobody verified.
+        with lock:
+            if ok and before is not None and _signature(path) == before:
+                passed[key] = before
+            else:
+                passed.pop(key, None)
+        return ok
+
+    setattr(memoized, _MEMO_MARK, True)
+    memoized.__wrapped__ = verify  # type: ignore[attr-defined]
+    return memoized
+
+
+def install() -> None:
+    """Put the memo in front of `babeldoc.assets.assets.verify_file`.
+
+    BabelDOC's asset functions call `verify_file` as a module global, so
+    replacing the attribute is enough to reach every stage's font check.
+    Re-check that against `assets.py` when bumping babeldoc. Blocking (imports
+    babeldoc's asset module); call off the event loop.
+    """
+    from babeldoc.assets import assets
+
+    with _install_lock:
+        if getattr(assets.verify_file, _MEMO_MARK, False):
+            return
+        assets.verify_file = memoize_verify(assets.verify_file)
+
+
+def preverify_cached_fonts() -> int:
+    """Check every embedding font already on disk, so the first page doesn't.
+
+    Only fonts that exist are checked, and through `verify_file` rather than
+    `get_font_and_metadata`, which downloads what is missing: this runs on the
+    boot warm thread, with no UI to report a download, and BabelDOC writes
+    downloads in place — racing the setup flow's copy can corrupt it. Returns
+    how many fonts passed. Blocking; call off the event loop.
+    """
+    install()
+    from babeldoc.assets import assets
+    from babeldoc.assets.embedding_assets_metadata import EMBEDDING_FONT_METADATA
+    from babeldoc.const import get_cache_file_path
+
+    started = time.perf_counter()
+    checked = passed = 0
+    for name, meta in EMBEDDING_FONT_METADATA.items():
+        path = get_cache_file_path(name, "fonts")
+        if not path.exists():
+            continue
+        checked += 1
+        if assets.verify_file(path, meta["sha3_256"]):
+            passed += 1
+    logger.info(
+        "BabelDOC fonts pre-verified: %d of %d on disk passed in %.1fs",
+        passed, checked, time.perf_counter() - started,
+    )
+    return passed

tests/test_engine_warm_gate.py

@@ -21,7 +21,7 @@
 
 from desktop_pdf_translator import engine_assets
 from desktop_pdf_translator.api import server
-from desktop_pdf_translator.processors import doc_layout_cache
+from desktop_pdf_translator.processors import doc_layout_cache, font_verify_cache
 import desktop_pdf_translator.processors as processors_pkg
 
 
@@ -32,10 +32,16 @@ def loads(monkeypatch: pytest.MonkeyPatch) -> list[int]:
     `from ..processors import processor` resolves through `getattr` on the
     package before it reaches the import machinery, so setting the attribute is
     enough to keep babeldoc + torch out of this test.
+
+    The font pre-check is stubbed too: run for real it imports babeldoc, hashes
+    the developer's whole font cache and leaves `verify_file` patched for the
+    rest of the session. The tests that are about it re-stub it with recorders.
     """
     monkeypatch.setattr(
         processors_pkg, "processor", types.ModuleType("processor"), raising=False
     )
+    monkeypatch.setattr(font_verify_cache, "install", lambda: None)
+    monkeypatch.setattr(font_verify_cache, "preverify_cached_fonts", lambda: 0)
     calls: list[int] = []
     monkeypatch.setattr(
         doc_layout_cache,
@@ -64,6 +70,40 @@ def test_skips_the_model_when_the_asset_is_missing(monkeypatch, loads):
     assert loads == []
 
 
+# ---------------------------------------------------------------------------
+# Font hash memo (BabelDOC re-hashes every cached font per pipeline stage)
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.parametrize("core_ready", [True, False])
+def test_warm_up_preverifies_the_fonts(monkeypatch, loads, core_ready):
+    """The pre-check only reads fonts already on disk, so it needs no
+    `babeldoc_core_ready()` gate. (It installs the memo itself — pinned by
+    `test_preverified_fonts_are_not_hashed_again_by_babeldoc`.)"""
+    monkeypatch.setattr(engine_assets, "babeldoc_core_ready", lambda: core_ready)
+    calls: list[str] = []
+    monkeypatch.setattr(
+        font_verify_cache, "preverify_cached_fonts", lambda: calls.append("preverify") or 0
+    )
+
+    server._warm_translation_engine()
+
+    assert calls == ["preverify"]
+
+
+def test_a_failing_font_precheck_does_not_cost_the_layout_model_warm_up(monkeypatch, loads):
+    monkeypatch.setattr(engine_assets, "babeldoc_core_ready", lambda: True)
+
+    def boom():
+        raise RuntimeError("font pre-check failed")
+
+    monkeypatch.setattr(font_verify_cache, "preverify_cached_fonts", boom)
+
+    server._warm_translation_engine()
+
+    assert loads == [1]
+
+
 # ---------------------------------------------------------------------------
 # Argos is warmed when it is what would run
 # ---------------------------------------------------------------------------