[AmitMY]

An audit of the documentation against the code after #73–#93, so the repository reads as one: every README describes what the code does now, and every number comes from a run of the current code.

Docs

• README: words come from words-segmentation or any HF tokenizer; images at their native size, with row/column embeddings; the decoder's input is the latent in place of BOS; the Baseline section links the language-modeling experiments. assets/architecture.png regenerated from the paper's figure.
• welt_training/README.md: data.dataset_tokenizer, overlapping chunks of long documents, truncation with other pretokenizers; a leftover HF Trainer line removed.
• welt_training/experiments/pretrain/README.md: new.
• benchmarks/README.md: modality dropout shortened, a stale cache paragraph removed, packed patches moved into "What made it faster", steps overrides, flops.py usage.
• Language modeling README: the earlier opus-100/wikitext step times predate the baseline's padded vocabulary and fused cross entropy.

Benchmarks rerun on the current code

• All eight task rows of benchmarks/README.md / tasks.csv (string repetition, OCR, letter count, English to Hebrew, SignWriting to text). Quality is the same or better (e.g. translation 0.531 bits per byte, chrF 47.6, was 0.547 / 45.4).
• Correction from the rerun: images barely help SignWriting (best checkpoints: both encoders 1.174 vs. bytes only 1.178; the README had 1.212 vs. 1.239).
• welt_training/evaluate.py: warm up with full-length generations. The 2-word warmup left vLLM's decode kernels (Triton) to compile during the measured generation, which halved the words per second on a cold cache (string repetition 209 → 548 words/s, translation 459 → 910).

🤖 Generated with Claude Code

https://claude.ai/code/session_015Xm62qoeaTunFSU5GYCQ1f

---

Note

Low Risk
Mostly documentation and benchmark CSV updates; the only code change is evaluation warmup timing, which affects reported throughput metrics not model behavior.

Overview
Aligns documentation with the current implementation and refreshes all benchmark numbers from reruns on today's code.

Docs update the main README and training README for words-segmentation/HF pretokenizers, native-size packed image patches, latent-as-BOS bytes decoding, dataset_tokenizer for token-id datasets, one-word-overlap document chunking, and a new pretrain experiment README. Benchmarks docs add packed patches to the speedups list, document flops.py usage, trim SignWriting/modality-dropout narrative (image encoder barely helps vs prior claims), and remove a stale dataset-cache note.

Benchmarks: tasks.csv and the tasks table get new ms/step, loss, and generation metrics (often faster steps and similar-or-better quality). welt-evaluate warmup now uses the full max_generated_words instead of 2-word warmup so Triton decode compile doesn't inflate measured words/s (e.g. string-repetition ~209→548, MT ~459→910).

^{Reviewed by Cursor Bugbot for commit 4afd2a8. Bugbot is set up for automated code reviews on this repo. Configure here.}