OrthoScribe - Training-Free Multimodal Orthodontic Report Generation via Photographic Retrieval and 3D Geometry Correction

Training-free pipeline for orthodontic report generation from intraoral data. Query photographs are embedded with a self-supervised image model to retrieve a similar training report; the top candidates are re-ranked by occlusal geometry measured from the 3D meshes; a precision filter removes unverifiable patient-specific claims; and overjet and overbite descriptors are overwritten with values measured directly from the meshes. No task-specific training is required. Submitted to the MICCAI 2026 ODIN Bite2Text challenge (Task 2).

Team IUCompPath - Sanket Kachole, Spyridon Bakas (Division of Computational Pathology, Indiana University School of Medicine). Contact: Spyridon Bakas (spbakas@iu.edu).

• Challenge: https://odin2026.grand-challenge.org/
• Assets (Zenodo): https://doi.org/10.5281/zenodo.22103859
• Dataset (ODIN 2026 Bite2Text): released by the challenge organizers; not redistributed here.

Method in brief

OrthoScribe separates the two modalities: photographs drive retrieval of a similar training report, while the 3D meshes measure overjet and overbite to re-rank candidates and correct occlusal descriptors. A precision filter removes unverifiable tooth-level claims, and the measured overjet/overbite overwrite the draft's descriptors. No task-specific training is required. On internal validation it reached RadFact F1 0.363 (vs. 0.326 constant-report, 0.274 fine-tuned VLM); on the held-out Test Phase, BLEU-4 0.173 and METEOR 0.388.

1. Setup

pip install -r requirements.txt

Python 3.11. The system is packaged as a Docker container and runs offline (no network access at inference). Inference fits on a single 16 GB GPU (NVIDIA T4) or a 24 GB GPU (A10G); no vision–language model is loaded at inference. Median runtime is well under the 5 min/case challenge budget.

2. Usage - Inference from released assets

The container bakes in the report bank, image embeddings, geometry statistics, and the DINOv2 encoder at build time, then reads each case from /input and writes a report to /output. There is no training step in this repository; the retrieval bank and embeddings are provided as released assets, and the procedure that produced them is described in the paper.

1. Download the asset bundle from Zenodo (DOI 10.5281/zenodo.22103859) and unzip it into the repository root, so that the following files sit next to the Dockerfile:

MICCAI_Challenge_Bite2Text/
├── Dockerfile
├── inference.py
├── requirements.txt
├── baselines.json                 # medoid fallback report (from Zenodo)
├── report_bank_full.json          # 781 training reports (from Zenodo)
├── train_img_embeds_full.npz      # DINOv2 image embeddings, 781 cases (from Zenodo)
├── train_geom_full.json           # per-case overjet/overbite, training set (from Zenodo)
├── geom_stats.json                # z-score constants for geometry re-ranking (from Zenodo)
└── dinov2/                         # self-supervised image encoder (from Zenodo)

# from the repo root
wget https://zenodo.org/records/22103859/files/orthoscribe_assets.zip
unzip orthoscribe_assets.zip
rm orthoscribe_assets.zip
# the zip unpacks the five asset files plus the dinov2/ folder into the repo root

2. Build the container:

docker build --platform=linux/amd64 -t orthoscribe .

3. Arrange a case in the official input schema and run. The container expects the registered meshes and the intraoral photographs under /input:

/input/
├── files/
│   ├── ios-upper/       # upper-arch mesh (STL or OBJ)
│   └── ios-lower/       # lower-arch mesh (STL or OBJ)
└── images/
    └── 2d-intraoral-photographs/   # intraoral photos (multi-frame TIFF or JPGs)

docker run --rm --gpus all \
  -v /path/to/case/input:/input:ro \
  -v /path/to/case/output:/output \
  orthoscribe

The container writes /output/diagnostic-imaging-report.json in the form {"report": "<text>"}. If the photographs are missing it falls back to a geometry-corrected medoid; if any stage fails it falls back to the constant medoid, so an output is always written.

3. Scoring

eval/score_preds.py computes RadFact logical F1 against reference reports using the challenge judge. It expects RADFACT_BACKEND and RADFACT_MODEL in the environment and an OpenAI key:

export RADFACT_BACKEND=openai
export RADFACT_MODEL=gpt-4.1-mini
export OPENAI_API_KEY=<your key>
python eval/score_preds.py <predictions.json>

On the 88-case validation split this reproduces the reported RadFact F1 of 0.363. BLEU-4 and METEOR for the held-out Test Phase (0.173 / 0.388) are produced by the official leaderboard and are not recomputed here.

4. Repository layout

MICCAI_Challenge_Bite2Text/
├── inference.py          the full pipeline: retrieval, re-ranking, precision filter, geometry correction
├── Dockerfile            the submitted container (offline, T4/A10G)
├── requirements.txt      container dependencies
├── .dockerignore
├── geom/
│   └── extract_geom.py   standalone geometry module (canonical frame, incisal-edge detection, overjet/overbite)
└── eval/
    └── score_preds.py    RadFact scoring against reference reports

The geometry logic in geom/extract_geom.py is the readable reference implementation of the method described in the paper; the container in inference.py carries a self-contained, file-based copy of the same computation.

5. Assets

The report bank, image embeddings, geometry statistics, medoid fallback, and the DINOv2 encoder are hosted on Zenodo: https://doi.org/10.5281/zenodo.22103859 (checksums in the record). The ODIN 2026 Bite2Text dataset is released by the challenge organizers and is not redistributed here; request access through the challenge.

Caveats

• Retrieval quality bounds the final report: descriptors other than overjet and overbite are inherited from the retrieved draft, so findings absent from the training report bank cannot be generated.
• Only overjet and overbite are corrected from geometry; among the descriptors examined, these were the only ones that exceeded their majority-class baselines (0.756 vs. 0.372 and 0.480 vs. 0.320).
• Test Phase factuality could not be assessed: the official leaderboard returned only lexical-overlap metrics.
• This is a challenge submission and is not intended for unsupervised clinical use.

Citation

Kachole, S., Bakas, S. OrthoScribe: Training-Free Multimodal Orthodontic Report Generation via Photographic Retrieval and 3D Geometry Correction. MICCAI 2026 ODIN Bite2Text Challenge.

Acknowledgements

Container scaffolding is derived from the ODIN 2026 Bite2Text starter kit provided by the challenge organizers.