GraDe - Grouped Adapter Decoding for Medical VQA under Domain Shift

Parameter-efficient framework for medical visual question answering that routes closed-ended and open-ended questions through two task-specific LoRA adapters on a shared Qwen2.5-VL-7B-Instruct backbone. Submitted to the MICCAI 2026 MedReason challenge.

Team IUCompPath - Sanket Kachole, Spyridon Bakas (Division of Computational Pathology, Indiana University School of Medicine). Contact: Spyridon Bakas (spbakas@iu.edu).

• Challenge: https://medreason26.github.io/
• Model weights (Zenodo): https://doi.org/10.5281/zenodo.22083522
• Full reproduction pipeline: docs/PIPELINE.md · Experiment log: EXPERIMENTS.md

Method in brief

The challenge asks two kinds of question about one medical image, and the system routes each to a different adapter on a shared frozen backbone. Closed-ended answers are decoded directly from the next-token logits of the option letters A-E, so no text is parsed and malformed outputs cannot affect exact-match scoring. Open-ended answers are generated with a short reasoning trace, using an adapter trained only on traces a grounding judge confirmed as visible in the image. Training and inference share one prompt module (src/infer/medreason_core.py), so the two cannot drift apart.

On the 1,600-case internal holdout, GraDe reached 96.4% MCQ accuracy, a 1.780/4 open-ended ground-truth score, and a 3.070/4 visual-answer score; the submitted configuration (G2, grounding-filtered traces) achieved the highest ground-truth score without changing closed-ended accuracy. The internal holdout does not contain the challenge's domain-shift conditions, so these numbers characterize in-distribution performance. On the official hidden pre-evaluation set, which does include the shift, GraDe scored 87.38% MCQ accuracy, 1.398/4 ground-truth, and 2.256/4 visual-answer.

1. Setup

pip install -r requirements.txt

Python 3.11. Training used NVIDIA A100-SXM4-40GB. Inference fits on one 40GB GPU. The 70B ground-truth judge needs roughly 140GB (four A100s or equivalent).

2. Usage

Path A - Inference from released weights (no training)

1. Download the backbone from Hugging Face:

hf download Qwen/Qwen2.5-VL-7B-Instruct --local-dir models/Qwen2.5-VL-7B-Instruct

2. Download the two trained adapters from Zenodo (DOI 10.5281/zenodo.22083522) and unzip them into models/, so the final layout is exactly:

models/
├── Qwen2.5-VL-7B-Instruct/   # backbone (from Hugging Face)
├── lora_mcq_v2/              # closed-ended adapter (from Zenodo)
└── lora_open_g2/             # open-ended adapter, recipe G2 (from Zenodo)

# from the repo root
mkdir -p models && cd models
wget https://zenodo.org/records/22083522/files/lora_mcq_v2.zip
wget https://zenodo.org/records/22083522/files/lora_open_g2.zip
unzip lora_mcq_v2.zip && unzip lora_open_g2.zip
rm lora_mcq_v2.zip lora_open_g2.zip
cd ..
# each zip already contains a top-level folder (lora_mcq_v2/ or lora_open_g2/),
# so the adapters land at models/lora_mcq_v2 and models/lora_open_g2

The folder names above are the paths the inference command expects. If you unzip them elsewhere, pass the correct paths to --mcq-adapter and --open-adapter below.

3. Run prediction on any input directory in the official schema:

python src/infer/predict.py \
  --model        models/Qwen2.5-VL-7B-Instruct \
  --mcq-adapter  models/lora_mcq_v2 \
  --open-adapter models/lora_open_g2 \
  --input-dir    /path/to/input \
  --out-dir      runs/eval/G2 \
  --name         "IUCompPath G2"

predict.py routes each case by its question-type flag, applies logit selection for closed-ended and grounded generation for open-ended, and writes results.json in the official schema plus raw.jsonl with every raw model output. The same medreason_core.py engine runs inside the submitted container (docker/).

Path B - Train from scratch

The full pipeline - dataset preparation, grounding-filtered trace construction, adapter training, judging, and scoring - is documented step by step in docs/PIPELINE.md. In short:

export DATA=/path/to/medreason_data     # obtained from the challenge organizers

# closed-ended adapter
python src/train/train_mcq.py  --model models/Qwen2.5-VL-7B-Instruct \
  --train-jsonl runs/sft/mcq_train.jsonl --dev-jsonl runs/sft/mcq_dev.jsonl \
  --out-dir models/lora_mcq_v2 --epochs 2 --batch-size 4 --grad-accum 4 --lr 1e-4

# open-ended adapter (recipe G2)
python src/train/train_open.py --model models/Qwen2.5-VL-7B-Instruct \
  --train-jsonl runs/sft_g2/open_sft_train.jsonl --dev-jsonl runs/sft_g2/open_sft_heldout.jsonl \
  --out-dir models/lora_open_g2 --epochs 3 --batch-size 2 --grad-accum 8 --lr 5e-5 \
  --lora-r 16 --lora-alpha 32

Slurm submitters for a cluster live in slurm/ (reference only); the commands they wrap also run on a single local GPU.

3. Repository layout

MedReason2026_IUCompPath/
├── src/
│   ├── data/     dataset exploration, splits, training-set construction (numbered: run in order)
│   ├── train/    LoRA training for each track
│   ├── infer/    the shared inference engine (medreason_core.py) and its variants
│   ├── eval/     judges, official scoring, error analysis
│   └── robust/   option-order robustness testing
├── configs/      one file per experiment in the paper (V1, G1-G6, M3, R1)
├── docker/       the submitted container (Dockerfile, custom_system.py, requirements)
├── slurm/        our cluster's job scripts - reference only
├── docs/
│   └── PIPELINE.md   full step-by-step reproduction
├── EXPERIMENTS.md    experiment log
└── ZENODO.md         trained-adapter download

Filenames describe what the code does; experiment identity lives in configs/. src/infer/medreason_core.py is imported by both the local runner and the container's system class, so what is tested locally is exactly what runs in the container.

4. Model weights

Trained adapters (lora_mcq_v2 + lora_open_g2) are on Zenodo: https://doi.org/10.5281/zenodo.22083522 (see also ZENODO.md for checksums). The backbone (Qwen2.5-VL-7B-Instruct) and the 70B ground-truth judge are downloaded from Hugging Face as shown above.

The MedReason 2026 data is released by the challenge organizers and is not redistributed here. Request access through the challenge.

Caveats

• The internal holdout is drawn from the released corpus and does not contain the challenge's domain-shift conditions, so development numbers characterize in-distribution performance.
• Open-ended configurations were each evaluated from a single training run; small differences between them should be read descriptively, not as statistically significant.
• Use the full-scale 70B ground-truth judge for model selection: an 8B proxy judge inflated GT by 0.413 on identical predictions, larger than the gaps between configurations.
• This is a challenge submission and is not intended for unsupervised clinical use.

Citation

Kachole, S., Bakas, S. GraDe: Grouped Adapter Decoding for Medical Visual Question-Answering under Domain Shift. MICCAI 2026 MedReason Challenge.

Acknowledgements

Container scaffolding is derived from the organizers' starter kit: https://github.com/medreason26/MedReason-Challenge-Docker