Get your own MRI out of a web viewer at full fidelity, and sanity-check what an AI says about it.
Two small, honest tools for anyone who has downloaded their own medical imaging and/or is tempted to paste it into an AI:
dicomweb_verify: prove you actually pulled every series and slice at original resolution from a DICOMweb server (OHIF / Weasis / most cloud PACS), byte-for-byte, with no hidden objects.airead_qc: sanity-check an AI's read of your scan: verify its orientation claims against the DICOM geometry, see where the model actually looked, and measure how much its answer swings on trivial prompt rewording.
⚠️ This is verification and education tooling. It is NOT a diagnostic tool, and it does not read or diagnose your scan. See DISCLAIMER.md. Nothing here replaces a licensed radiologist.
I fed my own knee MRI (a right knee, from a common 1.5T clinical scanner) to several AI agents with a careful, unbiased prompt. One of them returned a confident, beautifully formatted, richly detailed radiology read: it cited DICOM tags, quoted pixel intensities, ruled out anatomical mimics one by one, and localized a tear to specific slices with "HIGH confidence."
It had left and right reversed.
It read the DICOM ImagePositionPatient coordinates correctly, then applied left-knee laterality to a
right knee, silently flipping medial and lateral. Its confident "medial meniscus tear" was sitting in
the lateral compartment, right where a normal structure (the popliteus hiatus) mimics a tear. The
error was invisible unless you checked the read's spatial claims against the scanner geometry.
Full write-up: docs/orientation_flip_case_study.md.
The lesson isn't "AI is useless." It's that a confident, detailed, well-formatted AI read can be catastrophically wrong in a way that's invisible without independent verification, and almost nobody verifies. These tools make verification a two-minute command.
pip install -r requirements.txt# 0) make a small synthetic DICOM series to play with (NOT a real patient)
python sample_data/make_synthetic_dicom.py --out demo_series
# 1) derive the true orientation from the DICOM, and check an AI's claim against it
python -m airead_qc.orientation demo_series --laterality R \
--check-claim "slices 1-5 are lateral" # -> flags the mismatch
# 2) verify a local download is complete + full-fidelity vs a DICOMweb server
python -m dicomweb_verify.verify \
--server https://YOUR-DICOMWEB-HOST/dicomweb \
--study 1.2.3.4...your.study.uid \
--local /path/to/your/downloaded/dicomFor the model-based checks (saliency, prompt-sensitivity) and the "watch it fail" demo, see
colab/medvlm_reliability_demo.ipynb. Free GPU, your images
never leave the session, and the point is to show you why the reads are shaky.
| Path | What it does |
|---|---|
dicomweb_verify/ |
Completeness + fidelity verifier against a DICOMweb server |
airead_qc/orientation.py |
Derive medial/lateral/ant/post from DICOM; check an AI read's orientation claims |
airead_qc/saliency.py |
Occlusion saliency: where a medical VLM actually looked for a yes/no finding |
airead_qc/prompt_sensitivity.py |
Sweep N prompts × M slices; quantify how much the answer swings |
airead_qc/prompts/unbiased_read_prompt.txt |
A rigorous, anti-bias, anti-fabrication read prompt template |
colab/medvlm_reliability_demo.ipynb |
Run a medical VLM on de-identified slices + apply the QC above |
sample_data/make_synthetic_dicom.py |
Generate a synthetic DICOM series so everything runs out of the box |
docs/orientation_flip_case_study.md |
The full flip story, reproducible |
- The QC tools tell you when an AI read is internally inconsistent or unstable. They cannot tell you when it is right. Only a qualified human can.
dicomweb_verifychecks completeness and bit-level fidelity; it does not assess image quality or diagnostic adequacy.- Everything assumes your own data, your own access. Do not point these at systems you aren't authorized to use.
MIT, see LICENSE.