Multimodal image_column/text_column are never consumed: the documented {image, text} schema crashes with exit 2 after model load, and a renamed image_column silently trains text-o… (CR-A01-012) #399

Wave 1 · theme D1-5: Training, merging and export correctness · highest severity High · 2 finding(s) · effort small

Summary

An operator follows the template's multimodal example, or names the image column to match the dataset. The run either crashes after model load or produces a 'successful' VLM fine-tune that never saw an image, and the compliance artefacts describe it as multimodal training.

Primary finding

CR-A01-012 — Multimodal image_column/text_column are never consumed: the documented {image, text} schema crashes with exit 2 after model load, and a renamed image_column silently trains text-only with exit 0

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN CORE-08 — docs/roadmap/phase-16-trust-surface-hardening.md:587-588,604-605 (S10 open)

What happens. _passthrough_multimodal uses image_column only for an existence check and text_column only in a log line. The dataset reaches TRL unchanged. TRL keys VLM handling on the hard-coded column names 'image'/'images'. For vision datasets its signature columns are ['messages','prompt','completion','image','images'] (sft_trainer.py:1647-1648), so a 'text' column is removed. The schema advertised in config_template.yaml:144-147 and configuration.md:36-37 therefore fails at the first batch: 'KeyError: Unexpected input keys in examples: [image]', exit 2. A non-default image_column (e.g. 'img') passes ForgeLM's check, and ForgeLM logs 'Multimodal VLM dataset detected (image='img')'. TRL then sees no image column and trains text-only, with zero image loads and exit 0.

Where.

• forgelm/config.py:133-135
• forgelm/data.py:648-666
• forgelm/trainer.py:694
• config_template.yaml:138-143
• docs/reference/configuration.md:32-37
• forgelm/data.py:714-716
• forgelm/trainer.py:1014-1034
• forgelm/cli/_training.py:141-148
• docs/reference/data_ingestion_architecture.md:624-626

Evidence.

• cfgA.yaml ({image: url, text: ' caption'}): runA.log ends with KeyError "Unexpected input keys in examples: ['image']" and EXIT=2, after model load and trainer start. cfgC.yaml (image_column: img, rows {img: url, messages}): runC.log shows 'Multimodal VLM dataset detected (image='img', text='text')', train_loss 11.92, 'Completed Successfully', EXIT=0, and srv_hits.log has no new GET.

Recommended fix. Rename image_column/text_column to TRL's expected names (image/images, messages or text) in _passthrough_multimodal, or remove the fields. Reject datasets that TRL will treat as non-vision, and validate the schema before model load.

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

CR-A03-018 — Multimodal image_column/text_column are only validated or logged, never applied

severity Medium · reachability documented-opt-in · intent contract-violation · confidence confirmed · independently confirmed · not reproduced (code trace) · already tracked: KNOWN-OPEN CORE-08 — docs/roadmap/phase-16-trust-surface-hardening.md:587-588,604-605 (S10 open)

What happens. _passthrough_multimodal checks image_column and logs text_column, then returns the splits unchanged (data.py:648-668). The trainer hard-codes dataset_text_field='text' (trainer.py:694), and TRL detects vision data only by literal 'image'/'images' columns, so non-default column names do not work.

Failure scenario. A dataset with columns img/caption and multimodal.image_column: img, text_column: caption reaches TRL with columns img and caption. The text field 'text' is missing and 'image' is absent, so training fails or runs text-only.

Where.

• forgelm/data.py:648-666 (_passthrough_multimodal; the helper ends at line 666, not 668)
• forgelm/trainer.py:694 (dataset_text_field hard-coded to 'text')
• forgelm/config.py:128-135 (MultimodalConfig; image_column :134, text_column :135)
• forgelm/data.py:648-668 (confirmed)
• docs/reference/configuration.md:31-37 and docs/reference/configuration-tr.md:31-37
• forgelm/data.py:714-716
• forgelm/trainer.py:1013-1034
• forgelm/config.py:283-285 (ModelConfig.multimodal)
• config_template.yaml:142-147
• docs/reference/data_ingestion_architecture.md:624-626

Evidence.

• rg finds no use of text_column/image_column outside config.py and data.py, and none in trainer.py. Installed TRL SFTTrainer: '_is_vision_dataset = "image" in dataset_sample or "images" in dataset_sample' and 'texts = [example[self.dataset_text_field] ...]'. No VLM end-to-end run was performed.
• prepare_dataset returns columns ['img','caption'] unchanged (t7.py). The VLM training path was not run (no fixture).

Recommended fix. Rename image_column to 'image' and text_column to 'text' (or set dataset_text_field) before handing off. Load image paths/URLs into PIL or document that they must be pre-decoded. Add a VLM smoke test.

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}