No test resolves a DeepSpeed preset from an installed wheel; the only resolution test passes because it runs from the source tree (CR-A02-024) #111

Wave 0 · theme D0-8: Packaging and delivery promises · highest severity High · 1 finding(s) · effort small

Summary

On every pip/Docker install the documented preset names fail at training time with exit 1, while CI (editable/source-tree) is green.

Primary finding

CR-A02-024 — No test resolves a DeepSpeed preset from an installed wheel; the only resolution test passes because it runs from the source tree

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN OPS-01 — docs/roadmap/phase-16-trust-surface-hardening.md:642-650 (S12 open)

What happens. tests/test_distributed.py:141-218 reads configs/deepspeed/.json relative to the tests dir and tests/test_distributed.py:210-216 (test_valid_preset_still_resolves) calls _resolve_deepspeed_config('zero2') from the repository, where trainer.py:645-648 finds configs/ beside the forgelm package. tests/test_packaging.py:86-119 asserts package_data globs only for forgelm.templates; nothing checks that configs/deepspeed is shipped (pyproject.toml:250-263 packages only forgelm, with package-data for templates only). Dynamic check in the installed lane (bin/fpy -C (local review artefact) cwd without configs/): _resolve_deepspeed_config('zero2'), 'zero3' and 'zero3_offload' each raise ConfigError 'DeepSpeed preset ... not found at .../installed/configs/deepspeed/.json'.

Where.

• tests/test_distributed.py:144-216
• tests/test_packaging.py:86-119
• forgelm/trainer.py:632-658
• pyproject.toml:250-278
• .dockerignore:46
• .github/workflows/ci.yml:208-220
• docs/guides/troubleshooting.md:340-352
• tests/test_distributed.py:209-216
• MANIFEST.in:5
• Dockerfile:90-97

Evidence.

• Scratch probe t/probe9.py via the installed lane printed ConfigError for all three presets; coverage run on the four named suites shows trainer.py 632-666 uncovered there (resolution is exercised only by test_distributed.py from the source tree).

Recommended fix. Add an installed-wheel (or package-data introspection) test that resolves each preset and asserts the file exists, and add configs/deepspeed to the package data.

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.
• The release note names the contract this fix restores.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}