GGUF export never finds convert_hf_to_gguf.py in a real llama-cpp-python install, so the default forgelm export path always fails (CR-A13-005) #113

Wave 0 · theme D0-8: Packaging and delivery promises · highest severity High · 4 finding(s) · effort medium

Summary

An operator runs pip install 'forgelm[export]' and then forgelm export ./final_model --output m.gguf. The run exits 2 with "convert_hf_to_gguf.py not found inside the llama-cpp-python installation", whichever llama-cpp-python version is installed.

Primary finding

CR-A13-005 — GGUF export never finds convert_hf_to_gguf.py in a real llama-cpp-python install, so the default forgelm export path always fails

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced with stubs · already tracked: KNOWN-OPEN CLI-02 — docs/roadmap/phase-16-trust-surface-hardening.md:491-495 (S7 open; C-11)

What happens. _find_converter_script looks in three places only: llama_cpp/, llama_cpp/scripts/ and the parent of llama_cpp/ (site-packages). No published llama-cpp-python wheel puts the converter in any of them. The floor release named in the extra (0.2.90) ships it as bin/convert_hf_to_gguf.py at the wheel root (its RECORD says so), which installs to site-packages/bin/, a directory that is not searched. The current release, 0.3.36 (2026-10-01), does not ship it at all: its pyproject sets wheel.packages=["llama_cpp"], and llama.cpp master's CMakeLists has no install rule for the script. On llama.cpp master the script also imports a sibling conversion package and gguf, so it only runs from inside a llama.cpp checkout. Result: F-131 (and therefore F-125) works only when the operator sets FORGELM_GGUF_CONVERTER. The error hint suggests pip install 'llama-cpp-python>=0.2.90', which does not help. The comments in pyproject ("bundled convert_hf_to_gguf.py") and export.py:4,33 state a packaging fact that is false. The unit tests (tests/test_export.py:160-167) plant the script inside llama_cpp/, a layout no real wheel has, so the suite stays green.

Where.

• forgelm/export.py:129-136
• forgelm/export.py:33
• forgelm/export.py:143-146
• pyproject.toml:126-131
• tests/test_export.py:160-167
• forgelm/export.py:4
• forgelm/export.py:415-422
• forgelm/cli/subcommands/_export.py:66-69
• docs/usermanuals/en/getting-started/installation.md:72-79
• forgelm/cli/subcommands/_export.py:16

Evidence.

• Wheel llama_cpp_python-0.2.90-cp312-macosx_11_0_arm64 (GitHub release v0.2.90-metal, sha256 48aa12fb...) RECORD: bin/convert_hf_to_gguf.py. Wheel llama_cpp_python-0.3.36-py3-none-macosx_11_0_arm64 (v0.3.36-metal, sha256 64c98e6a...): no convert_hf_to_gguf.py among 103 entries. llama-cpp-python pyproject [tool.scikit-build] wheel.packages=["llama_cpp"]. llama.cpp master convert_hf_to_gguf.py:18 from conversion import (...). Repro: both wheels extracted into the work dir, sys.modules['llama_c…

Recommended fix. Stop claiming the converter is bundled. Either probe site-packages/bin/ and /bin/ as well (this covers only old wheels), or document FORGELM_GGUF_CONVERTER pointing into a llama.cpp checkout as the supported path and make the error message say that. Fix the test fixture so it uses a real wheel layout.

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

CR-A16-023 — The [export] extra cannot supply convert_hf_to_gguf.py: llama-cpp-python wheels package only llama_cpp/, so forgelm export fails unless FORGELM_GGUF_CONVERTER is set

severity Medium · reachability default · intent contract-violation · confidence likely (static only) · independently confirmed · reproduced with stubs · already tracked: KNOWN-OPEN CLI-02 (inferred) — docs/roadmap/phase-16-trust-surface-hardening.md:491-495; docs/roadmap/risks-and-decisions.md:191 (C-11)

What happens. pyproject.toml:126-131 describes the export extra as 'GGUF export via llama-cpp-python's bundled convert_hf_to_gguf.py'. _find_converter_script (forgelm/export.py:121-146) looks in llama_cpp/, llama_cpp/scripts/ and site-packages/. llama-cpp-python's pyproject sets [tool.scikit-build] wheel.packages = ["llama_cpp"], its CMakeLists installs only shared libraries, and the GitHub tree of llama_cpp/ at tag v0.2.90 (the declared floor) and on main contains no convert script. The converter lives in vendor/llama.cpp, which is not installed. A standard pip install forgelm[export] therefore yields FileNotFoundError with a hint to reinstall llama-cpp-python, which cannot help. The converter also imports gguf, which no extra declares. forgelm doctor reports the export extra present as long as llama_cpp imports (_doctor.py:107).

Failure scenario. An operator follows the docs (pip install forgelm[export], then forgelm export). Export fails with 'converter not found', and the suggested fix (reinstall llama-cpp-python) does not change the outcome.

Where.

• pyproject.toml:126-132
• forgelm/export.py:121-147
• forgelm/export.py:143
• forgelm/cli/subcommands/_doctor.py:107
• docs/guides/quickstart.md:194
• forgelm/export.py:414-422
• forgelm/cli/subcommands/_doctor.py:421-440
• forgelm/export.py:100
• forgelm/cli/subcommands/_export.py:14-23
• forgelm/cli/subcommands/_export.py:70

Evidence.

• WebFetch: raw llama-cpp-python pyproject.toml shows wheel.packages=['llama_cpp']; CMakeLists.txt install() covers libs only; github.com/abetlen/llama-cpp-python/tree/v0.2.90/llama_cpp and tree/main/llama_cpp list no file containing 'convert'. Stub check: a wheel-shaped fake llama_cpp on sys.path makes forgelm.export._find_converter_script raise FileNotFoundError ('convert_hf_to_gguf.py not found inside the llama-cpp-python installation...'). The real wheel was not installed or inspected (no lan…

Recommended fix. Verify against a real wheel. If confirmed, document FORGELM_GGUF_CONVERTER as required (with a matching llama.cpp checkout and gguf-py), or vendor or pin a converter source and declare gguf with a range. Fix the misleading hint at export.py:143.

TECH-T08-001 — Export and serving targets: llama-cpp-python>=0.2.90 as the source of a 'bundled' convert_hf_to_gguf.py ([export] extra and converter discovery) (incorrect-claim)

severity Now · already tracked: KNOWN-OPEN CLI-02|CLI-17|CLI-18|CLI-09|CLI-12|CLI-13 docs/roadmap/risks-and-decisions.md:191

Item. llama-cpp-python>=0.2.90 as the source of a 'bundled' convert_hf_to_gguf.py ([export] extra and converter discovery) · verdict: incorrect-claim · urgency: Now

State in ForgeLM. pyproject.toml:126-132 declares [export] = llama-cpp-python>=0.2.90 (non-win32) for 'GGUF export via llama-cpp-python's bundled convert_hf_to_gguf.py'. forgelm/export.py:4, :33, :90, :93 and :298 repeat the 'bundled' claim. _find_converter_script (export.py:89-147) uses FORGELM_GGUF_CONVERTER if set; otherwise it imports llama_cpp and checks /, /scripts/ and the parent of the package (:131-136). The error hint at :143-147 tells the user to reinstall llama-cpp-python. forgelm doctor checks the export extra by importing llama_cpp (forgelm/cli/subcommands/_doctor.py:107). The README quick-start runs forgelm export ... --quant q4_k_m (README.md:65). Tests create a fake llama_cpp/convert_hf_to_gguf.py in tmp_path (tests/test_export.py:157-189), so they never see the real package layout.

State of the ecosystem on 2026-10-01. llama-cpp-python is actively maintained: 0.3.36 reached PyPI on 2026-10-01 (0.3.35 on 2026-08-17, 0.3.34 on 2026-07-12). PyPI has never carried a wheel for it: 209 releases, 0 wheel files. The converter exists only inside the sdist, at llama_cpp_python-0.3.36/vendor/llama.cpp/convert_hf_to_gguf.py, next to a vendored conversion/ package (94 files) and gguf-py/. The sdist's pyproject.toml:66 sets wheel.packages = ["llama_cpp"] (v0.2.90 has the same setting), and its CMakeLists only installs shared libraries into llama_cpp/lib. The 0.3.36 metal wheel from GitHub has 103 members and no converter, gguf-py or conversion files. The script itself is now a 312-line shim: line 15 adds gguf-py to sys.path and line 18 runs from conversion import, so a copy of the lone file fails (llama.cpp issue #​23171, 'No module named conversion'). I reproduced the failure: I built a stub site-packages from the 0.3.36 wheel member list and ran the snapshot code. _find_converter_script raised FileNotFoundError, and export_model returned success=False with error_kind=runtime (exit 2).

Impact. forgelm export fails on every pip install of the [export] extra, at every version in the declared range, unless FORGELM_GGUF_CONVERTER points inside a full llama.cpp checkout (converter, conversion/ and gguf-py/, plus the converter's own requirements). The README front-door example exits with code 2. forgelm doctor reports the export extra as healthy, and the error message sends users to a reinstall that cannot help. The failure is loud and the inputs are TC-T, so this is a broken feature claim, not a vulnerability. The roadmap decision C-11 (docs/roadmap/risks-and-decisions.md:191) already plans real converter discovery, but the snapshot code and docs still make the claim.

Recommendation. Now: correct the claims and the error text, and make doctor test for an actual converter (script plus its conversion/ and gguf-py/ siblings) instead of import llama_cpp. Document that a llama.cpp checkout is required and that FORGELM_GGUF_CONVERTER must point into it. Next release: implement the C-11 discovery (PATH, sys.prefix and a configured llama.cpp directory). Add a contract test against the real sdist/wheel layout instead of a tmp_path fake. Either drop llama-cpp-python from [export] or describe it as runtime bindings only. Remove the reference to docs/guides/export.md, which does not exist.

Sources.

• PyPI llama-cpp-python JSON (0.3.36 sdist 2026-10-01; 209 releases, 0 wheel files) (2026-10-01)
• llama_cpp_python-0.3.36 sdist (pyproject.toml:66 wheel.packages; vendor/llama.cpp/convert_hf_to_gguf.py:15,18) (2026-10-01)
• llama-cpp-python 0.3.36 metal wheel (103 members, no converter) (2026-10-01)
• llama-cpp-python v0.2.90 pyproject.toml (wheel.packages = llama_cpp) (2024-08-29)
• llama.cpp #23171: convert_hf_to_gguf.py: No module named 'conversion' (2026-05-17)
• llama-cpp-python releases feed (v0.3.36 2026-10-01) (2026-10-01)

DOC-D4B06-003 — notebooks/post_training_workflow.ipynb: false claim — forgelm export wraps llama-cpp-python's bundled convert_hf_to_gguf.py; pip install 'forgelm[export]' # adds llama-cpp-python>=0.2.90.

severity High · already tracked: KNOWN-OPEN CLI-02|CLI-17|CLI-18|CLI-09|CLI-12|CLI-13 docs/roadmap/phase-16-trust-surface-hardening.md:491-495; docs/roadmap/risks-and-decisions.md:191

Where. notebooks/post_training_workflow.ipynb cell 8 (markdown) and cell 9 (pip extra comment) · type: false claim

The document says. forgelm export wraps llama-cpp-python's bundled convert_hf_to_gguf.py; pip install 'forgelm[export]' # adds llama-cpp-python>=0.2.90.

What the code does. export.py:101-142 looks for convert_hf_to_gguf.py only in the installed llama_cpp package root, its scripts/ dir and parent dir, else raises FileNotFoundError. The shared technology verdict (tech-verdicts.json T08 answer Q1, arbiter-confirmed) states PyPI has never carried a wheel and no wheel or pip build installs the converter (it exists only in the sdist under vendor/llama.cpp, and wheel.packages=['llama_cpp']). I could not confirm offline: llama_cpp is not installed in the review venv; the export attempt stopped at 'llama-cpp-python is required'.

Code references. forgelm/export.py:101-142, pyproject.toml:126-132

User impact. Following the notebook (pip install the extra, run forgelm export) likely ends with FileNotFoundError and exit 2 unless FORGELM_GGUF_CONVERTER points at a llama.cpp checkout script.

Proposed correction. State that the converter script must come from a llama.cpp checkout and show export FORGELM_GGUF_CONVERTER=/path/to/llama.cpp/convert_hf_to_gguf.py, until export discovery is fixed.

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.
• The release note names the contract this fix restores.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}