cicd.md 'JSON output mode' section documents a non-existent .verdict key and JSON log events on stdout (CR-A04-008) #433

Activity

1. cemililik

   added this to the v0.12.0 — Wave 1: public contract and documented usage milestone on Oct 9, 2026
2. cemililik

   added

   bugSomething isn't workingSomething isn't working

   area: pipelineCode area: pipelineCode area: pipeline

   severity: highWrong behaviour or a false claim users rely onWrong behaviour or a false claim users rely on

   source: review-2026-10Found by the October 2026 code and documentation reviewFound by the October 2026 code and documentation review

   wave: 1Roadmap wave 1: high severity, small effortRoadmap wave 1: high severity, small effort

   on Oct 9, 2026
3. cemililik

   added this to ForgeLM Project Planon Oct 9, 2026
4. cemililik

   added a parent issue on Oct 9, 2026

   • [Epic] D1-8 · Docs describing features that do not exist (doc side of code findings) #855
5. cemililik

   added

   good first issueGood for newcomersGood for newcomers

   on Oct 22, 2026
6. cemililik commented on Oct 22, 2026

   

   cemililik

   on Oct 22, 2026

   CollaboratorAuthor

   More actions

   Where to start

   Thanks for taking a look. Claim it by starting a comment with /assign so nobody duplicates the work.

   This section has two errors, and #216 reports the first of them on the same lines. One pull request fixes both issues; put Fixes #433 and Fixes #216 in its description.

   Files. The "JSON output mode" section of docs/usermanuals/en/operations/cicd.md (about lines 146-153) and of its Turkish mirror docs/usermanuals/tr/operations/cicd.md (about lines 147-154).

   What is wrong.

   • The example pipes the training run into jq '.verdict' and shows "success". The training result has no verdict key, so jq prints null; the key is success. The envelope is built by _build_result_json_envelope in forgelm/cli/_result.py.
   • The sentence under it says every event logged to stderr is also emitted as a JSON object on stdout. That is not true. In JSON mode, _setup_logging in forgelm/cli/_logging.py only raises the log level to WARNING; logs stay plain text on stderr and stdout carries only the envelope.

   What to change (already decided).

   1. In both languages, change the example to the block below. The block is identical in English and Turkish:

      $ forgelm --config configs/run.yaml --output-format json | jq '.success'
      true

   2. EN: replace the sentence under the block with: The JSON envelope goes to stdout; human-readable logs go to stderr. See [JSON output schemas](#/reference/json-output) for every field.

   3. TR: replace the matching sentence with: JSON envelope **stdout**'a gider; insana okunabilir loglar **stderr**'e gider. Tüm alanlar için [JSON çıktı şemaları](#/reference/json-output) sayfasına bakın. The first half is copied from the Turkish JSON-output page, so you do not need to write Turkish; a maintainer reads it in review.

   4. Leave the three forgelm audit ... | jq -e '.verdict != "errors" ...' lines higher up on the page alone. They belong to TR mirror drift in docs/usermanuals/en/operations/cicd.md: GitLab audit job: forgelm audit data/train.jsonl --output-format json | jq -e '.verdict != "errors" and .pii_summary.severity != "high"' (… (DOC-TR15-001) #357.

   Note for the PR description. #412 tracks real training runs that print some library text to stdout before the envelope. This PR corrects only the documented key and the logging sentence; it changes no code.

   Test. Add a test to tests/test_usermanual_reference_consistency.py and parametrize it over lang in ("en", "tr").

   • Read docs/usermanuals/<lang>/operations/cicd.md and find the --output-format json | jq '<filter>' line of the training run.
   • Assert that the filter's key is one of the keys of _build_result_json_envelope(TrainResult(success=True)). forgelm.cli._result and forgelm.results provide them; tests/test_strict_json.py imports both.
   • Assert that the retired sentence is gone: in English also emitted as a JSON object on stdout, in Turkish stdout'ta da JSON nesnesi olarak çıkar.

   The test fails today and passes after the fix.

   pytest tests/test_usermanual_reference_consistency.py --no-cov
   python3 tools/check_bilingual_parity.py --strict
   python3 tools/check_bilingual_code_blocks.py --strict
   python3 tools/check_anchor_resolution.py --strict
   python3 tools/check_tr_links_prefer_mirror.py --strict
   python3 tools/check_usermanual_self_contained.py --strict

   Add one line under ## [Unreleased] → ### Fixed in CHANGELOG.md.

   Open the PR against development with Fixes #433 and Fixes #216 (findings CR-A04-008 and DOC-D3B02-016).