Skip to content

docs/usermanuals/en/reference/exit-codes.md: broken example — if: steps.train.outcome == 'failure' && steps.train.conclusion == 'failure' ... if [ "${{ steps.train.outputs.exit-code }}" = "3" ] ... (… (DOC-BLEGAL-033) #137

Open

Description

@cemililik

Wave 1 · theme D1-10: Documentation-only fixes · highest severity High · 1 finding(s) · effort docs fix

Summary

The documented regression gate fails open. In EN the 'Block on regression' step is skipped. In TR (outcome-only condition) it runs, but the empty exit-code never equals 3. An exit-3 eval regression passes the pipeline either way.

Primary finding

DOC-BLEGAL-033 — docs/usermanuals/en/reference/exit-codes.md: broken example — if: steps.train.outcome == 'failure' && steps.train.conclusion == 'failure' ... if [ "${{ steps.train.outputs.exit-code }}" = "3" ] ... (…

severity High

Where. docs/usermanuals/en/reference/exit-codes.md:37,39,99 (Turkish mirror: docs/usermanuals/tr/reference/exit-codes.md:37,39,99) · type: broken example

The document says. if: steps.train.outcome == 'failure' && steps.train.conclusion == 'failure' ... if [ "${{ steps.train.outputs.exit-code }}" = "3" ] ... (e.g. steps.<id>.outputs.exit-code in GitHub Actions)

What the code does. GitHub Actions contexts reference: conclusion is the result 'after continue-on-error is applied', and 'When a continue-on-error step fails, the outcome is failure, but the final conclusion is success.' The conclusion == 'failure' condition is therefore never true, and the blocking step never runs. steps..outputs holds only outputs the step writes to $GITHUB_OUTPUT. There is no built-in exit-code output, so outputs.exit-code is empty.

User impact. The documented regression gate fails open. In EN the 'Block on regression' step is skipped. In TR (outcome-only condition) it runs, but the empty exit-code never equals 3. An exit-3 eval regression passes the pipeline either way.

Proposed correction. run: | set +e; forgelm --config configs/run.yaml; rc=$?; echo "exit-code=$rc" >> "$GITHUB_OUTPUT"; exit $rc — then if: steps.train.outcome == 'failure' and test steps.train.outputs.exit-code. At :99: 'steps..outputs.exit-code only exists if your step writes it to $GITHUB_OUTPUT'.

Turkish mirror. the Turkish page has the same error; fix both in the same PR

Acceptance criteria

  • Each statement is re-checked against the code at fix time.
  • Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
  • Every finding in this issue is resolved or explicitly closed with a reason.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationseverity: highWrong behaviour or a false claim users rely onsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 1Roadmap wave 1: high severity, small effort

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions