[cemililik]

Summary

Eight documents (and their Turkish mirrors) said that exit code 0 means every gate passed, that a safety-classifier load failure "halts" the pipeline and leaves no final_model/, and that classification errors count as unsafe. In code, a safety evaluation that produces no verdict (safety.evaluation_completed: false: the classifier failed to load, the probes file is missing or empty, or most verdicts were unreadable) keeps the model and the run exits 0 (4 with the approval gate) regardless of evaluation.auto_revert. This documents the real behaviour. The code is unchanged.

Related issues

Refs #87 — the documentation half. Making a missing verdict fail closed would turn exit 0 into a non-zero code for runs that exit 0 today, which is a breaking change to the public exit-code contract; the owner decided it waits for a major version.

Changes — each row shows what it said and what it says now

Where	Said	Now says
docs/standards/error-handling.md	exit 0 ⇒ every gate passed	…every gate that ran to a verdict passed; a safety evaluation with no verdict keeps the model and exits 0 regardless of auto_revert
docs/usermanuals/{en,tr}/reference/exit-codes.md	exit 0 "does mean every configured gate passed"	the same exception, and "check safety.passed and safety.evaluation_completed in the JSON output"
docs/usermanuals/{en,tr}/reference/json-output.md	success: true when the run completed	also true for a run whose safety evaluation produced no verdict; read safety.evaluation_completed
docs/reference/audit_event_catalog{,-tr}.md	all gates passed	every gate that produced a verdict passed
docs/guides/safety_compliance{,-tr}.md	"Classification errors are treated as unsafe"; "any gate failure triggers auto-revert"; flow ends "exit code 3"	a pair the classifier cannot answer is counted as unscored; no verdict ⇒ model kept, exit 0 (4 with the approval gate)
docs/guides/iso_soc2_deployer_guide{,-tr}.md (incident response)	classifier load failure ⇒ pipeline.failed propagates, no final_model/	the audit.classifier_load_failed event fires and evaluation_completed: false is recorded, the model stays, exit 0; alert on those, not on the exit code
docs/qms/risk_treatment_plan{,-tr}.md	"deployer pipeline halts on non-zero exit"	the run is not halted; branch on safety.evaluation_completed / safety.passed or on the audit event
docs/reference/architecture.md	"Errors are treated as unsafe (fail-safe principle)"	an unanswerable pair is unscored; no verdict ⇒ model kept

Checked against the code

• forgelm/safety/_orchestrator.py and _gates.py (_MAX_UNSCORED_RATIO = 0.5): when the classifier does not load, the probes file is missing or empty, or more than half the verdicts are unscored, the result has evaluation_completed=False.
• forgelm/trainer.py: if not getattr(safety_result, "evaluation_completed", True): self._log_gate_kept_no_revert(...); return True: the model is kept and the run continues to success (or to the approval gate).

What this changes for a deployer

A pipeline that gates promotion on "exit 0" would promote a model whose safety evaluation never produced a verdict. The pages now tell it to read safety.evaluation_completed and safety.passed. This lowers what could be relied on from the exit code and matches the behaviour that has always been there.

Not done here

• Making a missing verdict fail closed (breaking; major version).
• The site strings that say a failing gate always ends with exit 3 (Code does not match site/compliance.html: Benchmark gates per-task floor; if the post-train number drops below the floor, the run fails (exit 3). (CR-A01-010) #89), six locales.

Type

• Documentation

Testing

• The full CLAUDE.md gauntlet from a clean worktree of this branch: exit 0 (bilingual parity, code-block parity, anchors, source-path references, exit-code prose).

Checklist

• English and Turkish mirrors updated together (Turkish wording needs the owner's review)
• CHANGELOG entry under [Unreleased]

🤖 Generated with Claude Code

[sourcery-ai]

Sorry @cemililik, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 13 hours and 15 minutes by commenting @sourcery-ai review. Upgrade to get a review now.

[coderabbitai]

Warning

Review limit reached

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Next included review available in 17 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 3 included reviews currently available. Your 48 included PR review attempts over the past 7 days set your current allowance at 3 reviews per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Essentials
• Run ID: 5a542cb3-7401-47cc-b325-8655ec52b94b

📥 Commits

Reviewing files that changed from the base of the PR and between 56b4f0d and 9f5f699.

📒 Files selected for processing: 15

• CHANGELOG.md
• docs/guides/iso_soc2_deployer_guide-tr.md
• docs/guides/iso_soc2_deployer_guide.md
• docs/guides/safety_compliance-tr.md
• docs/guides/safety_compliance.md
• docs/qms/risk_treatment_plan-tr.md
• docs/qms/risk_treatment_plan.md
• docs/reference/architecture.md
• docs/reference/audit_event_catalog-tr.md
• docs/reference/audit_event_catalog.md
• docs/standards/error-handling.md
• docs/usermanuals/en/reference/exit-codes.md
• docs/usermanuals/en/reference/json-output.md
• docs/usermanuals/tr/reference/exit-codes.md
• docs/usermanuals/tr/reference/json-output.md

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

This documentation-only PR updates English and Turkish compliance, exit-code, JSON-output, audit, architecture, deployer, and risk-treatment guidance to reflect that a safety evaluation with no verdict keeps the model and exits 0 (or 4 with approval), while scored failures still revert and exit 3; it also adds a changelog entry.

Flow diagram for safety evaluation verdict handling

flowchart TD
    A[Safety evaluation] --> B{Verdict produced?}
    B -->|Yes| C{Safety gate passed?}
    C -->|Yes| D[Keep model]
    C -->|No, auto_revert enabled| E[Revert model and exit 3]
    B -->|No: evaluation_completed false| D
    D --> F[Exit 0]
    D --> G[Exit 4 with approval gate]
    F --> H[Inspect safety.evaluation_completed and safety.passed]
    G --> H

Loading

File-Level Changes

Change	Details	Files
Correct exit-code and gate semantics for safety evaluations that produce no verdict.	Document safety.evaluation_completed: false for classifier-load failures, missing or empty probes, and predominantly unreadable verdicts. Clarify that the model is kept and the run exits 0, or 4 with human approval, regardless of auto_revert. Distinguish no-verdict evaluations from scored gate failures, which still auto-revert and exit 3. Update deployer guidance to branch on JSON safety fields or audit events rather than exit status alone.	docs/standards/error-handling.md docs/usermanuals/en/reference/exit-codes.md docs/usermanuals/tr/reference/exit-codes.md docs/usermanuals/en/reference/json-output.md docs/usermanuals/tr/reference/json-output.md docs/reference/audit_event_catalog.md docs/reference/audit_event_catalog-tr.md docs/guides/safety_compliance.md docs/guides/safety_compliance-tr.md docs/guides/iso_soc2_deployer_guide.md docs/guides/iso_soc2_deployer_guide-tr.md docs/qms/risk_treatment_plan.md docs/qms/risk_treatment_plan-tr.md docs/reference/architecture.md
Align safety, audit, architecture, and operational documentation with the existing no-verdict runtime behavior.	Replace claims that classification errors are unsafe with the unscored-pair behavior. Explain that no-verdict runs still complete successfully and emit the relevant audit signal. Update audit-event descriptions so successful completion does not imply every safety gate produced a verdict. Mirror the behavior changes in English and Turkish documentation.	docs/guides/safety_compliance.md docs/guides/safety_compliance-tr.md docs/guides/iso_soc2_deployer_guide.md docs/guides/iso_soc2_deployer_guide-tr.md docs/qms/risk_treatment_plan.md docs/qms/risk_treatment_plan-tr.md docs/reference/audit_event_catalog.md docs/reference/audit_event_catalog-tr.md docs/reference/architecture.md
Record the documentation correction in the unreleased changelog.	Note the unchanged runtime behavior and the deferred fail-closed behavior due to its exit-code compatibility impact. Reference the affected documentation areas and issue Safety gate with evaluation_completed=False (no usable evidence) ends the run as a success: exit 0, success:true, pipeline.completed, even with auto_revert (CR-A02-001) #87.	CHANGELOG.md

Possibly linked issues

• #DOC-TR10-011: The PR directly corrects the issue’s English and Turkish exit-code statements, including the no-verdict exception and JSON fields to check.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[cemililik]

Superseded by #1000: the wave-0 fixes are combined by theme into five pull requests to cut the review load. This branch's commits are included there unchanged apart from the CHANGELOG.md merge; the branch stays on the remote.