Skip to content

alignment-overview describes KTO data as (prompt, response, good?), but the KTO loader reads prompt, completion and label #965

Closed

Description

@cemililik

Summary

The "Alignment overview" user-manual page describes KTO as learning from "binary (prompt, response, good?) signals". Those are not the names the KTO loader reads: it requires prompt, completion and a boolean label. A reader who follows the page's wording and names their answer column response gets the same KeyError that #885 / #938 removed from the "Choosing a Trainer" table.

What happens

docs/usermanuals/en/concepts/alignment-overview.md:56 and the Turkish mirror at the same line (tr/concepts/alignment-overview.md:56, (prompt, response, iyi mi?)) name the KTO fields as response and good?.

_validate_trainer_columns raises KeyError: KTO trainer requires 'completion' and 'label' (boolean) columns … for any other shape (forgelm/data.py:600-607), and format detection routes only completion + label to KTO (forgelm/data.py:36-37). The KTO training page, the data-formats page and docs/guides/alignment.md already say completion + label; this bullet is the one place that still disagrees.

This is prose, not a column name in a table, which is why it was left out of #938 (one concern per PR).

Where

Recommended fix

Name the real fields and keep the plain-language meaning of the label, in both languages. For example (English):

KTO — learns from binary (prompt, completion, label) rows, where label is true for a desirable response and false for an undesirable one. Use when you only have thumbs-up/down feedback.

The Turkish bullet keeps its structure and only swaps the field names (response → completion, iyi mi? → label, with the same true / false explanation). No wording beyond those names needs native review.

Out of scope

Acceptance criteria

  • The KTO bullet names the columns the loader reads (prompt, completion, label) in the English page and its Turkish mirror, in the same PR.
  • A ### Fixed entry is added under [Unreleased] in CHANGELOG.md.
  • tools/check_bilingual_parity.py --strict, tools/check_bilingual_code_blocks.py --strict, tools/check_anchor_resolution.py --strict, tools/check_tr_links_prefer_mirror.py --strict and tools/check_usermanual_self_contained.py --strict pass.

Related


Found while reviewing PR #938. Verified against development at ea82d68.

Activity

  1. added
    documentationImprovements or additions to documentation
    severity: lowMinor defect, inaccuracy or rough edge
    wave: 4Roadmap wave 4: low / info / watch, when touching the module
    on Oct 17, 2026
  2. cemililik commented on Oct 17, 2026

    @cemililik
    CollaboratorAuthor

    Where to start

    Thanks for taking a look. Please confirm the issue is unassigned and comment before starting so work does not overlap.

    This is a two-line documentation fix: one bullet, in two languages. Start with these files on development:

    Replace (prompt, response, good?) with (prompt, completion, label) and say what label means, as suggested in the issue. Change nothing else on the page: the other items are tracked in #784, and if a PR for it touches lines 54-56 too, whichever merges second will need a small rebase.

    In the Turkish page, keep the sentence as it is and swap only the field names (response → completion, iyi mi? → label), with the same true / false explanation as the English one. If you are unsure about the Turkish phrasing, say so in the PR and a maintainer will review it; please do not machine-translate new prose (see docs/standards/localization.md).

    There is no code change, so no new test is needed. The loader's KTO check is already covered by tests/test_data.py; if you want to confirm what the page should say, this should pass on your branch:

    pytest tests/test_data.py --no-cov

    Run these guards before opening the PR:

    python3 tools/check_bilingual_parity.py --strict
    python3 tools/check_bilingual_code_blocks.py --strict
    python3 tools/check_anchor_resolution.py --strict
    python3 tools/check_tr_links_prefer_mirror.py --strict
    python3 tools/check_usermanual_self_contained.py --strict
    python3 tools/check_usermanual_schema_drift.py --strict

    Add a one-line entry under [Unreleased] → ### Fixed in CHANGELOG.md, newest entry first; the existing ones show the style.

    Open the PR against development with Fixes #965 in the description. Please do not commit screenshots or other files that are not part of the fix, and keep the English and Turkish pages structurally aligned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationgood first issueGood for newcomersseverity: lowMinor defect, inaccuracy or rough edgewave: 4Roadmap wave 4: low / info / watch, when touching the module

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions