Skip to content

docs: fix KTO data-shape row to use completion column - #938

Merged
cemililik merged 4 commits into
HodeTech:developmentfrom
vortsghost2025:fix/kto-docs-completion-885
Oct 17, 2026
Merged

cemililik merged 4 commits into
HodeTech:developmentfrom
vortsghost2025:fix/kto-docs-completion-885

Conversation

@vortsghost2025

@vortsghost2025 vortsghost2025 commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #885
Resolves: F-W1-DOCS-TRAIN-02

Out of scope: the {prompt, completion} → SFT row is a separate defect tracked in #403 and is intentionally left unchanged here.

The "What to look at in your data" table in choosing-trainer.md documented KTO rows as {prompt, response, label: bool}, but _validate_trainer_columns in forgelm/data.py requires completion and label columns — data shaped the way the page recommended fails with a KeyError.

Changed the KTO row to {prompt, completion, label: bool} in both the English page and the Turkish mirror.

Summary by Sourcery

Correct the documented KTO data shape so users provide columns accepted by the trainer.

Enhancements:

  • Correct the KTO data-shape documentation to use the required completion column in the English and Turkish trainer-selection guides.

Documentation:

  • Update the changelog and localized trainer-selection documentation to reflect the correct KTO columns and clarify that the SFT row remains unchanged.

The choosing-a-trainer table documented KTO rows as
{prompt, response, label: bool}, but the KTO loader requires
completion and label columns. Updated in English and Turkish.

Fixes HodeTech#885
@sourcery-ai

sourcery-ai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

The documentation now directs KTO users to provide {prompt, completion, label: bool} rows, matching _validate_trainer_columns and preventing the documented example from causing a missing-column KeyError.

File-Level Changes

Change Details Files
Correct the documented KTO dataset schema to match the trainer’s required column names.
  • Replace response with completion in the KTO data-shape row.
  • Apply the correction consistently to the English and Turkish documentation mirrors.
docs/usermanuals/en/concepts/choosing-trainer.md
docs/usermanuals/tr/concepts/choosing-trainer.md

Assessment against linked issues

Issue Objective Addressed Explanation
#885 Correct the KTO data shape in the choosing-a-trainer table to use completion instead of response in both the English and Turkish documentation. ✅

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai

coderabbitai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

This review ran on the open-source allowance, not this organization's plan, because the pull request author doesn't have an assigned seat. Waiting won't change this — ask an organization admin to assign them a seat, or add seats in Billing if every seat is already assigned, then retry.

Next included review available in 47 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2dd395fa-0783-4808-8d29-a85c2c8f93e9
📥 Commits

Reviewing files that changed from the base of the PR and between 48aa82e and 10b2ce2.

📒 Files selected for processing: 3
  • CHANGELOG.md
  • docs/usermanuals/en/concepts/choosing-trainer.md
  • docs/usermanuals/tr/concepts/choosing-trainer.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

sourcery-ai[bot]
sourcery-ai Bot previously approved these changes Oct 14, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!

Sourcery assessment

Approved.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

@cemililik

Copy link
Copy Markdown
Collaborator

Hi @vortsghost2025, thanks for picking up #885! 🙏

The fix itself is correct, and I checked it against the loader rather than by reading alone:

  • _validate_trainer_columns (in forgelm/data.py) accepts {prompt, completion, label} for kto and still raises the KeyError for the old {prompt, response, label} shape, so the new row is what the loader actually requires.
  • The English and Turkish rows are changed together and stay aligned. The doc guards from the issue pass on your branch (bilingual parity and code blocks, anchors, TR-link mirror, user-manual self-containment and schema drift), as do the extra source_path_refs, numerical_claims and yaml_snippets guards, and tests/test_data.py (100 passed).

There are two small things I'd like added to the PR description so the issue closes cleanly. No code change is needed.

1. Please say explicitly that the SFT row is out of scope.
The acceptance criteria on #885 say "every data shape in the table loads with the trainer it recommends". The first row of the same table, {prompt, completion} → SFT, does not load today (I reproduced it on current development: KeyError: Dataset must contain 'User'/'instruction' and 'Assistant'/'output' columns ... Suggested trainer: grpo). That row is tracked in #403, and the "Where to start" comment on #885 asks to leave it to that issue, so leaving it unchanged is the right call. The issue also accepts a finding being "explicitly closed with a reason", so a line like this in the description is all that's needed:

Out of scope: the {prompt, completion} → SFT row is a separate defect tracked in #403 and is intentionally left unchanged here.

2. Please name the finding ID.
The convention is to list the finding ID a change resolves so it can be ticked off in the backlog. A line Resolves: F-W1-DOCS-TRAIN-02 in the description (and ideally the commit body) is enough. Keep Fixes #885.

FYI, no action needed in this PR: while checking I noticed docs/usermanuals/{en,tr}/concepts/alignment-overview.md describes KTO as "(prompt, response, good?)". That is prose rather than a column name and outside the issue's listed locations, so please leave it out of this PR (one concern per PR); we'll consider it separately.

The project CI shows action_required because workflows from first-time contributors need a maintainer's approval; that is routine and nothing you need to fix.

Thanks again, a clean and focused change!

cemililik and others added 2 commits October 17, 2026 20:00
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Adds the [Unreleased] > Fixed entry for the "Choosing a Trainer" KTO row. The
SFT row of the same table is a separate defect tracked in HodeTech#403 and is left
unchanged.

Resolves: F-W1-DOCS-TRAIN-02
Refs HodeTech#885

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sourcery assessment

Approved.

@cemililik

Copy link
Copy Markdown
Collaborator

Hi @vortsghost2025, thank you for taking on #885, and for keeping the English and Turkish pages in step. 🙏

To get this ready to merge, I pushed two small commits to your branch (a regular push, no force-push):

I also added the finding ID and the "out of scope" line about the SFT row to the PR description myself, since the squash commit takes its text from there, so you don't need to do anything further. The guards and the full CI matrix are green, and I'll merge it now.

Thanks again for the clean, focused fix!

@cemililik
cemililik merged commit ea82d68 into HodeTech:development Oct 17, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Choosing-a-trainer table documents KTO rows as {prompt, response, label}, but the KTO loader requires a completion column (F-W1-DOCS-TRAIN-02)

2 participants