Repository navigation
Evals: round 6 (use-the-details rule) - logged, rule reverted - #8
Merged
Merged
Conversation
Adds 'Use the user's details' to SKILL.md: use at least one stated number or limit to rule out a check, size or drop an option, or change the order, and keep each move doable with the user's stated resources (name any extra need in the Risk cell). Targets the Fit gap seen in every eval round. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X9xscdGXL5xQmmxnwEf9T7
Fit stayed at 4.09; overall gap +0.8 vs +1.6 in round 4, within judge noise. One run turned the rule into a wrong sample-size calculation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X9xscdGXL5xQmmxnwEf9T7
davehallmon
force-pushed
the
skill/fit-use-details
branch
from
October 4, 2026 18:30
5f968fc to
49463cf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Logs eval round 6, a test of a
SKILL.mdrule aimed at Fit, and then reverts the rule because it didn't work. Net effect on the Skill: none.SKILL.mdmatchesmain.Commits:
skill:adds "Use the user's details": let a stated number or limit change a check, option, or order, and keep moves doable with the user's resources.evals:logs round 6 inevals/runs/round-6_draft/and updatesevals/README.mdandCHANGELOG.md.Revertremoves the rule.Round 6 results (same 11 problems, same baselines as round 4)
If you want to keep the rule anyway
Drop the revert commit before merging.
Checklist
CONTRIBUTING.md.SKILL.mdunder 500 lines (unchanged after the revert).README.mdif needed (not needed: no Skill change).CHANGELOG.md.python3 evals/check_rules.py problem-lens/examples/*.md(3 of 3 pass).🤖 Generated with Claude Code
https://claude.ai/code/session_01X9xscdGXL5xQmmxnwEf9T7
Generated by Claude Code