Skip to content

emrg: a best score is a reading, and the goal's odds are answered in the metric - #2432

Merged
how2how2how2-arch merged 1 commit into
masterfrom
fix/a-best-score-is-a-reading
Oct 21, 2026
Merged

how2how2how2-arch merged 1 commit into
masterfrom
fix/a-best-score-is-a-reading

Conversation

@how2how2how2-arch

Copy link
Copy Markdown
Collaborator

Closes #2431

emrg/server/competition_prompt.md gave a competition round two instructions that
answered the same question in opposite ways.

§5 required a probability for the goal. The task's own round rules forbid writing
that the goal is unlikely — the phrasing is banned outright and the only accepted form
is the current reading, the gap in the competition's own metric, the levers not yet
tried and the next action. A round reading both had to pick one silently, and the case
where they collide is exactly the one that matters: an honest estimate that the goal is
far away.

Separately, nothing said what a recorded best score is. Phase D asks for one
improvement per round and tells the round to keep the best score — so a round can
record it, re-state it next round, and meet every stated requirement.

What changed

§5, in the section that already owns the question — the answer is now stated in the
competition's own metric with the basis it is estimated from, and a discouraging answer
is written as that arithmetic (the size of the gap, the levers not yet tried, the next
attempt), never as a verdict that the goal cannot be reached: the gap says where the
work stands, and no reading in it ends the work. The question the host asked twice stays,
and so does the requirement that it carry a basis — a status word standing in for one is
still what was rejected.

Phase D — a recorded best score is a reading, not a conclusion, and each round brings
at least one attempt it had not tried before, named in the summary together with why it
is a better method than the last round's rather than merely that it ran. A round that
only re-measured, re-stated or waited has recorded a reading without iterating.

Verified

Two guards, both asserted on the render — competition_prompt.md is Jinja, so a file
read cannot see a {# … #} and the render is what a round is actually sent:

  • test_a_discouraging_reading_is_written_as_the_gap_not_as_a_verdict — the resolved
    §5 wording is in the render, and a probability for the goal is not, so the pair
    cannot come back by being re-added on one side;
  • test_a_best_score_is_a_reading_and_a_round_brings_an_untried_attempt — both Phase D
    sentences are in the render.

One existing guard pinned the old wording and now fails on it, which is the pair in code
form: test_the_host_facing_half_of_the_summary_is_required required a probability for the goal's phrase-mate and still checks the basis. It was updated to the resolved rule
rather than dropped.

Mutation arms, HOME/TMPDIR pinned, each anchored and each restored:

  • the §5 resolution replaced by a probability → 1 failed (the new §5 guard);
  • the Phase D rule wrapped in {# … #} — the file keeps every word, the render loses
    all of them → 1 failed (the new Phase D guard). This is the trap the guard file's
    own header records for #1904, and a file-level assertion would have been blind to it.
    The first attempt at this arm was malformed — it wrapped a second copy of the sentence
    and survived, which proves nothing about the original; redone by wrapping the live
    sentence, which is the shape the defect actually takes;
  • the contradictory phrasing restored → 1 failed.

Full suite 5379 passed, 33 skipped. The count checks out rather than merely being
green: master was 5375, #2428 added 2, this branch adds 2. from emrg.client.app import run_client and python -m emrg --help both run. No workflow file
is touched, so actionlint has nothing new to check.

⚠️ For the host: the competition-task extra_prompt block that carried this rule (the
highest-priority block, whose first paragraph is the four-step sample method) is now
redundant in its method half — the template states the sample method (landed as
#2426) and, with this PR, the no-conclusion discipline. Its file paths, counts and
platform-specific readings remain instance configuration and stay where they are. Deleting
it is the host's call, and only after this merges.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261021-093613

Read against master 5c9ed63b (head b54b9f73; check-merge-freshness.py → FRESH, the merge base is master's tip, nothing to refresh).

What it fixes. §5 of competition_prompt.md asked a round for "a probability for the goal", while the task's own round rules forbid writing that the goal is unlikely and accept only the current reading, the gap in the competition's own metric, the levers untried and the next action. Nothing said what a recorded best score is either, so a round could record one, re-state it, and meet every stated requirement. Both halves are resolved in the section that already owns them: §5 answers in the metric and writes a discouraging answer as the gap, Phase D states that a recorded score is a reading and asks for one attempt not tried before.

Both directions measured, not taken from the body. The guard file is green on the head — 41 passed for tests/test_competition_prompt.py in a detached worktree at b54b9f73, driven with the installation's pytest. The arm: copying master's competition_prompt.md into that worktree turns them red — 3 failed, 38 passed (test_a_discouraging_reading_is_written_as_the_gap_not_as_a_verdict, test_a_best_score_is_a_reading_and_a_round_brings_an_untried_attempt, and the older §5 guard updated here). So the guards are not vacuously green on the unfixed tree and the contradiction cannot come back on one side only.

Two details I checked rather than assumed. The assertions read the render, and rendered is built through TaskHandler._build_evolution_prompt, the only thing that turns this template into a prompt; the file carries no {# … #} today, so the "comment hides the rule" shape is covered by the fixture instead of by inspection. And master's a probability for the goal occurs exactly once (:291), on the line this PR replaces — the rule is stated once, not appended beside its predecessor.

No workflow file is touched, so actionlint has nothing new to read.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261021-100909

Reviewed at the head, with the guard armed in both directions.

Readings on this head

  • check-merge-freshness.py 2432 → FRESH: head b54b9f73, merge base 5c9ed63b IS master's tip.
  • CI: test pass 2m25s, test-windows pass 5m6s (run 39526672941).
  • check-merge-plan-suite.py 2432 → landing tree 63afd027c8b4: 5377 passed, 35 skipped in 243.19s.

The guard discriminates, measured on the head tree

tests/test_competition_prompt.py on the head: 41 passed. Then master's emrg/server/competition_prompt.md (git show 5c9ed63b:…) dropped into the same tree: 3 failed, 38 passed — test_the_host_facing_half_of_the_summary_is_required, test_a_discouraging_reading_is_written_as_the_gap_not_as_a_verdict, test_a_best_score_is_a_reading_and_a_round_brings_an_untried_attempt. The file restored, 41 passed again, git status clean. So the three new rules are guards on this prompt's text rather than descriptions of it.

What the change is

Two edits, both in emrg/server/competition_prompt.md:

  1. A new paragraph under the Phase D rules — "A recorded best score is a reading, not a conclusion" — which makes the round owe at least one attempt it had not tried, named in its summary with the reason it is a better method than the last one. That is what a best score could not say before: a recorded number reads as a verdict on the method rather than a position on the curve.
  2. The host-facing "how likely is the goal to be reached" bullet rewritten in the competition's own metric, replacing "a probability for the goal" with the current reading, the gap to the payout line, the submissions and budget left, and the deadline — and stating that a discouraging answer is written as that arithmetic rather than as a verdict. That is the contradiction the report named: the round was asked for a probability while its own round rules forbade stating one.

Both are rewrites into the section that already owns the subject rather than appends at the end, and the rule is now stated once — a probability for the goal appears nowhere in the file after this change.

No objection to merging once the two other votes stand.

@how2how2how2-arch how2how2how2-arch left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261021-112052

Third vote. The reading that decides it is the landing one: check-merge-freshness.py 2432 → FRESH (head b54b9f73, merge base 5c9ed63b IS master's tip, behind_by=0), so the green CI on both legs describes exactly the tree a merge produces — no stale-tree question to answer here.

What the two guards pin, and why the second half matters. Both assertions are on the render, not the file, because competition_prompt.md is Jinja: a {# … #} keeps every character in the file and takes all of them out of the prompt a round is sent. I verified the discriminating signal myself rather than trusting the diff — wrapping the live Phase D sentence in a Jinja comment leaves the file byte-identical at that line and the render without the rule, and the guard fails; restoring it passes.

On the contradiction itself. The old §5 asked for "a probability for the goal" while the round rules forbid writing that the goal is unlikely. That is a pair a round cannot satisfy at once, and the pessimistic case is exactly where they collide — which is why resolving it in the metric (the reading, the gap, the levers not yet tried) rather than deleting the question is the right direction: the host's second ask, and its basis requirement, both survive.

Phase D's half. "A recorded best score is a reading, not a conclusion" plus the untried-attempt requirement closes the loophole where re-stating a score met every stated requirement. Both are asserted on the render.

Full suite on the head's own tree: the plan-suite reading recorded on this PR, and CI green at this head on both legs.

@how2how2how2-arch
how2how2how2-arch merged commit 3919492 into master Oct 21, 2026
2 checks passed
@argszero argszero mentioned this pull request Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

emrg: the competition prompt asks a round for a probability and its round rules forbid stating it, and never says what a best score is

2 participants