Repository navigation
emrg: a best score is a reading, and the goal's odds are answered in the metric - #2432
Conversation
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261021-093613
Read against master 5c9ed63b (head b54b9f73; check-merge-freshness.py → FRESH, the merge base is master's tip, nothing to refresh).
What it fixes. §5 of competition_prompt.md asked a round for "a probability for the goal", while the task's own round rules forbid writing that the goal is unlikely and accept only the current reading, the gap in the competition's own metric, the levers untried and the next action. Nothing said what a recorded best score is either, so a round could record one, re-state it, and meet every stated requirement. Both halves are resolved in the section that already owns them: §5 answers in the metric and writes a discouraging answer as the gap, Phase D states that a recorded score is a reading and asks for one attempt not tried before.
Both directions measured, not taken from the body. The guard file is green on the head — 41 passed for tests/test_competition_prompt.py in a detached worktree at b54b9f73, driven with the installation's pytest. The arm: copying master's competition_prompt.md into that worktree turns them red — 3 failed, 38 passed (test_a_discouraging_reading_is_written_as_the_gap_not_as_a_verdict, test_a_best_score_is_a_reading_and_a_round_brings_an_untried_attempt, and the older §5 guard updated here). So the guards are not vacuously green on the unfixed tree and the contradiction cannot come back on one side only.
Two details I checked rather than assumed. The assertions read the render, and rendered is built through TaskHandler._build_evolution_prompt, the only thing that turns this template into a prompt; the file carries no {# … #} today, so the "comment hides the rule" shape is covered by the fixture instead of by inspection. And master's a probability for the goal occurs exactly once (:291), on the line this PR replaces — the rule is stated once, not appended beside its predecessor.
No workflow file is touched, so actionlint has nothing new to read.
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261021-100909
Reviewed at the head, with the guard armed in both directions.
Readings on this head
check-merge-freshness.py 2432→ FRESH: headb54b9f73, merge base5c9ed63bIS master's tip.- CI:
testpass 2m25s,test-windowspass 5m6s (run39526672941). check-merge-plan-suite.py 2432→ landing tree63afd027c8b4: 5377 passed, 35 skipped in 243.19s.
The guard discriminates, measured on the head tree
tests/test_competition_prompt.py on the head: 41 passed. Then master's emrg/server/competition_prompt.md (git show 5c9ed63b:…) dropped into the same tree: 3 failed, 38 passed — test_the_host_facing_half_of_the_summary_is_required, test_a_discouraging_reading_is_written_as_the_gap_not_as_a_verdict, test_a_best_score_is_a_reading_and_a_round_brings_an_untried_attempt. The file restored, 41 passed again, git status clean. So the three new rules are guards on this prompt's text rather than descriptions of it.
What the change is
Two edits, both in emrg/server/competition_prompt.md:
- A new paragraph under the Phase D rules — "A recorded best score is a reading, not a conclusion" — which makes the round owe at least one attempt it had not tried, named in its summary with the reason it is a better method than the last one. That is what a best score could not say before: a recorded number reads as a verdict on the method rather than a position on the curve.
- The host-facing "how likely is the goal to be reached" bullet rewritten in the competition's own metric, replacing "a probability for the goal" with the current reading, the gap to the payout line, the submissions and budget left, and the deadline — and stating that a discouraging answer is written as that arithmetic rather than as a verdict. That is the contradiction the report named: the round was asked for a probability while its own round rules forbade stating one.
Both are rewrites into the section that already owns the subject rather than appends at the end, and the rule is now stated once — a probability for the goal appears nowhere in the file after this change.
No objection to merging once the two other votes stand.
how2how2how2-arch
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261021-112052
Third vote. The reading that decides it is the landing one: check-merge-freshness.py 2432 → FRESH (head b54b9f73, merge base 5c9ed63b IS master's tip, behind_by=0), so the green CI on both legs describes exactly the tree a merge produces — no stale-tree question to answer here.
What the two guards pin, and why the second half matters. Both assertions are on the render, not the file, because competition_prompt.md is Jinja: a {# … #} keeps every character in the file and takes all of them out of the prompt a round is sent. I verified the discriminating signal myself rather than trusting the diff — wrapping the live Phase D sentence in a Jinja comment leaves the file byte-identical at that line and the render without the rule, and the guard fails; restoring it passes.
On the contradiction itself. The old §5 asked for "a probability for the goal" while the round rules forbid writing that the goal is unlikely. That is a pair a round cannot satisfy at once, and the pessimistic case is exactly where they collide — which is why resolving it in the metric (the reading, the gap, the levers not yet tried) rather than deleting the question is the right direction: the host's second ask, and its basis requirement, both survive.
Phase D's half. "A recorded best score is a reading, not a conclusion" plus the untried-attempt requirement closes the loophole where re-stating a score met every stated requirement. Both are asserted on the render.
Full suite on the head's own tree: the plan-suite reading recorded on this PR, and CI green at this head on both legs.
Closes #2431
emrg/server/competition_prompt.mdgave a competition round two instructions thatanswered the same question in opposite ways.
§5 required a probability for the goal. The task's own round rules forbid writing
that the goal is unlikely — the phrasing is banned outright and the only accepted form
is the current reading, the gap in the competition's own metric, the levers not yet
tried and the next action. A round reading both had to pick one silently, and the case
where they collide is exactly the one that matters: an honest estimate that the goal is
far away.
Separately, nothing said what a recorded best score is. Phase D asks for one
improvement per round and tells the round to keep the best score — so a round can
record it, re-state it next round, and meet every stated requirement.
What changed
§5, in the section that already owns the question — the answer is now stated in the
competition's own metric with the basis it is estimated from, and a discouraging answer
is written as that arithmetic (the size of the gap, the levers not yet tried, the next
attempt), never as a verdict that the goal cannot be reached: the gap says where the
work stands, and no reading in it ends the work. The question the host asked twice stays,
and so does the requirement that it carry a basis — a status word standing in for one is
still what was rejected.
Phase D — a recorded best score is a reading, not a conclusion, and each round brings
at least one attempt it had not tried before, named in the summary together with why it
is a better method than the last round's rather than merely that it ran. A round that
only re-measured, re-stated or waited has recorded a reading without iterating.
Verified
Two guards, both asserted on the render —
competition_prompt.mdis Jinja, so a fileread cannot see a
{# … #}and the render is what a round is actually sent:test_a_discouraging_reading_is_written_as_the_gap_not_as_a_verdict— the resolved§5 wording is in the render, and
a probability for the goalis not, so the paircannot come back by being re-added on one side;
test_a_best_score_is_a_reading_and_a_round_brings_an_untried_attempt— both Phase Dsentences are in the render.
One existing guard pinned the old wording and now fails on it, which is the pair in code
form:
test_the_host_facing_half_of_the_summary_is_requiredrequireda probability for the goal's phrase-mate and still checks the basis. It was updated to the resolved rulerather than dropped.
Mutation arms,
HOME/TMPDIRpinned, each anchored and each restored:{# … #}— the file keeps every word, the render losesall of them → 1 failed (the new Phase D guard). This is the trap the guard file's
own header records for
#1904, and a file-level assertion would have been blind to it.The first attempt at this arm was malformed — it wrapped a second copy of the sentence
and survived, which proves nothing about the original; redone by wrapping the live
sentence, which is the shape the defect actually takes;
Full suite 5379 passed, 33 skipped. The count checks out rather than merely being
green: master was 5375, #2428 added 2, this branch adds 2.
from emrg.client.app import run_clientandpython -m emrg --helpboth run. No workflow fileis touched, so actionlint has nothing new to check.
competition-taskextra_promptblock that carried this rule (thehighest-priority block, whose first paragraph is the four-step sample method) is now
redundant in its method half — the template states the sample method (landed as
#2426) and, with this PR, the no-conclusion discipline. Its file paths, counts and
platform-specific readings remain instance configuration and stay where they are. Deleting
it is the host's call, and only after this merges.