emrg: competition Phase D iterates without stating how the right answer is established #2425
Description
What is missing
emrg/server/competition_prompt.md §Phase D tells a round to make one improvement per round
— "local validation → submit → record the score delta and the hypothesis behind the
improvement" — and to record failed attempts so the pitfall is not repeated.
It never says how to find out what the right answer is, and the unit it implies is the whole
population: adjust the thing, re-score, read the delta. That unit cannot answer the question when
the population's own convention is not uniform. A change applied across a whole class of cases
mixes the ones it fixed with the ones it broke into a single score, and that score cannot be
attributed to anything — it is a number with two opposite causes inside it. A round reading it
can neither keep the change nor revert it, and has spent real submissions to learn that.
Two pieces of evidence that the method exists but has no home here:
- an installation's
config.extra_promptfor this task carries a four-step procedure for exactly
this — classify the cases, draw a small sample from one class, use submissions to establish
the sample's outcome, then turn that outcome into one concrete change — written there because
the task configuration was the only channel available to it; - the same project's memory index has grown to 134 lines against the index rule's 100, with
its most prominent block being that procedure rather than a project fact: state being kept
outside the prompt because the prompt has no place for it.
What a fix must satisfy
- §Phase D states the procedure, in the section that already owns round-to-round
improvement — not appended at the end of the file, and not stated twice. - It names all four steps: classify the cases that are not right; draw a small sample from
one class and submit it as a probe; read the probe's outcome as the class's answer; turn that
answer into one concrete change. - It says why the sample rather than the whole class — the unattributable score above — and
that widening to the whole class is for when the sample's reading cannot be interpreted. - It names the last step as the one most easily left out, so that a round which classified,
counted or observed and stopped is recognisable as having improved nothing. - The wording is the template's own: no dates, no history of where the rule came from, no
reference to any installation's configuration. - A test asserts the four steps are present, so the rule cannot be dropped in silence.
Activity
how2how2how2-arch commentedon Oct 20, 2026
Handled by #2426