Skip to content

emrg: competition Phase D iterates without stating how the right answer is established #2425

Closed

Description

@how2how2how2-arch

What is missing

emrg/server/competition_prompt.md §Phase D tells a round to make one improvement per round
— "local validation → submit → record the score delta and the hypothesis behind the
improvement" — and to record failed attempts so the pitfall is not repeated.

It never says how to find out what the right answer is, and the unit it implies is the whole
population: adjust the thing, re-score, read the delta. That unit cannot answer the question when
the population's own convention is not uniform. A change applied across a whole class of cases
mixes the ones it fixed with the ones it broke into a single score, and that score cannot be
attributed to anything — it is a number with two opposite causes inside it. A round reading it
can neither keep the change nor revert it, and has spent real submissions to learn that.

Two pieces of evidence that the method exists but has no home here:

  • an installation's config.extra_prompt for this task carries a four-step procedure for exactly
    this — classify the cases, draw a small sample from one class, use submissions to establish
    the sample's outcome, then turn that outcome into one concrete change — written there because
    the task configuration was the only channel available to it;
  • the same project's memory index has grown to 134 lines against the index rule's 100, with
    its most prominent block being that procedure rather than a project fact: state being kept
    outside the prompt because the prompt has no place for it.

What a fix must satisfy

  1. §Phase D states the procedure, in the section that already owns round-to-round
    improvement — not appended at the end of the file, and not stated twice.
  2. It names all four steps: classify the cases that are not right; draw a small sample from
    one class and submit it as a probe; read the probe's outcome as the class's answer; turn that
    answer into one concrete change.
  3. It says why the sample rather than the whole class — the unattributable score above — and
    that widening to the whole class is for when the sample's reading cannot be interpreted.
  4. It names the last step as the one most easily left out, so that a round which classified,
    counted or observed and stopped is recognisable as having improved nothing.
  5. The wording is the template's own: no dates, no history of where the rule came from, no
    reference to any installation's configuration.
  6. A test asserts the four steps are present, so the rule cannot be dropped in silence.

Activity

  1. how2how2how2-arch commented on Oct 20, 2026

    @how2how2how2-arch
    CollaboratorAuthor

    Handled by #2426

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions