diff --git a/emrg/server/evolution_prompt.md b/emrg/server/evolution_prompt.md index 00eeb1b6..6e2590d6 100644 --- a/emrg/server/evolution_prompt.md +++ b/emrg/server/evolution_prompt.md @@ -448,7 +448,7 @@ That index is the only memory path this prompt names, because the rule is about **A round is billed for the whole conversation, not for the slice it added.** Measured 2026-10-15 over the 45 cycles that finished in one day: the median cycle ran **66** rounds, the median round brought **545** new tokens — most rounds re-send everything already known to do one small thing. Rounds are where a cycle's cost lives (host decision, item (a) of rant 2026-10-15T12:35:03, the requirement recorded as issue #2290). -So **when no call's arguments depend on what another call returns, issue them together**: several tool calls in one round, of any tool the cycle has (`bash`, `read`, `grep`, `glob`), their answers arriving together. The unit is the **round** — the same unit every session is given (`emrg/server/prompts/system.j2`) — and the reason is cost: three calls in one round are billed as one round, not three. This is not speed and not parallelism; the daemon runs a round's calls one after another, so nothing here overlaps. One shape stays apart, and it is not a licence to split a reading across rounds: where one call's answer decides what the next should do, that is a judgement, and a judgement gets its own round; a round that would carry a call the shell tool may time out on is split for the same reason. **Batch readings; never batch a judgement.** +So **issue every call you can already specify, in one response**: sending several tool calls in one round, of any tool the cycle has (`bash`, `read`, `grep`, `glob`), is the normal case — a reading that already knows its subject does not wait a round of its own. The unit is the **round** — the same unit every session is given (`emrg/server/prompts/system.j2`) — and the reason is cost: three calls in one round are billed as one round, not three, while every round re-sends the conversation and emits fresh output. This is not speed and not parallelism; the daemon runs a round's calls one after another, so nothing here overlaps. One shape stays apart, and it is not a licence to split a reading across rounds: **a call goes alone only when its arguments need an answer you have not read yet**, and then its `intent` names the specific answer it waits for; a round that would carry a call the shell tool may time out on is split for the same reason. **Batch readings; never batch a judgement.** Batching is bought with the round, never with the shell: the background-process red line in **Forbidden** below outranks this rule, and none of the forms it names is a batching technique here. diff --git a/emrg/server/prompts/system.j2 b/emrg/server/prompts/system.j2 index 45fed84c..eb25f85e 100644 --- a/emrg/server/prompts/system.j2 +++ b/emrg/server/prompts/system.j2 @@ -37,7 +37,7 @@ Each skill is a file of instructions. When one bears on what the host asked, rea - **Every call carries an `intent`**: one plain sentence on why it is made and what it should reach. It is never executed, but the host reads it, so it must be true and specific. - **Read a file before you change it**, so the change matches what is there. -- **Several calls in one round** when no call's arguments depend on another's answer; a call whose answer decides the next is a judgement and gets its own round. Every round re-sends the conversation, so three calls in one round cost one round. +- **Issue every call you can already specify, in one response.** Sending several is the normal case: a call goes alone only when its arguments need an answer you have not read yet, and then its `intent` names the specific answer it waits for. Every round re-sends the conversation and emits fresh output. ## Memory diff --git a/scripts/check-batching-share.py b/scripts/check-batching-share.py new file mode 100755 index 00000000..2fc0d645 --- /dev/null +++ b/scripts/check-batching-share.py @@ -0,0 +1,266 @@ +#!/usr/bin/env python3 +"""How many calls does one round carry, and does a lone one say what it is waiting for? + +The several-calls rule (`emrg/server/prompts/system.j2`, rulebook R10 of the evolution +prompt) changed its default on 2026-10-20: it used to permit several calls "when no call's +arguments depend on another's answer" and now asks for every call that can already be +specified, leaving a call alone only when its arguments need an answer not yet read. A +wording change is cheap to claim and cheap to get wrong -- the same rule was rewritten once +before and the share did not move -- so the claim is checked against what the rounds +actually did, and the check has to be the *same* one before and after. That is this tool: +the numbers it prints now are the "before" reading, and the same command run against a +session that grew up under the new wording is the "after" reading. + +Two readings, and they are not the same kind of thing: + + * **the share** -- of every assistant record carrying at least one tool call, the share + carrying two or more. Exact: it counts `tool_calls` and judges nothing. + * **the lone intent** -- of the records carrying exactly one call, the share whose + `intent` names no answer it waits for. This one is a **lexical proxy** and is reported + as one: an intent counts as naming an answer only when it carries one of the markers + printed under `--markers`, so the reading is auditable rather than trusted. The + distinction it exists for is why the rule needed a second number at all -- a total that + rises cannot tell "applied the new rule and judged wrongly" from "did not apply it", + while an intent that never mentions an answer is evidence of the second. + +What it reads, and it says so before any number: `history_*.jsonl` under the roots given, +one line per record, records on the `records_to_messages` shape -- `type: message`, +`role: assistant`, and `tool_calls[]` either at the top level or under a `content` object. + +Exit codes: 0 a reading was taken; 2 nothing measurable -- no history file under the roots +(or every one of them unreadable), which is never a pass. + + check-batching-share.py # the host's whole corpus under ~/.emrg + check-batching-share.py ~/.emrg/evolution/emrg/.emrg --since 2026-10-21 + check-batching-share.py --json # the same reading for another program + +A subject is a **corpus root** -- a directory to walk or one `history_*.jsonl` file -- and is a +positional, like `check-icon-transparency.py`'s subjects: this tool's subject is a set of session +histories, not a working tree, so it is classified among the guards that answer about something +other than a tree and prints the roots it read instead of a `tree:` line. +""" + +from __future__ import annotations + +import argparse +import json +import os +import re +import sys +from pathlib import Path + +#: Phrases that mark an `intent` as naming an answer it waits for. English is the language every +#: tool call's `intent` is written in (the prompt requires it), so the list is English-only and short +#: on purpose: a longer list would make the proxy agree with whatever the reader hoped for. Printed +#: by `--markers` so the reading can be argued with instead of taken on faith. +DEPENDENCY_MARKERS: tuple[str, ...] = ( + "the result", + "its result", + "the output", + "its output", + "from the", + "based on", + "depending on", + "depends on", + "whether", + "the answer", + "if the", + "if it", + "whatever", + "after seeing", + "so that i can", + "then the", + "what the", +) + +#: A `history_YYMMDD.jsonl` name is the only thing that makes a file part of the corpus; the +#: contents decide nothing about membership, so a file that cannot be parsed is counted and named. +HISTORY_NAME = re.compile(r"^history_\d{6}\.jsonl$") + + +def history_files(roots: list[Path]) -> list[Path]: + """Every `history_*.jsonl` under `roots`, deduplicated, in a stable order.""" + seen: set[Path] = set() + for root in roots: + if root.is_file() and HISTORY_NAME.match(root.name): + seen.add(root) + continue + if not root.is_dir(): + continue + for dirpath, _dirnames, filenames in os.walk(root): + for name in filenames: + if HISTORY_NAME.match(name): + seen.add(Path(dirpath) / name) + return sorted(seen) + + +def tool_calls_of(record: dict) -> list: + """The record's tool calls, wherever this shape keeps them -- never a guess beyond the two.""" + calls = record.get("tool_calls") + if not isinstance(calls, list): + content = record.get("content") + calls = content.get("tool_calls") if isinstance(content, dict) else None + return calls if isinstance(calls, list) else [] + + +def intent_of(call: dict) -> str: + """The `intent` inside a call's `arguments`, or `""` when it says nothing readable.""" + function = call.get("function") if isinstance(call, dict) else None + raw = function.get("arguments") if isinstance(function, dict) else None + if isinstance(raw, dict): + args = raw + elif isinstance(raw, str): + try: + args = json.loads(raw) + except (ValueError, TypeError): + return "" + else: + return "" + intent = args.get("intent") if isinstance(args, dict) else None + return intent if isinstance(intent, str) else "" + + +def names_an_answer(intent: str) -> bool: + """Whether an `intent` carries one of `DEPENDENCY_MARKERS` -- the proxy, stated plainly.""" + lowered = intent.casefold() + return any(marker in lowered for marker in DEPENDENCY_MARKERS) + + +def read_corpus(files: list[Path], since: str | None) -> dict: + """Walk the corpus and count. `since` is an ISO date; records before it are not counted.""" + reading = { + "files": len(files), + "records": 0, + "unreadable_lines": 0, + "rounds": 0, + "multi": 0, + "lone": 0, + "lone_without_answer": 0, + "first_timestamp": None, + "last_timestamp": None, + "counts": {}, # exact call-count histogram, so the share can be re-derived + } + for path in files: + try: + handle = path.open(encoding="utf-8", errors="replace") + except OSError: + reading["unreadable_lines"] += 1 + continue + with handle: + for line in handle: + try: + record = json.loads(line) + except ValueError: + reading["unreadable_lines"] += 1 + continue + if not isinstance(record, dict): + reading["unreadable_lines"] += 1 + continue + reading["records"] += 1 + stamp = record.get("timestamp") + if isinstance(stamp, str): + if reading["first_timestamp"] is None or stamp < reading["first_timestamp"]: + reading["first_timestamp"] = stamp + if reading["last_timestamp"] is None or stamp > reading["last_timestamp"]: + reading["last_timestamp"] = stamp + if since and stamp[: len(since)] < since: + continue + if record.get("type") != "message" or record.get("role") != "assistant": + continue + calls = tool_calls_of(record) + if not calls: + continue + reading["rounds"] += 1 + reading["counts"][len(calls)] = reading["counts"].get(len(calls), 0) + 1 + if len(calls) >= 2: + reading["multi"] += 1 + continue + reading["lone"] += 1 + if not any(names_an_answer(intent_of(call)) for call in calls): + reading["lone_without_answer"] += 1 + return reading + + +def share(part: int, whole: int) -> float: + """A share, and 0.0 rather than a division when the subject is empty.""" + return (part / whole) if whole else 0.0 + + +def render(reading: dict, roots: list[Path], since: str | None) -> str: + """The reading, naming its subject before any number.""" + window = f"{reading['first_timestamp'] or '-'} .. {reading['last_timestamp'] or '-'}" + lines = [ + f"subject: {reading['files']} history file(s) under " + + ", ".join(str(root) for root in roots), + f"window: {window}" + (f" (records before {since} not counted)" if since else ""), + f"records: {reading['records']} line(s) read" + + (f", {reading['unreadable_lines']} unreadable" if reading["unreadable_lines"] else ""), + ] + if not reading["rounds"]: + lines.append("rounds: 0 -- no assistant record carrying a tool call; nothing measurable") + return "\n".join(lines) + lines += [ + f"rounds: {reading['rounds']} assistant record(s) carrying at least one tool call", + f" share with two or more: {reading['multi']} " + f"({100 * share(reading['multi'], reading['rounds']):.1f}%)", + f" share with exactly one: {reading['lone']} " + f"({100 * share(reading['lone'], reading['rounds']):.1f}%)", + f" lone intents naming no awaited answer (lexical proxy): " + f"{reading['lone_without_answer']} of {reading['lone']} " + f"({100 * share(reading['lone_without_answer'], reading['lone']):.1f}%)", + " calls per round: " + + ", ".join(f"{n} -> {c}" for n, c in sorted(reading["counts"].items())), + ] + return "\n".join(lines) + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser( + description="Measure how many tool calls one round carries, and whether a lone call " + "names the answer it waits for.", + epilog="Exit 0 a reading was taken; 2 nothing measurable under the roots. Never 2 for 0.", + ) + parser.add_argument( + "roots", + nargs="*", + metavar="CORPUS", + help="a directory to walk for history_*.jsonl, or one such file; zero or more " + "(default: ~/.emrg)", + ) + parser.add_argument( + "--since", + default=None, + metavar="YYYY-MM-DD", + help="count only records whose timestamp starts on or after this date", + ) + parser.add_argument("--json", action="store_true", help="print the reading as JSON") + parser.add_argument( + "--markers", action="store_true", help="print the dependency markers the proxy uses" + ) + args = parser.parse_args(argv) + + if args.markers: + print("\n".join(DEPENDENCY_MARKERS)) + return 0 + + roots = [Path(raw).expanduser() for raw in (args.roots or ["~/.emrg"])] + files = history_files(roots) + if not files: + print( + "not measurable: no history_*.jsonl under " + + ", ".join(str(root) for root in roots), + file=sys.stderr, + ) + return 2 + + reading = read_corpus(files, args.since) + if args.json: + print(json.dumps({**reading, "roots": [str(r) for r in roots], "since": args.since}, + ensure_ascii=False, indent=2)) + else: + print(render(reading, roots, args.since)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/check-release-prompt.py b/scripts/check-release-prompt.py index 018f8981..eadd2e99 100644 --- a/scripts/check-release-prompt.py +++ b/scripts/check-release-prompt.py @@ -23,8 +23,10 @@ `source/emrg/server/prompts/system.j2`. Measured on the published `EMRG-0.3.21-linux-x86_64.tar.gz`: one member, `runtime/source/emrg/server/prompts/system.j2`, 24,558 characters. That product predates -the rewrite, and it is this tool's "before" reading: the same member of a v0.3.22 artifact -must read 7,360 characters with `## Hard limits` where the old one has `## Tool Usage`. +the rewrite, and it is this tool's "before" reading: the same member of a later artifact must +carry `## Hard limits` where the old one has `## Tool Usage`. Its size is not restated here — +the prompt is edited, so any figure written beside it is stale the next time a rule moves; the +tool compares against this tree's copy at run time, which is the only number that stays true. Subjects it reads: diff --git a/tests/test_a_tree_reading_guard_names_its_tree.py b/tests/test_a_tree_reading_guard_names_its_tree.py index 9889f769..8e688e87 100644 --- a/tests/test_a_tree_reading_guard_names_its_tree.py +++ b/tests/test_a_tree_reading_guard_names_its_tree.py @@ -330,6 +330,17 @@ def _refusal_reason(out: str) -> str: "instead, as check-issue-links.py does)" ), "check-patch-files.py": "reads the patch files it is given as arguments", + "check-batching-share.py": ( + "reads a corpus of session history files - `history_*.jsonl` under the roots given " + "as positional subjects, defaulting to the host's `~/.emrg` - and counts the tool " + "calls in each assistant record. A working tree is not its subject and no tree's " + "name would answer anything: the readings are about what agent rounds did, which " + "lives in gitignored host data a clean checkout does not carry. It names every root " + "it read, the file count and the record window before any number, as " + "check-icon-transparency.py prints the subject it read, and its empty-corpus " + "refusal (rc 2, with the roots named) is pinned by tests/test_check_batching_share.py " + "- added by issue #2406, whose two acceptance numbers had no command behind them" + ), "check-merge-freshness.py": "requires PR numbers; answers per head and its base", "check-merge-order.py": "requires PR numbers; names the base it plans against", "check-merge-pairs.py": "requires PR numbers; names the base it folds onto", diff --git a/tests/test_check_batching_share.py b/tests/test_check_batching_share.py new file mode 100644 index 00000000..7b8e263e --- /dev/null +++ b/tests/test_check_batching_share.py @@ -0,0 +1,250 @@ +"""`check-batching-share.py` measures two different kinds of thing, and both are measured here. + +The tool exists because a wording change asked for a before/after number twice — the share of +rounds carrying more than one tool call, and the share of *lone* rounds whose `intent` names no +answer they wait for — and a number a cycle re-derives from scratch each time is not the same +number. So the corpus is built here the way the daemon writes one (`history_YYYYMMDD.jsonl`, a +JSON object per line, `tool_calls[]` at the top level or under a `content` object), the counts +are known by construction, and every reading the tool prints is checked against them. + +Two properties are worth more than the arithmetic: + + * **the exact reading stays exact** — the share and the call-count histogram come from + `len(tool_calls)` alone, so a fixture with one, two and three calls must move both; + * **the proxy says it is a proxy** — the lone-`intent` share is lexical, so both directions + are built: an intent carrying a dependency marker counts as naming an answer, an intent + carrying none counts against, and the marker list is readable from the tool itself rather + than restated here (a copy in this file would drift from the list it describes). + +And the family's third answer: a corpus that does not exist is `2`, a refusal to measure, +never a `0` that reads as "no rounds, so nothing to fix". +""" + +from __future__ import annotations + +import importlib.util +import json +import sys +from pathlib import Path + +import pytest + +REPO_ROOT = Path(__file__).resolve().parent.parent +TOOL = REPO_ROOT / "scripts" / "check-batching-share.py" + + +def _load(): + spec = importlib.util.spec_from_file_location("check_batching_share", TOOL) + module = importlib.util.module_from_spec(spec) + sys.modules[spec.name] = module + spec.loader.exec_module(module) + return module + + +@pytest.fixture() +def tool(): + return _load() + + +def _call(name: str, intent: str) -> dict: + """One tool call in the daemon's shape: `arguments` is JSON text, `intent` rides inside.""" + return { + "id": f"call_{name}", + "type": "function", + "function": {"name": name, "arguments": json.dumps({"intent": intent})}, + } + + +def _round(*calls: dict, at: str = "2026-10-20T10:00:00", nested: bool = False) -> dict: + """An assistant record carrying `calls`, in either of the two shapes the reader accepts.""" + record = {"timestamp": at, "type": "message", "role": "assistant", "content": ""} + if nested: + record["content"] = {"tool_calls": list(calls)} + else: + record["tool_calls"] = list(calls) + return record + + +def _corpus(root: Path, records: list[dict], name: str = "history_261020.jsonl") -> Path: + """Write `records` as one session's history file and answer the directory holding it.""" + target = root / "s_000000_0000_test" + target.mkdir(parents=True, exist_ok=True) + (target / name).write_text( + "\n".join(json.dumps(record) for record in records) + "\n", encoding="utf-8" + ) + return root + + +def _run(tool, *args: str, capsys) -> tuple[int, str, str]: + rc = tool.main(list(args)) + captured = capsys.readouterr() + return rc, captured.out, captured.err + + +def test_the_share_and_the_histogram_count_what_the_records_carry(tool, tmp_path, capsys): + """Three rounds — one lone, one double, one triple — answer 2 of 3, and the histogram says so.""" + root = _corpus( + tmp_path, + [ + _round(_call("bash", "read the config")), + _round(_call("bash", "read the config"), _call("read", "read the tests")), + _round( + _call("bash", "read the config"), + _call("read", "read the tests"), + _call("grep", "find the callers"), + ), + ], + ) + + rc, out, _err = _run(tool, str(root), capsys=capsys) + + assert rc == 0, out + assert "rounds: 3 assistant record(s)" in out + assert "share with two or more: 2 (66.7%)" in out + assert "share with exactly one: 1 (33.3%)" in out + assert "1 -> 1, 2 -> 1, 3 -> 1" in out + + +def test_a_record_without_tool_calls_is_not_a_round(tool, tmp_path, capsys): + """The denominator is rounds, not messages: prose and tool results are neither.""" + root = _corpus( + tmp_path, + [ + _round(_call("bash", "read the config")), + {"timestamp": "2026-10-20T10:00:01", "type": "message", "role": "assistant", + "content": "an answer with no call in it"}, + {"timestamp": "2026-10-20T10:00:02", "type": "message", "role": "user", + "content": "a question"}, + {"timestamp": "2026-10-20T10:00:03", "type": "tool_result", "content": "output"}, + ], + ) + + rc, out, _err = _run(tool, str(root), capsys=capsys) + + assert rc == 0, out + assert "rounds: 1 assistant record(s)" in out + + +def test_a_nested_content_object_is_read_the_same_way(tool, tmp_path, capsys): + """Two shapes exist in the corpus, so both are counted — one reader, not two.""" + root = _corpus( + tmp_path, + [ + _round(_call("bash", "a"), _call("read", "b"), nested=True), + _round(_call("bash", "a")), + ], + ) + + rc, out, _err = _run(tool, str(root), capsys=capsys) + + assert rc == 0, out + assert "rounds: 2 assistant record(s)" in out + assert "share with two or more: 1 (50.0%)" in out + + +def test_the_lone_intent_proxy_reads_both_directions(tool, tmp_path, capsys): + """An intent that names an answer counts for the rule; one that does not counts against it.""" + root = _corpus( + tmp_path, + [ + _round(_call("bash", "check whether the suite is green before deciding")), + _round(_call("read", "look at the settings file")), + ], + ) + + rc, out, _err = _run(tool, str(root), capsys=capsys) + + assert rc == 0, out + assert "lone intents naming no awaited answer (lexical proxy): 1 of 2 (50.0%)" in out + + +def test_the_marker_list_is_readable_from_the_tool_itself(tool, capsys): + """The proxy is auditable: `--markers` prints the list the reading was taken with.""" + rc, out, _err = _run(tool, "--markers", capsys=capsys) + + assert rc == 0 + printed = out.splitlines() + assert printed == list(tool.DEPENDENCY_MARKERS) + assert printed, "an empty marker list would make every lone intent count against the rule" + + +def test_an_intent_that_is_not_json_counts_against_the_rule(tool, tmp_path, capsys): + """A call whose `arguments` cannot be read has no intent to name an answer with.""" + broken = {"id": "x", "type": "function", + "function": {"name": "bash", "arguments": "not json"}} + root = _corpus(tmp_path, [_round(broken)]) + + rc, out, _err = _run(tool, str(root), capsys=capsys) + + assert rc == 0, out + assert "lone intents naming no awaited answer (lexical proxy): 1 of 1 (100.0%)" in out + + +def test_since_counts_only_records_from_that_date_on(tool, tmp_path, capsys): + """The after reading is taken over a window, so the window has to actually exclude.""" + root = _corpus( + tmp_path, + [ + _round(_call("bash", "a"), at="2026-10-19T23:59:59"), + _round(_call("bash", "a"), _call("read", "b"), at="2026-10-20T00:00:01"), + ], + ) + + rc, out, _err = _run(tool, str(root), "--since", "2026-10-20", capsys=capsys) + + assert rc == 0, out + assert "rounds: 1 assistant record(s)" in out + assert "naming no awaited answer (lexical proxy): 0 of 0" in out + assert "share with two or more: 1 (100.0%)" in out + assert "records before 2026-10-20 not counted" in out + + +def test_no_history_file_answers_two_and_names_the_roots(tool, tmp_path, capsys): + """Nothing measurable is a refusal, never a zero that reads as a clean corpus.""" + empty = tmp_path / "sessions" + empty.mkdir() + + rc, out, err = _run(tool, str(empty), capsys=capsys) + + assert rc == 2, out + assert "not measurable" in err + assert str(empty) in err + + +def test_a_corpus_of_no_rounds_still_names_its_subject(tool, tmp_path, capsys): + """A real corpus with nothing in it is a `0` reading that says so — not a silent pass.""" + root = _corpus(tmp_path, [{"timestamp": "2026-10-20T10:00:00", "type": "message", + "role": "user", "content": "hello"}]) + + rc, out, _err = _run(tool, str(root), capsys=capsys) + + assert rc == 0, out + assert "1 history file(s)" in out + assert "rounds: 0" in out + assert "nothing measurable" in out + + +def test_json_carries_the_same_reading(tool, tmp_path, capsys): + """Another program reads the same numbers, so the two forms must not disagree.""" + root = _corpus(tmp_path, [_round(_call("bash", "a"), _call("read", "b"))]) + + rc, out, _err = _run(tool, str(root), "--json", capsys=capsys) + + assert rc == 0, out + payload = json.loads(out) + assert payload["rounds"] == 1 + assert payload["multi"] == 1 + assert payload["counts"] == {"2": 1} + assert payload["files"] == 1 + + +def test_a_file_named_for_the_corpus_is_read_directly(tool, tmp_path, capsys): + """A subject may be one history file, which is how a single session is measured.""" + root = _corpus(tmp_path, [_round(_call("bash", "a"), _call("read", "b"))]) + single = root / "s_000000_0000_test" / "history_261020.jsonl" + + rc, out, _err = _run(tool, str(single), capsys=capsys) + + assert rc == 0, out + assert "1 history file(s)" in out + assert "share with two or more: 1 (100.0%)" in out diff --git a/tests/test_evolution_prompt_round_cost.py b/tests/test_evolution_prompt_round_cost.py index db110d4c..3a548cbb 100644 --- a/tests/test_evolution_prompt_round_cost.py +++ b/tests/test_evolution_prompt_round_cost.py @@ -60,7 +60,7 @@ #: rule's home; §1 is where the cycle is actually reading. REQUIRED_TERMS: dict[str, tuple[str, ...]] = { R10_HEADING: ( - "issue them together", # the rule + "issue every call you can already specify", # the rule, and its direction "several tool calls in one round", # the unit, spelled as system.j2 spells it "Batch readings; never batch a judgement", # the boundary "outranks", # the red line's precedence, named @@ -78,14 +78,15 @@ #: #: These are the longest phrases the two carriers genuinely share, and that is a measurement rather #: than a preference: R10 is a rulebook entry a cycle reads and `system.j2` is one bullet in every -#: session's prefix, so the two state the same rule at different lengths — `several tool calls in one -#: round, of any tool the cycle has (…)` against `**Several calls in one round** when no call's -#: arguments depend on another's answer`. The phrases below are where they agree word for word (the -#: concrete three-call case, the criterion's consequence, the boundary that decides it), and a drift -#: in any of them still moves both — which is the property the comparison exists for. -SHARED_UNIT = "three calls in one round" -SHARED_CRITERION = "gets its own round" -SHARED_BOUNDARY = "no call's arguments depend on" +#: session's prefix, so the two state the same rule at different lengths — `issue every call you can +#: already specify, in one response: sending several tool calls in one round … is the normal case` +#: against `**Issue every call you can already specify, in one response.** Sending several is the +#: normal case: …`. The phrases below are where they agree word for word (the rule's direction, +#: the boundary that decides a lone call, and what that lone call must still say), and a drift in +#: any of them still moves both — which is the property the comparison exists for. +SHARED_UNIT = "every call you can already specify" +SHARED_CRITERION = "its arguments need an answer you have not read yet" +SHARED_BOUNDARY = "names the specific answer it waits for" #: The unit this rule may no longer be stated in. The first version of R10 taught exactly #: these; the host retired the merge-several-calls-into-one reading explicitly at @@ -151,7 +152,7 @@ def test_both_sides_of_the_loop_state_the_batching_rule() -> None: def test_the_rule_is_stated_once() -> None: """One home for the rule: a second copy drifts from the first.""" text = TEMPLATE.read_text(encoding="utf-8") - found = text.count("issue them together") + found = text.count("issue every call you can already specify") assert found == 1, ( f"the batching rule is stated {found} times in emrg/server/evolution_prompt.md; " "it lives in rulebook R10 and every other section cites R10" diff --git a/tests/test_several_calls_rule_reach.py b/tests/test_several_calls_rule_reach.py index 7dfae5f5..65c63376 100644 --- a/tests/test_several_calls_rule_reach.py +++ b/tests/test_several_calls_rule_reach.py @@ -26,6 +26,17 @@ actor, stated in a carrier some actors never open — here, stated *incorrectly* in the one carrier all of them open. +The default direction was reversed on 2026-10-20 (issue #2406), because the sentence above — +kept word for word through that rewrite — put the burden of proof on showing independence and was +never met. The reading is `scripts/check-batching-share.py`'s, not this file's: over every +`history_*.jsonl` under `~/.emrg` (2026-07-27 .. 2026-10-20, **133,276 assistant rounds** carrying +at least one tool call) **9.3%** hold two or more, and of the lone rounds **92.5%** carried an +`intent` naming no awaited answer — a lexical proxy, whose markers the tool prints. The bullet now +states issuing several as the normal case and requires a lone call's `intent` to name what it waits +for, so the observable that separates "judged it necessary" from "did not engage" is the sentence +itself. That is still a placement and wording guard: the share is re-measured by the instrument, +over the window a later cycle names, never here. + What is asserted ---------------- 1. The rule is in `system.j2` **and in the render**, and once only: a block in the file that no render @@ -65,15 +76,15 @@ #: The rule's identity: the bullet's lead phrase. Written as the phrase a rewrite would keep, so a second #: copy anywhere in the template is visible as a count of 2 rather than as longer prose. -RULE_LEAD = "**Several calls in one round**" +RULE_LEAD = "**Issue every call you can already specify, in one response.**" #: One term per part of the ruling, each a verbatim substring of the shipped wording — so this list cannot #: drift away from the sentence it checks without this file saying so. RULE_TERMS = ( - RULE_LEAD, # the unit: a round, not a command - "when no call's arguments depend on another's answer", # the boundary - "a call whose answer decides the next is a judgement and gets its own round", # what stays split - "Every round re-sends the conversation, so three calls in one round cost one round", # the reason: money + RULE_LEAD, # the unit: one response, several calls + "a call goes alone only when its arguments need an answer you have not read yet", # the boundary + "names the specific answer it waits for", # what the lone call must still say + "Every round re-sends the conversation and emits fresh output", # the reason: money ) #: The precedence clause's carrier: the section that owns the red lines, whose own paragraph says no