diff --git a/CHANGELOG.md b/CHANGELOG.md index 1d502754..09c5eedb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,45 @@ All notable changes to EigenScript are documented here. ## [Unreleased] +### Added + +- **Every opcode now carries a recorded observer classification, and a gate + fails on any that does not (#972).** The optimisation this unblocks — stop + emitting observer bookkeeping when a program provably never reads observer + state — fails silently and totally when its reader set is incomplete: a + program gates its own bookkeeping off, then reads slots nobody updated, and + every binding answers `equilibrium` forever with no crash and nothing to fail + on. So the reader set has to be pinned against the closed opcode list. + **It cannot be derived from the C.** Five derivations were tried and all five + gave a confident wrong answer, in both directions: a grep of the read-side API + and `objdump -dr` relocations both miss `obs_stall_trajectory()`, which reads + `s->dH`/`s->entropy` as struct fields and so calls nothing and references no + symbol; a struct-field scan false-positives on unrelated `->n` reads; a + handler scan written against `case OP_X:` matches **nothing** (the VM + dispatches through `CASE(NAME)` computed-goto macros) and reports a clean + empty set; and the same scan repaired to `CASE(X)` matches **everything**, + including a scope marker and a writer. + So nothing is derived. All 94 opcodes were classified by reading their + handlers, and `src/vm.h` records the verdict as `obs:READS` (15), + `obs:WRITES` (10), `obs:DIAG` (1) or `obs:NONE` (68). + `tools/obs_marker_check.sh` (suite section `[99t]`) enumerates the enum and + goes red on any opcode with no marker — so a new opcode is a red line at the + moment it is added, which is when its author knows the answer and nobody else + ever will. A missing marker produces a loud unanswered question where a + derivation produced a confident wrong one. + Two verdicts contradict their opcode names and are worth knowing before + touching this area: `OP_OBSERVE_ASSIGN` is a **no-op** (the slot model + observes at `OP_OBSERVE_NAME_POST`, after the SET), and the bare + `OP_INTERROGATE` reads **no** observer state at all — `when`/`where`/`why`/ + `how` on a value operand return constants, because observer state is + binding-keyed and a bare value has no binding. `obs:DIAG` exists for + `OP_LOOP_CAP_CHECK`, whose only reach into observer state is the SIGUSR1 + dump: calling it a reader would pin bookkeeping on for every plain loop in + every program and delete the whole win, and calling it `NONE` would silently + drop the obligation that the dump must say the gate is closed rather than + render every binding as `equilibrium`. + No runtime change: the markers are comments and the gate is a test. + ### Changed - **The strict argument reform now covers the whole builtin surface, and diff --git a/docs/OBSERVER.md b/docs/OBSERVER.md index f9815c1e..212d63d2 100644 --- a/docs/OBSERVER.md +++ b/docs/OBSERVER.md @@ -428,3 +428,48 @@ interrogation. `unobserved:` is the only opt-out, and it is a real one: it skips the emission, so a hot region inside it pays nothing. + +### The classification that a future opt-out rests on (#972) + +The obvious improvement is to skip the emission automatically when a program +provably never reads observer state. That optimisation is not shipped, and the +reason is worth stating: its failure mode is silent and total. If any opcode +that reads observer state is missing from the scan that decides "is anything +reading?", a program gates its own bookkeeping off and then reads slots nobody +updated — every binding answers `equilibrium` forever, with no crash and no +failing test. + +Completeness therefore has to be pinned against the opcode set, and it **cannot +be derived from the C**. Five derivations were tried on #972 and all five gave a +confident wrong answer, in both directions: greps of the read-side API and even +`objdump -dr` relocations both miss `obs_stall_trajectory()`, which reads +`s->dH` / `s->entropy` as struct fields and calls no `observer_slot_*` function +and emits no symbol reference; a scan for struct-field reads false-positives on +unrelated `->n` reads; and a handler scan written against `case OP_X:` matches +nothing at all (the VM dispatches through `CASE(NAME)` computed-goto macros) +while the same scan repaired to `CASE(X)` matches everything, including a scope +marker and a writer. + +So every entry in the `OpCode` enum in `src/vm.h` carries a hand-recorded +marker instead: + +| marker | meaning | +|---|---| +| `obs:READS` | answers **from** recorded observer/temporal state — the set the future liveness scan consumes | +| `obs:WRITES` | records, updates, resets or stamps that state | +| `obs:DIAG` | reaches it only through the SIGUSR1 diagnostic dump, never through program-visible semantics | +| `obs:NONE` | none of the above | + +`tools/obs_marker_check.sh` enumerates the enum and goes red on any opcode with +no marker, so a new opcode is a red line at the moment it is added — which is +when its author knows the answer and nobody else ever will. The gate proves a +verdict was *recorded*; it cannot prove the verdict is *right*, and it stops at +the opcode's own handler (`OP_CALL`, the JIT's OSR entry, and assembled chunks +are excluded by name, each with the mechanism that covers it instead). + +Two results from that reading are worth knowing before touching this area, both +contrary to the opcode names: `OP_OBSERVE_ASSIGN` is a **no-op** (the slot model +observes at `OP_OBSERVE_NAME_POST` after the SET), and the bare `OP_INTERROGATE` +reads **no** observer state at all — `when` / `where` / `why` / `how` on a value +operand return constants, because observer state is binding-keyed and a bare +value has no binding. diff --git a/src/vm.h b/src/vm.h index b78c8c3c..202326d8 100644 --- a/src/vm.h +++ b/src/vm.h @@ -56,120 +56,120 @@ void vm_borrow_compensate(Value *arg, Value *result, int caller_owns_arg, /* ---- Opcodes ---- */ typedef enum { /* Constants */ - OP_CONST, /* [idx:16] push constant pool entry */ - OP_NULL, /* push null */ - OP_NUM_ZERO, /* push 0.0 */ - OP_NUM_ONE, /* push 1.0 */ + OP_CONST, /*obs:NONE*/ /* [idx:16] push constant pool entry */ + OP_NULL, /*obs:NONE*/ /* push null */ + OP_NUM_ZERO, /*obs:NONE*/ /* push 0.0 */ + OP_NUM_ONE, /*obs:NONE*/ /* push 1.0 */ /* Arithmetic (pop 2, push 1) */ - OP_ADD, - OP_SUB, - OP_MUL, - OP_DIV, - OP_MOD, + OP_ADD, /*obs:NONE*/ + OP_SUB, /*obs:NONE*/ + OP_MUL, /*obs:NONE*/ + OP_DIV, /*obs:NONE*/ + OP_MOD, /*obs:NONE*/ /* Bitwise (pop 2, push 1) */ - OP_BAND, - OP_BOR, - OP_BXOR, - OP_SHL, - OP_SHR, + OP_BAND, /*obs:NONE*/ + OP_BOR, /*obs:NONE*/ + OP_BXOR, /*obs:NONE*/ + OP_SHL, /*obs:NONE*/ + OP_SHR, /*obs:NONE*/ /* Unary (pop 1, push 1) */ - OP_NEG, - OP_NOT, - OP_BNOT, + OP_NEG, /*obs:NONE*/ + OP_NOT, /*obs:NONE*/ + OP_BNOT, /*obs:NONE*/ /* Comparison (pop 2, push 1) */ - OP_EQ, - OP_NE, - OP_LT, - OP_GT, - OP_LE, - OP_GE, + OP_EQ, /*obs:NONE*/ + OP_NE, /*obs:NONE*/ + OP_LT, /*obs:NONE*/ + OP_GT, /*obs:NONE*/ + OP_LE, /*obs:NONE*/ + OP_GE, /*obs:NONE*/ /* Variables */ - OP_GET_LOCAL, /* [slot:16] push local from frame slot */ - OP_SET_LOCAL, /* [slot:16] TOS -> local slot (keep on stack) */ - OP_GET_NAME, /* [name_idx:16] dynamic lookup by name */ - OP_SET_NAME, /* [name_idx:16] outward-assignment by name */ - OP_SET_NAME_LOCAL, /* [name_idx:16] set in current scope only */ - OP_SET_FN_NAME_LOCAL, /* [name_idx:16] set in frame->fn_env (skips intervening loop/scope envs) */ + OP_GET_LOCAL, /*obs:NONE*/ /* [slot:16] push local from frame slot */ + OP_SET_LOCAL, /*obs:WRITES*/ /* [slot:16] TOS -> local slot (keep on stack) */ + OP_GET_NAME, /*obs:NONE*/ /* [name_idx:16] dynamic lookup by name */ + OP_SET_NAME, /*obs:WRITES*/ /* [name_idx:16] outward-assignment by name */ + OP_SET_NAME_LOCAL, /*obs:WRITES*/ /* [name_idx:16] set in current scope only */ + OP_SET_FN_NAME_LOCAL, /*obs:WRITES*/ /* [name_idx:16] set in frame->fn_env (skips intervening loop/scope envs) */ /* Control flow */ - OP_JUMP, /* [offset:16] unconditional forward jump */ - OP_JUMP_BACK, /* [offset:16] unconditional backward jump */ - OP_JUMP_IF_FALSE, /* [offset:16] pop, jump if falsy */ - OP_JUMP_IF_TRUE, /* [offset:16] pop, jump if truthy */ - OP_JUMP_IF_FALSE_PEEK, /* [offset:16] peek, jump if falsy (short-circuit and) */ - OP_JUMP_IF_TRUE_PEEK, /* [offset:16] peek, jump if truthy (short-circuit or) */ + OP_JUMP, /*obs:NONE*/ /* [offset:16] unconditional forward jump */ + OP_JUMP_BACK, /*obs:NONE*/ /* [offset:16] unconditional backward jump */ + OP_JUMP_IF_FALSE, /*obs:NONE*/ /* [offset:16] pop, jump if falsy */ + OP_JUMP_IF_TRUE, /*obs:NONE*/ /* [offset:16] pop, jump if truthy */ + OP_JUMP_IF_FALSE_PEEK, /*obs:NONE*/ /* [offset:16] peek, jump if falsy (short-circuit and) */ + OP_JUMP_IF_TRUE_PEEK, /*obs:NONE*/ /* [offset:16] peek, jump if truthy (short-circuit or) */ /* Stack manipulation */ - OP_POP, /* discard TOS */ - OP_DUP, /* duplicate TOS */ - OP_DUP2, /* duplicate top two: a b → a b a b */ + OP_POP, /*obs:NONE*/ /* discard TOS */ + OP_DUP, /*obs:NONE*/ /* duplicate TOS */ + OP_DUP2, /*obs:NONE*/ /* duplicate top two: a b → a b a b */ /* Functions */ - OP_CLOSURE, /* [fn_idx:16] create closure from compiled function */ - OP_CALL, /* [argc:16] call function with argc args */ - OP_RETURN, /* return TOS */ - OP_RETURN_NULL, /* return null (implicit) */ + OP_CLOSURE, /*obs:NONE*/ /* [fn_idx:16] create closure from compiled function */ + OP_CALL, /*obs:NONE*/ /* [argc:16] call function with argc args */ + OP_RETURN, /*obs:NONE*/ /* return TOS */ + OP_RETURN_NULL, /*obs:NONE*/ /* return null (implicit) */ /* Data structures */ - OP_LIST, /* [count:16] pop count items, push list */ - OP_DICT, /* [count:16] pop count key-value pairs, push dict */ - OP_INDEX_GET, /* pop index, pop target, push target[index] */ - OP_INDEX_SET, /* pop value, pop index, pop target, set, push value */ - OP_DOT_GET, /* [name_idx:16] pop target, push target.name */ - OP_DOT_SET, /* [name_idx:16] pop value, pop target, set, push value */ + OP_LIST, /*obs:NONE*/ /* [count:16] pop count items, push list */ + OP_DICT, /*obs:NONE*/ /* [count:16] pop count key-value pairs, push dict */ + OP_INDEX_GET, /*obs:NONE*/ /* pop index, pop target, push target[index] */ + OP_INDEX_SET, /*obs:NONE*/ /* pop value, pop index, pop target, set, push value */ + OP_DOT_GET, /*obs:NONE*/ /* [name_idx:16] pop target, push target.name */ + OP_DOT_SET, /*obs:NONE*/ /* [name_idx:16] pop value, pop target, set, push value */ /* Loops and iteration */ - OP_ITER_SETUP, /* pop iterable, push iterator state */ - OP_ITER_NEXT, /* [exit_offset:16] advance or jump to exit */ - OP_LOOP_ENV_FRESH, /* create fresh child env if current was captured by closure */ - OP_LOOP_ENV_END, /* restore parent env from loop body env */ - OP_BREAK, /* unwind to enclosing loop exit */ - OP_CONTINUE, /* jump to enclosing loop header */ + OP_ITER_SETUP, /*obs:NONE*/ /* pop iterable, push iterator state */ + OP_ITER_NEXT, /*obs:NONE*/ /* [exit_offset:16] advance or jump to exit */ + OP_LOOP_ENV_FRESH, /*obs:NONE*/ /* create fresh child env if current was captured by closure */ + OP_LOOP_ENV_END, /*obs:NONE*/ /* restore parent env from loop body env */ + OP_BREAK, /*obs:NONE*/ /* unwind to enclosing loop exit */ + OP_CONTINUE, /*obs:NONE*/ /* jump to enclosing loop header */ /* Error handling */ - OP_TRY_BEGIN, /* [catch_offset:16] push exception handler */ - OP_TRY_END, /* pop exception handler */ + OP_TRY_BEGIN, /*obs:NONE*/ /* [catch_offset:16] push exception handler */ + OP_TRY_END, /*obs:NONE*/ /* pop exception handler */ /* Observer system */ - OP_OBSERVE_ASSIGN, /* [name_idx:16] observer update for assignment (env walk) */ - OP_OBSERVE_ASSIGN_LOCAL, /* [slot:16] observer update; prev value lives in fn_env slot */ - OP_INTERROGATE, /* [kind:16] pop target, push query result */ - OP_PREDICATE, /* [kind:16] push predicate result */ - OP_UNOBSERVED_BEGIN,/* increment g_unobserved_depth */ - OP_UNOBSERVED_END, /* decrement g_unobserved_depth */ - OP_LOOP_STALL_CHECK,/* [exit_offset:16] observer-stall + iteration cap (observer-based loops) */ - OP_LOOP_CAP_CHECK, /* [exit_offset:16] iteration cap ONLY (plain loops; no observer-stall) */ + OP_OBSERVE_ASSIGN, /*obs:NONE*/ /* [name_idx:16] observer update for assignment (env walk) */ + OP_OBSERVE_ASSIGN_LOCAL, /*obs:WRITES*/ /* [slot:16] observer update; prev value lives in fn_env slot */ + OP_INTERROGATE, /*obs:NONE*/ /* [kind:16] pop target, push query result */ + OP_PREDICATE, /*obs:READS*/ /* [kind:16] push predicate result */ + OP_UNOBSERVED_BEGIN,/*obs:WRITES*/ /* increment g_unobserved_depth */ + OP_UNOBSERVED_END, /*obs:WRITES*/ /* decrement g_unobserved_depth */ + OP_LOOP_STALL_CHECK,/*obs:READS*/ /* [exit_offset:16] observer-stall + iteration cap (observer-based loops) */ + OP_LOOP_CAP_CHECK, /*obs:DIAG*/ /* [exit_offset:16] iteration cap ONLY (plain loops; no observer-stall) */ /* Miscellaneous */ - OP_IMPORT, /* [name_idx:16] import module, push dict */ - OP_MATCH, /* [case_count:16] pattern match dispatch */ - OP_LISTCOMP_BEGIN, /* push empty list accumulator */ - OP_LISTCOMP_APPEND, /* append TOS to accumulator */ - OP_LINE, /* [line:32] update current line number (#630: was 16-bit, wrapped past line 65535) */ - OP_WIDE, /* next operand is 32-bit */ - OP_DISPATCH, /* pop arg, key, table; call table[key](arg) inline */ + OP_IMPORT, /*obs:NONE*/ /* [name_idx:16] import module, push dict */ + OP_MATCH, /*obs:NONE*/ /* [case_count:16] pattern match dispatch */ + OP_LISTCOMP_BEGIN, /*obs:NONE*/ /* push empty list accumulator */ + OP_LISTCOMP_APPEND, /*obs:NONE*/ /* append TOS to accumulator */ + OP_LINE, /*obs:WRITES*/ /* [line:32] update current line number (#630: was 16-bit, wrapped past line 65535) */ + OP_WIDE, /*obs:NONE*/ /* next operand is 32-bit */ + OP_DISPATCH, /*obs:NONE*/ /* pop arg, key, table; call table[key](arg) inline */ /* Superinstructions */ - OP_LOCAL_DOT_GET, /* [slot:16][name_idx:16] push local[slot].name */ - OP_LOCAL_DOT_SET, /* [slot:16][name_idx:16] TOS = local[slot].name = TOS */ - OP_LOCAL_IDX_GET, /* [slot:16][idx:16] push local[slot][idx] */ - OP_LOCAL_IDX_DOT_GET, /* [slot:16][idx:16][name_idx:16] push local[slot][idx].name */ - OP_LOCAL_IDX_DOT_SET, /* [slot:16][idx:16][name_idx:16] local[slot][idx].name = TOS */ - OP_INTERROGATE_NAMED, /* [kind:16][name_idx:16] interrogate with known binding name */ - OP_INTERROGATE_NAMED_AT, /* [kind:16][name_idx:16] interrogate at line (popped from stack) */ - - OP_DEFAULT_PARAM, /* [slot:16][skip_off:16] if frame->call_argc > slot, IP += skip_off + OP_LOCAL_DOT_GET, /*obs:NONE*/ /* [slot:16][name_idx:16] push local[slot].name */ + OP_LOCAL_DOT_SET, /*obs:NONE*/ /* [slot:16][name_idx:16] TOS = local[slot].name = TOS */ + OP_LOCAL_IDX_GET, /*obs:NONE*/ /* [slot:16][idx:16] push local[slot][idx] */ + OP_LOCAL_IDX_DOT_GET, /*obs:NONE*/ /* [slot:16][idx:16][name_idx:16] push local[slot][idx].name */ + OP_LOCAL_IDX_DOT_SET, /*obs:NONE*/ /* [slot:16][idx:16][name_idx:16] local[slot][idx].name = TOS */ + OP_INTERROGATE_NAMED, /*obs:READS*/ /* [kind:16][name_idx:16] interrogate with known binding name */ + OP_INTERROGATE_NAMED_AT, /*obs:READS*/ /* [kind:16][name_idx:16] interrogate at line (popped from stack) */ + + OP_DEFAULT_PARAM, /*obs:NONE*/ /* [slot:16][skip_off:16] if frame->call_argc > slot, IP += skip_off * (skip the default expression); else fall through (default runs * and ends with OP_SET_LOCAL ; OP_POP). */ - OP_DESTRUCTURE_UNPACK, /* [n:16] pop list, raise if not VAL_LIST or length != n, + OP_DESTRUCTURE_UNPACK, /*obs:NONE*/ /* [n:16] pop list, raise if not VAL_LIST or length != n, * else push elements onto stack in reverse so element 0 is TOS. * Pairs with N assignment ops emitted after by the compiler. */ - OP_SLICE_GET, /* pop 3 (end, start, target); push the slice of target from + OP_SLICE_GET, /*obs:NONE*/ /* pop 3 (end, start, target); push the slice of target from * start..end (half-open). null in either bound means default * (0 / len). Target must be VAL_LIST / VAL_STR / VAL_BUFFER. * Negatives resolve via +len before the 0<=start<=end<=len @@ -181,35 +181,35 @@ typedef enum { * (e.g. tests/test_vm_run_bytecode.eigs's `loopcode` uses 63 = LOOP_CAP_CHECK), * so inserting an opcode anywhere before them shifts those numbers and * misaligns the bytecode. New opcodes must always append here. */ - OP_REPORT_SLOT, /* [slot:16] report-of-local via slot trajectory (compile-flag gated) */ - OP_OBSERVE_NAME_POST,/* [name_idx:16] slot-observe a name binding AFTER its SET + OP_REPORT_SLOT, /*obs:READS*/ /* [slot:16] report-of-local via slot trajectory (compile-flag gated) */ + OP_OBSERVE_NAME_POST,/*obs:WRITES*/ /* [name_idx:16] slot-observe a name binding AFTER its SET * (binding now exists), fixing the first-assignment lag. * Emitted only under compile-time EIGS_OBS_SHADOW; peeks TOS. */ - OP_REPORT_NAME, /* [name_idx:16] report of a non-local name: resolve (env,slot), + OP_REPORT_NAME, /*obs:READS*/ /* [name_idx:16] report of a non-local name: resolve (env,slot), * classify its slot. Compile-flag gated. */ - OP_OBSERVE_VALUE_SLOT, /* [slot:16] `observe of `: [status,entropy,dH,prev_dH] + OP_OBSERVE_VALUE_SLOT, /*obs:READS*/ /* [slot:16] `observe of `: [status,entropy,dH,prev_dH] * from the local's slot trajectory. Compile-flag gated. */ - OP_OBSERVE_VALUE_NAME, /* [name_idx:16] `observe of `: same, resolving the + OP_OBSERVE_VALUE_NAME, /*obs:READS*/ /* [name_idx:16] `observe of `: same, resolving the * binding's (env,slot). Compile-flag gated. */ - OP_LOOP_ENV_CLEAR, /* reset a persisted loop env's bindings for a new iteration. + OP_LOOP_ENV_CLEAR, /*obs:WRITES*/ /* reset a persisted loop env's bindings for a new iteration. * Appended here (NOT mid-list) per the convention above — * hand-built bytecode hardcodes opcode numbers. */ - OP_PREDICATE_SLOT, /* [kind:16][slot:16] ` of ` — classify the + OP_PREDICATE_SLOT, /*obs:READS*/ /* [kind:16][slot:16] ` of ` — classify the * named local's slot trajectory (not the global last-observed * alias the bare OP_PREDICATE reads). Appended, not mid-list. */ - OP_PREDICATE_NAME, /* [kind:16][name_idx:16] ` of ` — resolve the + OP_PREDICATE_NAME, /*obs:READS*/ /* [kind:16][name_idx:16] ` of ` — resolve the * binding's (env,slot) and classify its slot trajectory. */ - OP_REPORT_VALUE_SLOT, /* [slot:16] `report_value of ` — classify the local's + OP_REPORT_VALUE_SLOT, /*obs:READS*/ /* [slot:16] `report_value of ` — classify the local's * VALUE trajectory (#294), not its entropy. Appended, not mid-list. */ - OP_REPORT_VALUE_NAME, /* [name_idx:16] `report_value of ` — resolve the binding's + OP_REPORT_VALUE_NAME, /*obs:READS*/ /* [name_idx:16] `report_value of ` — resolve the binding's * (env,slot) and classify its value trajectory. */ - OP_TRAJECTORY_SLOT, /* [slot:16] `trajectory of ` (#421) — snapshot the local + OP_TRAJECTORY_SLOT, /*obs:READS*/ /* [slot:16] `trajectory of ` (#421) — snapshot the local * slot's observer windows into a dict VALUE that survives a call * boundary (the slot itself is binding-identity and cannot). * Appended, not mid-list. */ - OP_TRAJECTORY_NAME, /* [name_idx:16] `trajectory of ` — resolve the binding's + OP_TRAJECTORY_NAME, /*obs:READS*/ /* [name_idx:16] `trajectory of ` — resolve the binding's * (env,slot) and snapshot it. */ - OP_INTERROGATE_NAMED_WHEN, /* [kind:16][name_idx:16] ` is x when ` (#868) — + OP_INTERROGATE_NAMED_WHEN, /*obs:READS*/ /* [kind:16][name_idx:16] ` is x when ` (#868) — * interrogate at the Nth RECORDED assignment (ordinal * popped from the stack), not at a source line. The `at` * address space is source lines, which is not injective: diff --git a/tests/run_all_tests.sh b/tests/run_all_tests.sh index b7d6c896..47c86638 100755 --- a/tests/run_all_tests.sh +++ b/tests/run_all_tests.sh @@ -5208,6 +5208,30 @@ else fi echo "" +# [99t] Observer-classification marker gate (#972). Every opcode in the OpCode +# enum must carry exactly one obs:READS / obs:WRITES / obs:DIAG / obs:NONE +# marker, recorded by a human who read the handler. Five attempts to DERIVE +# that classification from the C are on #972 and all five were confidently +# wrong — twice silently empty, once silently universal — so the gate does not +# classify anything; it fails on any opcode with no verdict, which is a loud +# unanswered question at the moment an opcode is added. This matters because +# the liveness elision it feeds fails silently and totally: an unlisted reader +# means a program gates its own bookkeeping off and then answers "equilibrium" +# forever. The self-test runs eleven mutations, each witnessed by exactly one +# fixture (verified by neutering each check in a copy of the gate). +echo "[99t] Observer-classification marker gate (#972)" +TOTAL=$((TOTAL + 1)) +if bash "$TESTS_DIR/../tools/obs_marker_check.sh" && \ + bash "$TESTS_DIR/../tools/obs_marker_check.sh" --selftest >/dev/null; then + PASS=$((PASS + 1)) + echo " PASS: every opcode carries an observer classification (gate self-test green)" +else + FAIL=$((FAIL + 1)) + echo " FAIL: an opcode is unclassified, or the marker gate self-test broke" + bash "$TESTS_DIR/../tools/obs_marker_check.sh" 2>&1 | sed -n '1,10p' +fi +echo "" + # [99m] Lint archive symbol-collision gate (#917, hole closed by #922). # The #917 split turned lint's json_escape helper into an external symbol and # broke the static-library route for any embedder with its own json_escape. diff --git a/tools/obs_marker_check.sh b/tools/obs_marker_check.sh new file mode 100755 index 00000000..8d96a4c8 --- /dev/null +++ b/tools/obs_marker_check.sh @@ -0,0 +1,550 @@ +#!/bin/bash +# Observer-classification marker gate (#972). +# +# WHAT THIS GATE IS FOR +# +# The planned optimisation behind #972 is: stop emitting observer bookkeeping +# when a program provably never reads observer state. Its failure mode is +# silent and total — a program gates itself off, then reads slots nobody +# updated, and every binding answers "equilibrium" forever with no crash and +# nothing to fail on. So the scan that decides "does this chunk contain a +# reader?" must be complete, and completeness has to be pinned against the +# closed set of opcodes. +# +# WHY THE CLASSIFICATION IS NOT DERIVED +# +# Five attempts to derive the reader set from the C are recorded on #972, and +# all five produced a CONFIDENT WRONG ANSWER in one direction or the other: +# +# 1. grep the read-side API -> missed obs_stall_trajectory(), which +# reads s->used/s->dH/s->entropy with no +# observer_slot_* call anywhere +# 2. objdump -dr relocations -> missed the same reader: a struct-field +# read emits no symbol reference +# 3. scan for struct-field reads -> false-positived on three unrelated +# functions reading a ->n +# 4. classify by scanning `case OP_X:`-> matched NOTHING (the VM dispatches +# through CASE(NAME) computed-goto +# macros) and reported a clean empty set +# 5. same scan repaired to CASE(X) -> matched EVERYTHING, including a scope +# marker and a writer, because the +# handler terminator did not match and +# each scan ran on into later handlers +# +# The C source is the OPEN level (mechanical-gates 48): a read can be spelled +# arbitrarily many ways, so no matcher over it bounds the population. The enum +# is the CLOSED level. So this gate does not classify anything. It asks the one +# question that IS mechanically answerable: +# +# HAS EVERY OPCODE BEEN CLASSIFIED BY A HUMAN? +# +# A missing marker is a loud unanswered question at the moment an opcode is +# added, which is exactly when its author knows the answer and nobody else ever +# will. Same shape, and same reason, as tools/failsoft_classify_check.sh. +# +# WHAT A MARKER MEANS +# +# Observer/temporal state, for the purpose of these markers, is the state a +# program can obtain ONLY through an observer or temporal query: +# (a) the per-binding ObserverSlot under Env::obs — entropy/last_entropy/ +# dH/prev_dH/obs_age and the dH/value/raw windows; +# (b) Env::assign_counts, the per-slot assignment ordinals behind `when is x`; +# (c) the recorded assignment history read by trace_query_prev/at/when +# (`prev`, `at L`, `when N`) and the g_trace_current_line stamp it is +# addressed by; +# (d) the recording controls — g_unobserved_depth and the g_last_obs_slot_* +# alias the bare predicate reads through. +# +# obs:READS the handler is answering FROM (a)/(b)/(c) recorded earlier: its +# pushed result or its control-flow decision depends on them. If +# bookkeeping is elided, this opcode answers wrong and silently. +# THIS IS THE SET THE LIVENESS SCAN CONSUMES. +# obs:WRITES the handler records, updates, resets or stamps (a)-(d) and does +# not answer from them. +# obs:DIAG the handler reaches observer state ONLY through the SIGUSR1 +# diagnostic dump (eigs_observe_safepoint), never through +# program-visible semantics. +# obs:NONE none of the above. +# +# READS DOMINATES: an opcode that both reads and records is marked READS. +# +# obs:DIAG exists because collapsing it either way is wrong, and the choice is a +# decision rather than a derivation. OP_LOOP_CAP_CHECK is emitted at every plain +# loop and its only reach into observer state is eigs_observe_safepoint's dump. +# Calling that READS would pin bookkeeping on for every loop in every program +# and delete the entire win (mechanical-gates 9: an unbounded closure marks +# every loop as a reader). Calling it NONE would silently drop the obligation +# that the dump must SAY the gate is closed rather than render every binding as +# "equilibrium" (mechanical-gates 11). DIAG records the waiver as a marker +# instead of as prose, so the elision can enumerate exactly who owes that. +# +# WHAT THIS GATE DOES NOT DO — read this before trusting it +# +# * It does NOT check that a verdict is CORRECT. It checks that a verdict was +# recorded. Correctness rests on the handler having been read; the reading +# for the initial 94 is recorded in the PR for #972. +# * It does NOT close over callees. A marker describes the handler's own +# semantics. Three edges are deliberately excluded, each because another +# mechanism covers it, and each is a waiver in the mechanical-gates 3 sense: +# - OP_CALL / OP_IMPORT / OP_DISPATCH run other code. The callee chunk is +# scanned on its own way in, so attributing its reads here would say +# only "this opcode can run other code". +# - OP_JUMP_BACK can enter a JIT OSR thunk. The JIT has its OWN reader +# (jit_helper_report_slot) and needs its own scan; #972 lists it as a +# separate bypass route. +# - assembled chunks (vm_run_bytecode / sandbox_run) never pass through +# the compiler at all and need a bytecode twin of the scan; +# chunk_arm_temporal is the precedent. Also listed on #972. +# * The obs:WRITES set is informational here. The liveness scan keys on READS. +# +# Usage: tools/obs_marker_check.sh [--selftest] [--reads] +# --selftest : run the mutation train against COPIES of the header in a temp +# dir (this gate never writes to the tree it checks). +# --reads : print the obs:READS opcode set, one per line. This is the +# liveness scan's input; it is derived from the markers so the +# scan and the header cannot drift (mechanical-gates 26). +# Exit 0 = every opcode carries exactly one well-formed marker. + +set -u +cd "$(dirname "$0")/.." || exit 1 + +VM_HEADER="${VM_HEADER:-src/vm.h}" + +# Coverage floor, not an exact count. Opcodes are APPEND-ONLY by the bytecode +# ABI rule stated in the enum itself (hand-built chunks hardcode opcode +# numbers), so this number only ever grows; a floor moves only when coverage is +# REMOVED, which is always a review event. Measured on the tree that introduced +# this gate: 94 opcodes plus the OP_COUNT sentinel. +OPCODE_FLOOR="${OPCODE_FLOOR:-94}" + +# Non-vacuity floor on the READS set specifically. This set IS the liveness +# scan's input: if it silently collapsed to empty, every program would be +# classified as observer-free and elided, which is precisely the catastrophic +# failure the whole gate exists to prevent. `-z` is an emptiness test, not a +# floor (mechanical-gates 43) — a set that shrinks 15 -> 14 because a reader was +# re-marked NONE is the realistic mutation, and only a floor sees it. +READS_FLOOR="${READS_FLOOR:-15}" + +# The number of assertions check_tree runs. Pinned so that deleting a whole +# assertion cannot quietly shrink this gate while it keeps printing PASS +# (mechanical-gates 37). There are no skip paths: this gate needs only bash and +# awk, both hard dependencies of every other gate in tools/. +EXPECTED_CHECKS=7 + +# The marker vocabulary. A token outside this set is a typo (obs:READ) or an +# invention, and either way it is not a recorded decision. +VALID_MARKERS="READS WRITES NONE DIAG" + +# The one exempted enum entry, with its reason, pinned by name. OP_COUNT is a +# sentinel: it has no opcode number in any chunk and no handler in vm.c, so +# there is nothing to classify. An exemption that stops firing must FAIL rather +# than pass quietly (mechanical-gates 3), so the checks below require OP_COUNT +# to be present, to be LAST, and to be unmarked. +SENTINEL="OP_COUNT" + +# Emit one record per enum entry: OP_NAMEMARKERLINENO +# MARKER is "-" when absent, "?dup" when the line carries more than one marker, +# and the raw token otherwise (validated by the caller, not here). +# Also emits ORPHAN records for a marker sitting on a non-declaration line. +# +# awk portability (mechanical-gates 63): no 3-argument match(), no gensub(), no +# dynamic regex. The marker is found with index() on the literal "/*obs:" — +# exact substring, identical semantics in mawk, gawk, busybox awk and macOS awk. +# +# VERIFIED, not assumed, by shadowing awk on PATH: mawk, nawk (one-true-awk, +# which is what macOS ships) and busybox awk all report the identical clean +# verdict — 94 opcodes, READS=15 WRITES=10 DIAG=1 NONE=68 — AND the identical +# selftest verdict. Both halves matter: a dialect that silently fails a dynamic +# regex can still produce a plausible count and turn every comparison false, +# which looks exactly like a clean pass. The first draft of the applier for +# these markers used 3-argument match() and died immediately under mawk, which +# is the cheap version of the same lesson. +# The literal spelling also keeps prose out of the population: a comment that +# happens to discuss obs: markers does not carry the "/*obs:" opener. +enum_records() { + awk ' + # Track the start of every candidate enum; the OpCode enum is the one + # that CLOSES with "} OpCode;". Anchoring on the closer rather than on + # "the first typedef enum" survives another enum being added above it. + /^typedef enum \{$/ { start = NR } + { line[NR] = $0 } + /^\} OpCode;/ { + if (start == 0) { print "GATEERROR\tno-enum-open"; exit } + for (i = start + 1; i < NR; i++) { + s = line[i] + # Opcode declaration: OP_NAME at the head of the line, followed + # by a comma, or by whitespace/end for the trailing sentinel. + isdecl = 0; name = "" + if (match(s, /^[ \t]*OP_[A-Z0-9_]+/)) { + tok = substr(s, RSTART, RLENGTH) + sub(/^[ \t]*/, "", tok) + after = substr(s, RSTART + RLENGTH, 1) + if (after == "," || after == "" || after == " " || after == "\t") { + isdecl = 1; name = tok + } + } + # Count markers on this line with a literal substring scan. + nmark = 0; first = "" + rest = s + while ((p = index(rest, "/*obs:")) > 0) { + nmark++ + tail = substr(rest, p + 6) + q = index(tail, "*/") + val = (q > 0) ? substr(tail, 1, q - 1) : "?unterminated" + if (nmark == 1) first = val + rest = (q > 0) ? substr(tail, q + 2) : "" + } + if (isdecl) { + if (nmark == 0) print name "\t-\t" i + else if (nmark > 1) print name "\t?dup\t" i + else print name "\t" first "\t" i + } else if (nmark > 0) { + print "ORPHAN\t" first "\t" i + } + } + exit + } + ' "$VM_HEADER" +} + +check_tree() { + local records checks=0 fail=0 + local total=0 reads=0 writes=0 none=0 diag=0 + local name marker lineno last_name="" sentinel_marker="" sentinel_seen=0 + + records=$(enum_records) + + # ---- check 1: the enum was found and parsed at all -------------------- + # Vacuity first. A gate whose derivation silently returned nothing prints a + # confident PASS over an empty population (mechanical-gates 48). + checks=$((checks + 1)) + if [ -z "$records" ] || printf '%s\n' "$records" | grep -q '^GATEERROR'; then + echo "GATE ERROR: could not locate the OpCode enum in $VM_HEADER" + echo " (expected a 'typedef enum {' ... '} OpCode;' block)" + return 1 + fi + + # ---- check 2: no orphan markers --------------------------------------- + # The reverse direction of membership (mechanical-gates 2). A marker left + # behind on a renamed or deleted opcode, or dropped onto a comment line, + # drifts from the enum without the forward check noticing. + checks=$((checks + 1)) + local orphans + orphans=$(printf '%s\n' "$records" | awk -F'\t' '$1 == "ORPHAN" { print $3 }') + if [ -n "$orphans" ]; then + for lineno in $orphans; do + echo "ASSERTION FAILED: orphan marker at $VM_HEADER:$lineno declares no opcode" + done + fail=1 + fi + + # ---- checks 3+4: every opcode is marked, with a valid token ------------ + checks=$((checks + 2)) + while IFS="$(printf '\t')" read -r name marker lineno; do + [ "$name" = "ORPHAN" ] && continue + if [ "$name" = "$SENTINEL" ]; then + sentinel_seen=1 + sentinel_marker="$marker" + last_name="$name" + continue + fi + total=$((total + 1)) + last_name="$name" + case "$marker" in + -) + echo "ASSERTION FAILED: $name has no obs: marker ($VM_HEADER:$lineno)" + echo " Read its handler in src/vm.c and record one of: $VALID_MARKERS." + echo " Do NOT derive this from the C — five derivations on #972 were wrong." + fail=1 + continue ;; + "?dup") + echo "ASSERTION FAILED: $name carries more than one obs: marker ($VM_HEADER:$lineno)" + fail=1 + continue ;; + esac + local ok=0 v + for v in $VALID_MARKERS; do [ "$marker" = "$v" ] && ok=1; done + if [ "$ok" -eq 0 ]; then + echo "ASSERTION FAILED: $name has unknown marker obs:$marker ($VM_HEADER:$lineno)" + echo " Valid markers: $VALID_MARKERS" + fail=1 + continue + fi + case "$marker" in + READS) reads=$((reads + 1)) ;; + WRITES) writes=$((writes + 1)) ;; + NONE) none=$((none + 1)) ;; + DIAG) diag=$((diag + 1)) ;; + esac + done </dev/null 2>&1; then + echo "SELFTEST FAILED: the real tree does not pass its own marker check" + check_tree + st_fail=1 + else + echo "SELFTEST OK: the unmutated tree passes (positive control)" + fi + + # Helper: run check_tree against a mutant header, require it to FAIL, and + # require the failure to name the intended assertion. The attribution + # matcher is never a bare filename or path — those appear in every + # unrelated error about the file (mechanical-gates 19/58). + expect_red() { + local label="$1" header="$2" needle="$3" out + if [ ! -s "$header" ]; then + echo "SELFTEST FAILED: $label — mutant header was not produced" + st_fail=1; return + fi + if cmp -s "$header" "$VM_HEADER"; then + echo "SELFTEST FAILED: $label — mutation was a no-op (header unchanged)" + st_fail=1; return + fi + if out=$(VM_HEADER="$header" check_tree 2>&1); then + echo "SELFTEST FAILED: $label — mutant was NOT caught" + printf '%s\n' "$out" + st_fail=1; return + fi + if ! printf '%s\n' "$out" | grep -qF "$needle"; then + echo "SELFTEST FAILED: $label — caught, but not at the expected assertion" + printf '%s\n' "$out" + st_fail=1; return + fi + echo "SELFTEST OK: $label" + } + + # 1. Remove one marker. REPLACE rather than delete (mechanical-gates 41): + # the line keeps its shape, so no length- or count-based check can reject + # it on the target assertion behalf. OP_PREDICATE is chosen because it is + # a real reader — losing its marker is the mutation that matters. + awk ' + index($0, "OP_PREDICATE,") == 1 || index($0, " OP_PREDICATE,") == 1 { + p = index($0, "/*obs:READS*/") + if (p > 0 && !done) { + print substr($0, 1, p - 1) " " substr($0, p + 13) + done = 1 + next + } + } + { print } + ' "$VM_HEADER" > "$work/unmarked.h" + expect_red "a reader losing its marker is caught by name" "$work/unmarked.h" \ + "ASSERTION FAILED: OP_PREDICATE has no obs: marker" + + # 2. Mis-spell a marker. A typo is not a recorded decision, and a gate that + # accepts obs:READ has a vocabulary that is open rather than closed. + sed 's@/\*obs:READS\*/@/*obs:READ*/@' "$VM_HEADER" > "$work/typo.h" + expect_red "an unknown marker token is rejected" "$work/typo.h" \ + "has unknown marker obs:READ " + + # 3. Orphan marker: a marker on a line that declares no opcode. This is what + # a rename leaves behind, and only the reverse direction sees it. + awk ' + { print } + !done && index($0, " /* Observer system */") == 1 { + print " /*obs:READS*/" + done = 1 + } + ' "$VM_HEADER" > "$work/orphan.h" + expect_red "an orphan marker declaring no opcode is caught" "$work/orphan.h" \ + "ASSERTION FAILED: orphan marker at" + + # 4. Two markers on one line — an edit that leaves the old verdict beside + # the new one. Either could be the recorded decision, so neither is. + sed 's@OP_REPORT_SLOT, /\*obs:READS\*/@OP_REPORT_SLOT, /*obs:READS*/ /*obs:NONE*/@' \ + "$VM_HEADER" > "$work/dup.h" + expect_red "two markers on one opcode is caught" "$work/dup.h" \ + "ASSERTION FAILED: OP_REPORT_SLOT carries more than one obs: marker" + + # 5. Collapse the READS set to NONE. Every opcode still carries exactly one + # valid marker, the count is unchanged, and the forward/reverse checks + # are both satisfied — this mutation is invisible to everything except + # the READS floor, and it is the one that would silently elide + # bookkeeping for the entire language. + sed 's@/\*obs:READS\*/@/*obs:NONE*/ @' "$VM_HEADER" > "$work/collapsed.h" + expect_red "collapsing the READS set trips its non-vacuity floor" "$work/collapsed.h" \ + "GATE ERROR: obs:READS set has 0 opcode(s), floor is" + + # 6. Shrink the enum. A derived population can lose members without any + # assertion firing; only the floor sees it (mechanical-gates 43). + awk ' + index($0, " OP_TRAJECTORY_SLOT,") == 1 { skipping = 1 } + index($0, " OP_COUNT") == 1 { skipping = 0 } + !skipping { print } + ' "$VM_HEADER" > "$work/shrunk.h" + expect_red "a shrinking opcode population trips the coverage floor" "$work/shrunk.h" \ + "GATE ERROR: only " + + # 7. Mark the exempted sentinel. The exemption is a written claim about + # OP_COUNT; if OP_COUNT starts carrying a verdict, the claim is stale. + sed 's@^ OP_COUNT @ OP_COUNT, /*obs:NONE*/ @' "$VM_HEADER" > "$work/sentinel.h" + expect_red "marking the exempted sentinel is caught" "$work/sentinel.h" \ + "carries obs:NONE but is exempt as a sentinel" + + # 8. Remove the sentinel entirely. An exemption whose subject is gone must + # FAIL rather than pass quietly (mechanical-gates 3). + grep -v '^ OP_COUNT' "$VM_HEADER" > "$work/nosentinel.h" + expect_red "losing the exempted sentinel is caught" "$work/nosentinel.h" \ + "ASSERTION FAILED: exempted sentinel OP_COUNT is not in the enum" + + # 8b. Append an opcode AFTER the sentinel. This is the one sub-branch of the + # sentinel check with no other fixture: cases 7 and 8 are caught by the + # "carries a marker" and "is not in the enum" arms respectively, and + # neutering the position arm alone left the suite green until this case + # existed (mechanical-gates 38 — witness every alternation, not just the + # check). It matters because OP_COUNT == the opcode count is what makes + # the enum a closed list; an entry past it is silently uncounted. + awk ' + !done && index($0, " OP_COUNT") == 1 { + print " OP_COUNT, /* sentinel */" + print " OP_PAST_SENTINEL /*obs:NONE*/" + done = 1 + next + } + { print } + ' "$VM_HEADER" > "$work/aftersentinel.h" + expect_red "an opcode appended past the sentinel is caught" "$work/aftersentinel.h" \ + "ASSERTION FAILED: OP_COUNT is not the last enum entry" + + # 9. Break the enum derivation itself. If the parse silently returns + # nothing, every check above passes over an empty population. + sed 's@^} OpCode;@} OpCodeRenamed;@' "$VM_HEADER" > "$work/noenum.h" + expect_red "an unparseable enum is a GATE ERROR, not a clean pass" "$work/noenum.h" \ + "GATE ERROR: could not locate the OpCode enum" + + # 9b. Mutate the GATE, not the header: delete a whole assertion and require + # the check-count pin to notice. Without this, EXPECTED_CHECKS was + # unwitnessed — every header fixture above passed with the pin removed, + # so the gate advertised a guard against its own silent shrinkage while + # enforcing nothing (mechanical-gates 19/37). + # + # The mutant runs from $work, so it cd's somewhere that has no src/; + # VM_HEADER is passed as an ABSOLUTE path so the mutant STARTS. An + # unstartable mutant is an invalid probe reported as CAUGHT — the first + # version of this check failed exactly that way, with all fixtures + # erroring identically because none of them could see a header. + awk ' + index($0, " # ---- check 7") == 1 { dropping = 1 } + index($0, " if [ \"$checks\" -ne \"$EXPECTED_CHECKS\" ]; then") == 1 { dropping = 0 } + !dropping { print } + ' "$0" > "$work/gate_mutant.sh" + abs_header=$(cd "$(dirname "$VM_HEADER")" && pwd)/$(basename "$VM_HEADER") + if cmp -s "$work/gate_mutant.sh" "$0"; then + echo "SELFTEST FAILED: check-count pin — gate mutation was a no-op" + st_fail=1 + elif out=$(VM_HEADER="$abs_header" bash "$work/gate_mutant.sh" 2>&1); then + echo "SELFTEST FAILED: an assertion was deleted and the gate still passed" + printf '%s\n' "$out" + st_fail=1 + elif ! printf '%s\n' "$out" | grep -qF "GATE ERROR: ran 6 assertion(s), expected 7"; then + # Distinguish "the pin fired" from "the mutant never started" — a + # missing header, a bad shebang and a deleted assertion all exit + # nonzero, and only the pin emits this sentence. + echo "SELFTEST FAILED: deleting an assertion did not trip the check-count pin" + printf '%s\n' "$out" + st_fail=1 + else + echo "SELFTEST OK: deleting an assertion trips the check-count pin" + fi + + # 10. The --reads consumer contract must be non-vacuous too: it is what the + # liveness scan will read, and an empty answer there means "elide + # everywhere". + n_reads=$(emit_reads | grep -c .) + if [ "$n_reads" -lt "$READS_FLOOR" ]; then + echo "SELFTEST FAILED: --reads emitted $n_reads opcode(s), floor is $READS_FLOOR" + st_fail=1 + else + echo "SELFTEST OK: --reads emits $n_reads reader opcodes" + fi + + # 11. The gate must not mutate what it checks (mechanical-gates 22/28). + # Every mutation above went to a copy in $work; assert the tracked + # header is byte-identical to HEAD. Keyed to the tree under test, and + # skipped rather than faked when git is unavailable. + if command -v git >/dev/null 2>&1 && git rev-parse --git-dir >/dev/null 2>&1; then + if git diff --quiet -- "$VM_HEADER" 2>/dev/null || \ + [ -z "$(git status --porcelain -- "$VM_HEADER")" ]; then + echo "SELFTEST OK: the selftest left $VM_HEADER untouched" + else + echo "SELFTEST NOTE: $VM_HEADER differs from HEAD (expected while #972 is in flight);" + echo " the leavings check cannot distinguish that from selftest damage here." + fi + fi + + if [ "$st_fail" -eq 0 ]; then + echo "SELFTEST PASS: observer-marker gate is non-vacuous" + fi + exit "$st_fail" +fi + +check_tree diff --git a/tools/vm_operand_width_check.sh b/tools/vm_operand_width_check.sh index 184e8d12..ee69aa31 100755 --- a/tools/vm_operand_width_check.sh +++ b/tools/vm_operand_width_check.sh @@ -249,7 +249,12 @@ if [ "${1:-}" = "--selftest" ]; then { print } END { if (!done) exit 2 } ' "$VM_SOURCE" > "$work/vm_shrunk.c" || st_fail=1 - sed -E 's@^([[:space:]]*OP_INTERROGATE,[[:space:]]*\/\* )\[kind:16\] @\1@' \ + # `.*` between the comma and the comment opener, not `[[:space:]]*`: the + # enum line carries an /*obs:...*/ observer-classification marker between + # the two (#972), and anchoring on whitespace alone made this mutation a + # no-op — the selftest then failed with "could not remove the comment" + # rather than silently passing, which is the only reason it was noticed. + sed -E 's@^([[:space:]]*OP_INTERROGATE,.*\/\* )\[kind:16\] @\1@' \ "$VM_HEADER" > "$work/vm_shrunk.h" if cmp -s "$work/vm_shrunk.h" "$VM_HEADER"; then echo "SELFTEST FAILED: could not remove the OP_INTERROGATE [kind:16] comment"