Observer: add a compile-time liveness gate so bookkeeping isn't emitted when never read (consumers can drop `unobserved:`)

[InauguralPhysicist]

Summary

The observer — the language's defining feature — is emitted unconditionally on every assignment, with no compile-time liveness gate. Performance-sensitive consumers therefore disable it wholesale with unobserved: (EigenMiniSat ships tseitin_ladder_unobserved.eigs; measured ~88% of runtime / 8.6× on a CDCL solver, EigenScript#915). When every serious consumer switches off the moat to be usable, that avoidance is the loudest architectural signal in the tree.

This is the Rank-3 finding of the 2026-08-18 whole-language architecture review.

The gap (verified)

• emit_assign_for_tos emits OBSERVE_ASSIGN(_LOCAL) after every assignment, unconditionally — src/compiler.c:1804-1808.
• The scalar path runs entropy_of_num (two log2 + a divide) then observer_slot_record_value (a divide + two ring-buffer pushes) on every write — src/eigenscript.c:290-292,523-532,583-587. The comment states it is updated eagerly "no dirty/lazy step" (eigenscript.c:399-403).
• The precedent already exists one subsystem over. The temporal tape (prev/at/state_at) is gated by g_trace_hist, armed only when the program contains a temporal query — src/trace.c:294-321, documented in .claude/rules/c-runtime-memory.md. The observer entropy/dH channel has no equivalent.
• Half the machinery is already there: scan_for_interrogated (src/compiler.c:1261-1268) whole-program-walks the AST — but collects only AST_INTERROGATE names (not AST_PREDICATE/report/classify) and uses them merely to force the env-binding path (:1754), never to gate observation OFF.

Note: the per-assignment container-walk cost was already fixed to O(direct children) in #685 (4.93s→0.06s at N=8000); this issue is not about that. It's about paying for measurements nobody reads.

Proposed fix

Extend scan_for_interrogated (compiler.c:1261) to also collect AST_PREDICATE and report/classify targets, producing a per-unit "is any observer state ever read?" answer. When a binding's observer state is provably never read, don't emit OBSERVE_ASSIGN(_LOCAL) at all (compiler.c:1804) — no opcode, no dispatch, no entropy_of_num, no ring pushes. Same mechanism the temporal tape already uses safely.

Does it touch identity? No.

The crux answer from the review: "always-on, per-assignment, eager" is an early implementation default, not a founding requirement. The thesis requires the observer be available (a binding can locate itself by its trajectory), not that entropy be computed on every write regardless of whether anyone looks. A program that reads observer state gets byte-identical behavior; one that never reads it pays nothing. The moat stays semantically always-on and on-by-default — and consumers can delete unobserved: wholesale, stopping the observer-disabled variants that are the avoidance signal, without opt-in and without weakening "observer with no outside."

Leverage: high. Class: REALIGN.

[InauguralPhysicist]

Completeness map for the gate, derived mechanically

Before writing the gate I enumerated every reader of observer state from the source rather than by hand, because mechanical-gates §9 was bought on this exact issue: a first attempt scanned each dispatch body for reader calls and was blind to OP_LOOP_STALL_CHECK, which reaches the observer one level down through obs_stall_trajectory().

Opcodes that read observer state — 12:

opcode	reaches
LOOP_STALL_CHECK	obs_stall_trajectory, converged, diverging, equilibrium, improving, oscillating, report, stable
PREDICATE, PREDICATE_NAME, PREDICATE_SLOT	vm_slot_predicate
REPORT_NAME, REPORT_SLOT	observer_slot_report
REPORT_VALUE_NAME, REPORT_VALUE_SLOT	observer_slot_report_value
TRAJECTORY_NAME, TRAJECTORY_SLOT	observer_slot_trajectory
OBSERVE_VALUE_NAME, OBSERVE_VALUE_SLOT	observer_slot_report

Plus a reader outside the opcode table: eigs_loop_stall_step (src/vm.c:1805) — the JIT's twin of LOOP_STALL_CHECK. A scan over CASE(...) bodies alone misses it. This is §9's "close over callees" in the flesh.

Three bypass routes the proposal does not yet mention

1. Assembled chunks bypass any source scan. chunk_arm_temporal (src/chunk.c:915) exists precisely because of this for the temporal tape (#831): a descriptor assembled by sandbox_run / vm_run_bytecode never went through the compiler, so its own prev of reads answered null unless the host program's text happened to contain a temporal query. A source-only observer gate has the identical hole, and needs the identical bytecode twin — walking the verified code stream and arming on the 12 opcodes above. iLambdaAi's grader runs generated code through sandbox_run, so this is the consumer that would hit it first.

2. The JIT path. Per above — the gate has to be visible to eigs_loop_stall_step, not only to the interpreter's CASE.

3. Diagnostic readers, which are mechanical-gates §11. Four things read observer state without the program asking:

reader	armed
obs_dump_scope (vm.c:187) — SIGUSR1 live dump	asynchronously — unknowable at compile time
show_trajectory (step.c:139) — the stepper	at startup
handle_variables (eigsdap.c:306) — DAP	at startup
tape_classify_at (tape_read.c:304) — replay	at startup

The startup-armed three can force the gate open like g_trace_hist does. SIGUSR1 cannot, and that is exactly the §11 failure this issue's own history produced: gating bookkeeping off left the dump printing every binding as equilibrium with an empty trajectory — reading as everything is settled when the truth is nothing was ever recorded. So the dump must state that the gate is closed and name what reopens it, rather than rendering absence as rest.

Local ceiling, measured

n=5 each on this box, a 400k-iteration assignment loop:

observed:    0.16 0.16 0.15 0.16 0.15   (median 0.155s)
unobserved:  0.04 0.03 0.03 0.04 0.04   (median 0.035s)

~4.4× — the shape-specific ceiling here, consistent with the 8.5× reported on the CDCL workload. A correct gate captures some fraction; the full n=5-vs-n=5 on the real consumer workload is still owed per #915.

Implementation order

The safety net before the thing that needs it: a drift gate proving every observer-reading opcode and out-of-table reader is covered by both scans (source and bytecode) should land before the elision, or the first missed reader is a silent wrong answer rather than a failure — a binding certifying converged off state nobody ever recorded.

Not started yet; recording the map so the build starts from facts rather than from the opcode list someone remembers.

[InauguralPhysicist]

Correction to the map above: the scan cannot be derived from the C at all

I tried to build the completeness gate first — pin the set of observer readers before the elision ships, so a missed one is a red line rather than a silent wrong answer. I could not make its derivation trustworthy, so I am not shipping it, and the reason is worth more than the tool would have been.

Three derivations, all inadequate

1. Grep the read-side API. Misses readers that touch the struct directly. obs_stall_trajectory (src/vm.c:47) is the live example:

static inline int obs_stall_trajectory(double *dH, double *ent) {
    const ObserverSlot *s = env_obs_slot(g_last_obs_slot_env, g_last_obs_slot_idx);
    if (s && s->used) { *dH = s->dH; *ent = s->entropy; return 1; }

No observer_slot_* call anywhere — it reads ->used, ->dH, ->entropy. This is the same reader the first attempt at #972 already missed once, and it is invisible to an API grep for a second, different reason.

2. Ask the compiler (objdump -dr over vm.c, i.e. the authoritative anchor mechanical-gates §1 prescribes). Clean output — exactly four functions reference the read symbols:

jit_helper_report_slot   obs_dump_scope   vm_run_ex   vm_slot_predicate

But obs_stall_trajectory is not among them, for the same reason: relocations only see symbol references, and a direct field read produces none. The authoritative tool is authoritative about its own invariant, which is narrower than the one needed here (§21).

3. Scan for ObserverSlot field reads. Catches obs_stall_trajectory, and immediately produces false positives — task_virtual_now, sched_wake_sleepers and task_request_sleep all read a ->n off unrelated pointers. Type inference by regex is not a foundation for a gate whose failure mode is a confident wrong answer.

What this means for the implementation

The scan must be AST/opcode-level, not C-level. The right question is "can this program EXPRESS a read?" — AST_INTERROGATE, AST_PREDICATE, report, classify, and the implicit loop while not converged stall check — because that set is closed and enumerable from the grammar. The C-side enumeration is then useful only as a cross-check on the opcode list, not as the source of truth.

And the cross-check still has to be written by hand-reading, because none of the three mechanical derivations above covers the population. That is a real cost of this issue and should be priced in: the completeness argument is a review artifact, not a generated one.

Unchanged from the map above

The three bypass routes still stand and are the load-bearing part: assembled chunks need a bytecode twin of the scan (chunk_arm_temporal is the precedent), the JIT has its own reader, and the SIGUSR1 dump cannot be known at compile time so it must say the gate is closed rather than render every binding as equilibrium.

Local ceiling remains ~4.4× (n=5, measured above).

[InauguralPhysicist]

The scan does not need to CLASSIFY readers — it needs to detect an UNCLASSIFIED opcode

The last comment concluded the completeness scan "cannot be derived from the C at all", after three inadequate derivations. I tried to build it at the level that comment prescribes (the grammar) and hit the same wall twice more — which turned out to be the useful result, because it points at a different gate design that sidesteps the derivation entirely.

Two more failed C-level derivations, mine, ten minutes apart

4. Classify each opcode by scanning its VM handler for observer reads. Written against case OP_X: — output: nothing at all reads the observer. The VM dispatches through CASE(NAME) macros for computed goto, so the scanner matched no handler and reported a clean, confident, empty answer.

5. Same scan, repaired to CASE(X). Output: everything reads the observer, including OP_UNOBSERVED_BEGIN and OP_OBSERVE_ASSIGN, which are a scope marker and a writer. The handler terminator did not match, so each scan ran past its own handler into later ones and found a read eventually.

So that is five derivations and five wrong answers, in two directions — silently empty and silently universal. The previous comment's conclusion is not pessimism; it is the measured behaviour of this surface.

The one thing that IS mechanically solid

$ grep -rn "emit_op[a-z0-9_]*(.*OP_\(INTERROGATE\|PREDICATE\|LOOP_STALL_CHECK\)" src/*.c | grep -v "^src/compiler.c" | wc -l
0

compiler.c is the sole emitter of every reader opcode. chunk.c's references are the disassembler/verifier, vm.c's are dispatch, jit.c's are compilation. And the opcode enum is a closed list — 95 entries in src/vm.h.

That pair is enough, and it is enough without anyone classifying anything.

The design that follows

The gate should not answer "which opcodes read observer state". It should answer "has every opcode been classified by a human?" — exactly the shape tools/failsoft_classify_check.sh uses for the fail-soft population, and for the same reason: the distinction is not derivable from the code, only from reading it, so the gate decides nothing and instead requires that a decision was recorded.

Concretely:

• a marker per opcode in src/vm.h (obs:READS / obs:WRITES / obs:NONE), reviewed once;
• the gate enumerates the enum (mechanical, closed, cannot silently shrink — pin the count at 95) and goes red on any opcode with no marker;
• the liveness scan reads the obs:READS set from those markers, so scan and reality cannot drift;
• a new opcode is a red line at the moment it is added, which is precisely when the author knows the answer and nobody else ever will.

This inverts the failure mode. Five derivations produced a confident wrong answer; a missing marker produces a loud unanswered question. For a gate whose failure mode is "a program gates itself off and then reads slots nobody updated, returning equilibrium forever with no crash", that inversion is the whole ballgame.

The three bypass routes from the map above are unaffected and still need their own handling: assembled chunks need a bytecode twin of the scan (chunk_arm_temporal is the precedent), the JIT has its own reader (jit_helper_report_slot), and the SIGUSR1 dump must say the gate is closed rather than render every binding as equilibrium.

Scope note

I am not shipping this in the current session — the elision itself touches the compiler, the VM, the JIT and the dump, and it is a perf change on a correctness-sensitive path that deserves its own run with a measured before/after (the 4.4x local ceiling from #915 is the bar). What this comment removes is the blocker: the completeness argument no longer requires a derivation nobody could make trustworthy.

[InauguralPhysicist]

The marker gate is landed (PR #1024). The blocker this issue named is now closed; the elision is not.

Markers only, no elision, no runtime change. tools/obs_marker_check.sh, suite section [99t].

The population was 94, not 95

Re-measured before scoping, per the rule that an issue's count is an instrument reading rather than a specification. The previous comment said "95 entries in src/vm.h" — that is the enum's entry count and it includes OP_COUNT, which is a sentinel with no opcode number and no handler. 94 opcodes are markable; OP_COUNT is an explicit pinned exemption, and the gate fails if it ever gains a marker, moves off the end, or disappears — an exemption that stops firing has to go red, not quiet.

All 94 read by hand

READS=15 WRITES=10 DIAG=1 NONE=68

obs:READS (15) — the set the liveness scan consumes:
REPORT_SLOT, REPORT_NAME, REPORT_VALUE_SLOT, REPORT_VALUE_NAME, TRAJECTORY_SLOT, TRAJECTORY_NAME, OBSERVE_VALUE_SLOT, OBSERVE_VALUE_NAME, PREDICATE, PREDICATE_SLOT, PREDICATE_NAME, INTERROGATE_NAMED, INTERROGATE_NAMED_AT, INTERROGATE_NAMED_WHEN, LOOP_STALL_CHECK.

obs:WRITES (10) — SET_LOCAL, SET_NAME, SET_NAME_LOCAL, SET_FN_NAME_LOCAL, OBSERVE_ASSIGN_LOCAL, OBSERVE_NAME_POST, LOOP_ENV_CLEAR, UNOBSERVED_BEGIN, UNOBSERVED_END, LINE.

obs:DIAG (1) — LOOP_CAP_CHECK.

Three findings that only reading produces

1. OP_OBSERVE_ASSIGN is a no-op. Its entire handler is ip += 2; DISPATCH();. The slot model observes at OP_OBSERVE_NAME_POST, after the SET. The opcode named "observe assign" touches no observer state at all — and derivation 5 marked it a reader.

2. The bare OP_INTERROGATE reads no observer state. when/where/why/how on a value operand return the literal constants 0, 0, 0, 1 — since #262 Step E, observer state is binding-keyed and a bare value has no binding. The named forms (INTERROGATE_NAMED/_AT/_WHEN) are the readers.

3. LOOP_CAP_CHECK forced a fourth marker. It is emitted at every plain loop, and its only reach into observer state is eigs_observe_safepoint's SIGUSR1 dump. Marking it READS pins bookkeeping on for every loop in every program and deletes the entire win — that is the unbounded-closure failure this issue already hit once. Marking it NONE silently drops the obligation that the dump must say the gate is closed rather than render every binding as equilibrium. obs:DIAG records that waiver as a marker the elision can enumerate instead of as prose nobody re-reads. That is a deviation from the three markers the previous comment specified, made deliberately.

The state the markers are about

Written into the gate header so the next reader does not re-derive it: (a) the ObserverSlot under Env::obs; (b) Env::assign_counts, the ordinals behind when is x; (c) the recorded assignment history behind prev / at L / when N, and the g_trace_current_line stamp addressing it; (d) the recording controls g_unobserved_depth and g_last_obs_slot_*.

READS means the handler answers from (a)/(b)/(c) recorded earlier. Reading only the recording GATE is not a read — TRY_BEGIN saves g_unobserved_depth into its handler record and is obs:NONE for that reason, because classifying it READS would pin bookkeeping on for every program with a try block.

What the gate proves, and what it does not

It proves a verdict was recorded. It cannot prove the verdict is right — that rests on the handlers having been read, which is what #1024 is. It stops at each opcode's own handler; OP_CALL/OP_IMPORT/OP_DISPATCH, OP_JUMP_BACK's JIT OSR entry, and assembled chunks are excluded by name, each with the mechanism that covers it instead.

Validation: eleven mutations, each verified witnessed by exactly one fixture by neutering every check in a copy of the gate. That verification found two guards enforcing nothing (the "OP_COUNT is not last" branch and the check-count pin both survived being deleted) and both now have their own fixture. Verified non-vacuous under mawk, nawk and busybox awk — clean verdict and planted-fault verdict identical in all three.

Gates: release 4070/4070, ASan+UBSan detect_leaks=1 4059/4059 leak tally 0, CI 18/18.

Still open on this issue

1. The elision itself — compiler, VM, JIT and the SIGUSR1 dump, with a measured n=5 before/after against Observer entropy is computed on EVERY assignment regardless of use: 88% of runtime / 8.50x ceiling on a consumer that uses no observer features #915's ceiling.
2. Assembled chunks (vm_run_bytecode / sandbox_run) need a bytecode twin of the scan; chunk_arm_temporal is the precedent.
3. The JIT's own reader (jit_helper_report_slot) needs its own scan.
4. The SIGUSR1 dump must say the gate is closed rather than render every binding as equilibrium — the obligation obs:DIAG now makes enumerable.