Docs: agent handover and AOT correctness inventory (2026-08-17)

docs/AGENT-HANDOVER-2026-08-17.md

@@ -0,0 +1,193 @@
+# Agent handover — 2026-08-17
+
+Written after a session that started by re-establishing ground truth on a tree that had moved
+3,069 commits since the previous handover. Every number below was measured in
+`php-compiler:22.04-dev`; nothing is quoted from memory or from a prior document.
+
+Where a claim is **not** verified, it says so. Please keep that habit — two of the findings here
+exist only because an earlier claim in `docs/roadmap/RELEASE-PLAN.md` was believed rather than
+re-checked.
+
+---
+
+## 1. The headline: the compiler could not produce a working binary
+
+`script/aot-smoke.sh` on master at `8084f14996`:
+
+```
+aot-smoke: 0 passed, 8 failed
+```
+
+Every one of the eight trivial programs — `echo`, arithmetic, concat, function call, branch, loop,
+array, class — failed at **link**, identically:
+
+```
+/opt/llvm9/ld: loop.bin.o: in function `phpc_gc_collect_cycles_impl':
+main:(.text+0x47afe): undefined reference to `memset.1'
+```
+
+### Cause
+
+`GcCollectCyclesRuntime::ensureExternal()` called `module->addFunction()` whenever
+`lookupFunction()` threw, without first checking `getNamedFunction()`:
+
+```php
+try {
+    $context->lookupFunction($name);
+} catch (\Throwable $e) {
+    $fn = $context->module->addFunction($name, $ft);   // no getNamedFunction() check
+    $context->registerFunction($name, $fn);
+}
+```
+
+`LibcExtern` adds `memset` to the module **and gives it a body** via `implementMemsetBody()`, but
+does not always leave it in the context registry. So `lookupFunction()` throws while the symbol
+already exists. `addFunction()` on an existing name does not fail — LLVM silently renames the
+second one to `memset.1`, which carries no body.
+
+Fixed in **#31894 (merged)** by reusing the existing declaration, matching the
+`getNamedFunction()`-first pattern `LibcExtern` already used. Verified `0/8 → 8/8`.
+
+### Two "separate" bugs that were this one wearing a costume
+
+Both now match Zend exactly:
+
+| probe | before | after |
+|---|---|---|
+| `var_dump(7)` | abort, rc=134 | `int(7)` |
+| `(new Exception)->getMessage()` | segfault | `msg=[]` |
+
+Anything diagnosed against a tree where AOT cannot link is measuring the link failure, not the
+feature. Run `script/aot-smoke.sh` **first**, every time — that is exactly what it is for, and its
+own failure message says so.
+
+---
+
+## 2. Why nobody noticed: the gate was red for an unrelated reason
+
+`compiler-gate.yml` had failed **19 of its last 20 master pushes** — and never on the checks it
+exists to run. It died during setup:
+
+```
+ERROR: failed to apply php-cfg-bare-variable-read-stmt.patch
+error: corrupt patch at line 47
+```
+
+A hunk header in `patches/php-cfg-bare-variable-read-stmt.patch` declared 24 new lines over a body
+containing 23 (3 context + 17 added + 3 context). `git apply` reads to the declared count, runs off
+the end of a 46-line file, and reports the file as corrupt. `d7134a6625` (#31881) added a line and
+did not update the count.
+
+`apply-patches.sh` runs **before every check**, so the gate never reached `aot-smoke.sh` — the one
+check that would have caught §1 on the commit that introduced it.
+
+**This is the finding worth internalising.** A gate that is permanently red for an irrelevant reason
+carries exactly as much information as one that is permanently green. Nobody reads the log of a job
+that always fails; they read the colour, see red, and merge, because red is the normal colour. The
+repo already had a catalogue of checks that reported success without executing — this is the same
+defect with the sign flipped.
+
+**Suggested follow-up:** make a setup failure and a check failure *look different*. A gate should be
+able to say "I could not run" in a way that is visibly distinct from "the code is broken".
+
+### Status: partially fixed
+
+**#31909 (open)** corrects the header to `+1426,23`. Measured on that branch
+(run `32004632440`): the `corrupt patch` abort is gone and the sequence advances past that patch —
+then fails on the **next** one, `php-cfg-match-multi-cond-block.patch`. It is a necessary step, not
+the whole fix.
+
+**A trap for whoever picks this up.** That next patch has the *same* arithmetic inconsistency
+(`@@ -2001,6 +2001,9 @@` over a 7-old/10-new body), so the obvious move is to "fix" it the same way.
+**That is not the cause.** `git apply --check -p0` on it returns **rc=0** — git tolerates the
+miscount. The real problem is context: its anchor line
+
+```php
+$testBlock = $nextBlock;
+```
+
+**does not exist anywhere in pristine `ircmaxell/php-cfg`** (checked against `72df227` in the
+composer cache). The hunk depends on context introduced by an earlier patch in the match-lowering
+family. Correcting the header would produce a change that looks like a fix, verifies nothing, and
+buries the real problem a layer deeper.
+
+Diagnosing it properly needs a fresh `composer install` to reproduce CI's vendor state. That was not
+possible during this session (10 agents were using the live `vendor/` tree; a worktree attempt timed
+out at load ~130 on a 16-core box).
+
+---
+
+## 3. AOT correctness is much weaker than the release plan implies
+
+A 10-agent differential hunt was run: each agent takes one language area, writes 15–25 deterministic
+probes, and compares AOT output against Zend byte-for-byte, with adversarial per-finding
+verification.
+
+Two areas had reported at the time of writing:
+
+| area | probes run | mismatches |
+|---|---|---|
+| statics | 41 | 21 |
+| numerics | 44 | 23 |
+| **total** | **85** | **44** |
+
+A ~50% mismatch rate on ordinary PHP. See `docs/roadmap/AOT-CORRECTNESS-PLAN.md` for the grouped
+inventory and the order to attack it in.
+
+### Confound, stated precisely
+
+The hunt ran while the box was at load 129–228 on 16 cores (mostly *unrelated* long-running jobs —
+seven `zahlenjagd` processes had each held ~97% of a core for 18 days). A wall-clock timeout under
+that load would be a contention artifact.
+
+It does not invalidate the haul: the findings carry **deterministic** signatures — exit 2/255 with
+specific compiler error strings, exit 139 (SIGSEGV), exit 134 (SIGABRT). Those are not produced by a
+busy machine. No finding of kind `hang` was reported. Re-verify anyway before acting; some findings
+in this document are probe-reported and not yet independently reproduced, and they are marked.
+
+---
+
+## 4. What is verified, and what is not
+
+**Verified by me, in the pinned image, exit codes read directly:**
+
+- `aot-smoke` 0/8 → 8/8 after #31894
+- `var_dump(7)` and `Exception::getMessage()` now match Zend
+- inherited property defaults are broken under AOT (#31895) — see below
+- the `corrupt patch` error is gone after #31909, and the gate advances one patch further
+- `$testBlock = $nextBlock` is absent from pristine php-cfg
+
+**Not verified — do not treat as established:**
+
+- that `apply-patches.sh` succeeds end to end after #31909
+- the 44 mismatches beyond the four re-tested by hand; they are probe-reported
+- anything about the eight areas that had not reported when this was written
+
+---
+
+## 5. Open issues filed
+
+| issue | what |
+|---|---|
+| #31895 | AOT: inherited property defaults are never initialised — every subclass loses its parent's defaults |
+| (see plan) | grouped AOT correctness issues, listed in `docs/roadmap/AOT-CORRECTNESS-PLAN.md` |
+
+`#31895` is worth reading as a template for the rest: minimal repro, a Zend-vs-AOT table, and an
+explicit note that the untyped variant produces *silently wrong output* rather than an error.
+
+---
+
+## 6. Operational notes for the next agent
+
+- **This box is the production server** (~53 containers: leitstand, lakehaus, and others). Check
+  `uptime` before starting compile fan-out. Do not add compile load above ~load 60.
+- **Concurrent containers sharing the bind-mounted helper cache corrupt each other.** Give every
+  concurrent `docker run` its own `PHP_COMPILER_HELPER_RUNTIME_CACHE_DIR`.
+- **Never run `composer install` or `git worktree add` against `/root/php-compiler` while agents are
+  using that tree.**
+- **Keep whole-corpus compliance runs manual.** `compliance-pr.yml` and `compliance-host-compare.yml`
+  are `workflow_dispatch`-only for a reason: an earlier 48-job-per-PR trigger queued ~2,830 jobs and
+  took repository CI down for ~7 hours.
+- `docs/roadmap/RELEASE-PLAN.md` is **stale** (dated 2026-07-29) and still claims
+  *"P1.1 CI on `lib/`, `ext/` — done, green on master"*. That was false for a long stretch. Update it
+  once the gate genuinely reports.

docs/roadmap/AOT-CORRECTNESS-PLAN.md

@@ -0,0 +1,153 @@
+# AOT correctness — inventory and plan
+
+Written 2026-08-17 from a 10-agent differential hunt (each agent: one language area, 15–25
+deterministic probes, AOT output compared against Zend byte-for-byte).
+
+**Read `docs/AGENT-HANDOVER-2026-08-17.md` first.** In particular: none of this was measurable until
+#31894, because every AOT binary failed to link. Anything diagnosed against a tree where
+`script/aot-smoke.sh` is not 8/8 is measuring the toolchain, not the feature.
+
+## Coverage so far
+
+| area | probes run | mismatches | status |
+|---|---|---|---|
+| statics | 41 | 21 | reported |
+| numerics | 44 | 23 | reported |
+| inheritance, closures, exceptions, arrays, strings, generators, references, enums-traits | — | — | hunt still running when this was written |
+
+**85 probes, 44 mismatches — roughly one in two.** These are ordinary PHP constructs, not exotic
+corners.
+
+Provenance: only the group marked **[verified]** below was re-run by hand in the pinned image with
+exit codes read directly. Everything else is **probe-reported** and should be reproduced before anyone acts on
+them. That distinction is the point — do not collapse it.
+
+---
+
+## Group 1 — float → string conversion (probably ONE bug, five crashes)
+
+Every one of these segfaults or aborts on a float. They are almost certainly a single conversion
+defect, which makes this the highest value-per-fix item in the inventory.
+
+| probe | Zend | AOT |
+|---|---|---|
+| `printf("%.1f", 1.5)` | `3` | exit 139 |
+| `number_format(1234567.891, 2)` | `1,234,567.89` | exit 139 |
+| `json_encode(0.1)` | `0.1` | segfault |
+| `serialize(0.1)` | `d:0.1;` | segfault |
+| `strval(0.1)` | `"0.1"` | **compiler itself** segfaults (exit 139) |
+
+The last one is notable: the crash is in the compiler, not the produced binary.
+
+**Do this first.** One root cause, five user-visible crashes, and float formatting is unavoidable in
+real programs.
+
+## Group 2 — integer overflow does not promote to float
+
+PHP semantics: integer arithmetic that overflows becomes a float. AOT wraps instead.
+
+| probe | Zend | AOT |
+|---|---|---|
+| `PHP_INT_MAX + 1` | `float(9.2233720368548E+18)` | `int(-9223372036854775808)` |
+| `PHP_INT_MAX * 2` | `float(1.8446744073709552E+19)` | `int(2)` |
+
+Silent wrong output on arithmetic — the characteristic failure class named in `AGENTS.md` §3. No
+crash, no warning; the program simply computes a different number.
+
+Related numeric mismatches, same area:
+
+- `intdiv`-style const-folded division truncates: `7/2` const-folds to `int(3)` instead of `float(3.5)`
+- float post-increment truncates: `$x = 1.5; $x++` gives `int(2)`, Zend gives `float(2.5)`
+- `PHP_INT_MIN % -1` gives `int(-1)` / `int(-9223372036854775808)`; Zend gives `int(0)` for both
+- `abs(PHP_INT_MIN + 1)` returns a float; Zend returns `int(9223372036854775807)`
+- `--$null` decrements; in Zend decrementing null is a no-op
+- `var_dump` of a float uses `precision` rather than `serialize_precision`
+- `echo` of `1.0E+100` prints `1e+100`; Zend prints `1.0E+100`
+- `var_dump(INF)` prints `float(inf)`; Zend prints `float(INF)`
+- an `(int)`/`(float)` cast passed **directly as a call argument** yields `NULL`
+
+## Group 3 — static property storage
+
+Several wrong-output cases. One deserves separate, urgent handling:
+
+**`static-prop-write-from-closure` returns an unstable garbage integer** — a different value between
+runs (`-6341068275337658368` in one). That is a read of uninitialised memory, not a logic error, and
+it should be filed and fixed on its own terms.
+
+Others in the group:
+
+- a static array property's default is lost (`3` → `0`)
+- a static property increment inside a method loses one update (`2` → `1`)
+- a child class does not share its parent's static property (`42` → `1`)
+- a static array property returned by value aliases instead of copying (`1` → `99`)
+- `??=` to a static property is not stored (`int(7)` → `NULL`)
+- a function-static array element write is lost across calls (`12` → `11`)
+- `&` reference to a static property is not bound
+
+Plus three aborts (exit 134): array-callable to a static method, string-callable to a static method,
+and a static method called on an instance.
+
+## Group 4 — LLVM module verification failures
+
+These fail with `Uncaught RuntimeException: Module verification failed`, i.e. the emitted IR is
+invalid. Likely a small number of shared causes.
+
+- `2 ** 10` (pow operator)
+- `base_convert(...)`
+- `property_exists()` on a class with a static property
+- writing to a function-`static` string variable
+
+## Group 5 — unimplemented lowering (honest errors, not corruption)
+
+These abort with a clear message rather than producing wrong output, so they are lower priority than
+groups 1–3 — but they are ordinary PHP:
+
+- `"5" + 5` → `Reached end of switch, can't handle binary operation yet: TYPE_PLUS`
+- `NAN <=> 1.0` → same, `TYPE_SPACESHIP`
+- enum case as a class-constant value → `Unsupported compile-time constant for JIT (vm type 9)`
+- `$obj::method()` / variable class in a static call → `Static call class must be a literal`
+- interface constant via `self::` in a const expression → `Undefined constant C::X`
+- array literal stored to a static property → `JIT static property boxed store does not support value type __string__`
+
+---
+
+## Group 6 — inherited property defaults **[verified]** (#31895)
+
+Filed separately because it is verified and self-contained.
+
+```php
+class A { public string $p = 'hi'; }
+class B extends A {}
+echo (new B)->p;
+```
+
+| case | Zend | AOT |
+|---|---|---|
+| instantiate `A` directly | `hi` | `hi` |
+| instantiate `B extends A`, typed | `hi` | `Uncaught Error: Typed property A::$p must not be accessed before initialization` |
+| instantiate `B extends A`, untyped | `hi` | `Warning: Undefined property: A::$p` then empty |
+
+Any class hierarchy with a base-class property default is broken. The untyped variant produces
+silently wrong output.
+
+---
+
+## Suggested order
+
+1. **Group 1** — one fix, five crashes, unavoidable in real code.
+2. **Group 3's uninitialised-memory read** — unstable output is worse than wrong output, because it
+   is not reproducible and will not stay caught.
+3. **Group 2** — silent wrong arithmetic; wide blast radius, no diagnostic.
+4. **Group 6** (#31895) — wide blast radius, already verified and minimal.
+5. **Group 4** — invalid IR; probably few causes.
+6. **Group 5** — honest failures; safe to schedule last.
+
+## How to work these
+
+- `script/aot-smoke.sh` must be 8/8 **before** and after any change. If it is not 8/8, stop: you are
+  measuring the toolchain.
+- Add a compliance case per fix; a fix without a case will regress unnoticed.
+- Fan out with `isolation: 'worktree'` if using subagents — concurrent containers sharing the
+  bind-mounted helper cache corrupt each other, and each needs its own
+  `PHP_COMPILER_HELPER_RUNTIME_CACHE_DIR`.
+- Groups 1–4 are independent; they parallelise cleanly. Group 5 items mostly do not share causes.