O13 (draft, pre-registration): estimate the verdict from a real interaction log

OPEN_QUESTIONS.md

@@ -25,7 +25,7 @@ Status is one of `open`, `claimed`, `answered`. Needs is what the thread is wait
 | O10 | Field the pre-registered three-wave algorithmacy panel with a real cohort. | open | data | `org_frontier/survey/cohort_algorithmacy/` | `PYTHONPATH=. python org_frontier/survey/cohort_algorithmacy/analysis.py` |
 | O11 | Can trained coders look at a real arrangement and agree on dyadic vs triadic reliably enough to use the construct as an instrument? | open | data | `org_frontier/designs/WAVE10_PARKED_DESIGNS.md` #42 | read the design, then the field protocol's step 4 |
 | O12 | Do human triads on a joint-commit interface show the difficulty gap the verdict predicts, where selfish bandit agents did not (Probe 98)? | open | data | `org_frontier/designs/WAVE10_PARKED_DESIGNS.md` #41 | read the design sketch and its pre-registration outline |
-| O13 | Can the verdict be read from a real interaction log, estimating the TPM over a few discrete units? | open | data | `org_frontier/designs/WAVE10_PARKED_DESIGNS.md` #24; runnable-now datasets in `org_frontier/research/DATA_SOURCES.md` | `python tools/build_catalog.py --find "interaction log TPM estimate"` |
+| O13 | Can the verdict be read from a real interaction log, estimating the TPM over a few discrete units? | claimed | data | `org_frontier/designs/WAVE10_PARKED_DESIGNS.md` #24; runnable-now datasets in `org_frontier/research/DATA_SOURCES.md` | `python tools/build_catalog.py --find "interaction log TPM estimate"` |
 | O14 | Why does the ergodic-components gap stay unexplained by phase, path, observable, or panel mechanisms (`GAP_UNEXPLAINED`)? | open | compute | `org_frontier/ergodicity/AGENDA.md`, `org_frontier/studies/ergodic_gap_decomposition/` | `python tools/build_catalog.py --find "ergodic gap"` |
 | O15 | Review a submission: reproduce its numbers, audit its pre-commitment, judge its argument. The board needs reviewers. | open | review | `MAINTAINERS.md`, `PUBLISHING.md` | `python ci/reproduce.py <check-name>` on an open pull request |
 

org_frontier/recurrence/log_estimation/HYPOTHESES.md

@@ -0,0 +1,95 @@
+# O13 (with O09) — the verdict read from a real interaction log: estimating the TPM
+
+Pre-registration, Antonio Mele, 30 Oct 2026. Committed before any data is fetched or any estimate on the full window is computed. Follows
+`org_frontier/recurrence/packets/template/PRE_REGISTRATION.md` and `WAVE10_PARKED_DESIGNS.md` #24.
+The pilot below used only the CSVs already in `bot_merged/`; no
+new data has been fetched.
+
+## Question
+Can the dyadic/triadic verdict (Φ>0 with a multi-party major complex vs not) be *estimated*, with calibrated
+uncertainty, from a logged real coordination, instead of being read off an elicited Boolean model?
+Secondary (O09): does the recurrence instrument (CRQA lead-lag) on the same log agree with the estimated
+structure?
+
+## Prior work in the repo
+- `org_frontier/designs/WAVE10_PARKED_DESIGNS.md` #24: estimability shown on synthetic trajectories
+  (Φ_AR AUC 0.925, Probe 84; random-forest surrogate AUC 1.000, Probe 85; ΦID Φ_R fails, Probes 86-87).
+- `org_frontier/recurrence/bot_merged/` (v11): Kubernetes Prow/Tide, 150 PRs, 100% bot-merged; the
+  *elicited* Prow model is triadic, Φ=2.0, core {approval, bot, codebase}; bot is a member but a functional
+  conduit (collapsing it keeps Φ=2.0). Reproduced on this box.
+- `recurrence/event_series` (v9, human merger), `review_heavy` (v10), `real_series` (commit activity).
+- `org_frontier/field/rule_to_phi.py` (coding disagreement → CI), `proxy_bridge/`.
+- Open PR #842 (q221): pairwise recurrence summaries do not identify the transition law, while transition likelihood does; this plan therefore reads Φ from an estimated TPM, with CRQA only as the H5 cross-check.
+- Gap this fills: every Φ so far is on an elicited or synthetic TPM; none estimated from a log.
+
+## Arrangement and data
+Primary: **GitHub bot-merged projects via GH Archive** (gharchive.org; hourly JSON.gz of public GitHub events
+since 2011; one hour checked = ~21.8 MB compressed; open). Parties per PR-repo, on an hourly (or 4-hour) clock:
+- Worker W = contributor activity (PushEvent / PullRequestEvent opened|synchronize by non-maintainers).
+- Mediator M = merge bot (k8s-ci-robot / Prow / bors / mergify / github-actions merge, from
+  PullRequestEvent closed+merged with bot `merged_by`, or bot IssueCommentEvent).
+- Counterpart C = human gatekeepers (PullRequestReviewEvent approved, `/lgtm`/`/approve` comments by
+  OWNERS).
+- Optional 4th/5th nodes: CI status (CheckRun/Status events are *not* in GH Archive — use bot comments),
+  codebase change (push to default branch).
+Design contrast (the instrument's lever): projects with a merge bot (kubernetes/kubernetes, rust-lang/rust
+via bors) vs matched human-merge projects. Fallback/second site: Wikipedia bot edits (mediawiki_history
+dumps; bot flag logged) for a W/bot/patroller triad.
+
+## Encoding
+Node bit = 1 if ≥1 qualifying event in the bin. Pre-committed: bin = 1 hour primary; 15 min and 4 h as
+robustness. Lag 1 bin (Markov order 1) primary; order-2 tested as a check. Per-repo marginal rates must lie
+in [0.05, 0.95] or the bin is changed *before* Φ is computed (pilot below shows daily bins saturate).
+
+## Estimation
+1. State-by-node TPM, n ≤ 5 primary (≤ 8 exploratory; exact IIT 4.0 cost limits to ~10-12).
+2. Each cell P(x_i,t+1=1 | x_t=s) gets a Beta(a,a) prior (a=1 primary, 0.5 sensitivity) → conjugate posterior.
+3. Structural alternative: per-node Boolean/logistic rule over parents with lag; parent sets chosen by BIC
+   (or Bayesian model averaging over ≤ 2^(n-1) parent sets); TPM implied by the selected rules + a noise ε.
+4. Uncertainty → Φ: draw K=500 posterior TPMs (or block-bootstrap the series, block = 1 day), compute
+   max-over-states major complex Φ and core for each; report median, 90% interval, P(multi-node core with
+   W, M, C ⊂ core), and core frequency table.
+5. Nulls: per-node circular-shift surrogates (keep autocorrelation, break coupling); same pipeline.
+6. Identification: unvisited states are prior-filled → report coverage; IIT perturbs all states, so
+   Φ on prior cells is assumption-driven; interventional states only identifiable from natural variation
+   (bot outages, holiday freezes, policy changes = quasi-experiments). Common drivers (time-of-day) handled
+   by detrending/conditioning on hour-of-week, or adding a clock node.
+
+## Hypotheses (each with its null)
+- H1 (coverage). With hourly bins ≥ 6/8 states of (W,M,C) are visited ≥ 30 times in a 90-day window.
+  Null: coverage fails → the estimate is prior-dominated and no verdict is read.
+- H2 (structure). In bot-merged repos the estimated TPM gives a multi-node major complex containing M and C
+  in ≥ 80% of posterior draws. Null: core is single-node or excludes M/C (dyadic reading).
+- H3 (conduit). Collapsing M (C reads approvals directly) changes posterior median Φ by < 10%, reproducing
+  v11's "member but conduit" result on estimated rather than elicited structure. Null: Φ drops ≥ 10%.
+- H4 (contrast). Bot-merge repos vs human-merge repos differ in core composition, not in Φ>0 rate.
+  Null: no difference in either.
+- H5 (instrument agreement, O09). CRQA cross-recurrence of C→M shows a positive lag peak (approval leads
+  merge) with prominence above threshold, matching the estimated C→M edge. Null: flat/zero-lag profile.
+
+## Decision rules
+- Instrument controls must pass first (`run_study.py`): decoupled → no multi-node complex; coupled cycle → Φ>0
+  full core. Note from pilot: a self-loop node alone yields a single-node complex with Φ=1, so the verdict
+  is read from the *multi-node* core, not raw Φ>0.
+- Verdict "triadic" only if P(multi-node core incl. M & C) ≥ 0.8 AND observed Φ exceeds the 95th percentile
+  of the surrogate null. "Dyadic" if ≤ 0.2. Otherwise "undetermined" — reported as a result.
+- H1 failing halts H2-H4 for that bin width.
+- Estimated-vs-elicited comparison: agreement = same core as the documented Prow model.
+
+## Instrument check on a known case
+Simulate logs from the elicited Prow TPM (v11) with ε noise at the empirical marginal rates and series
+length; run the identical estimation blind; require core recovery in ≥ 80% of 100 replicate logs before
+touching real data. Also feed the v9 human-merge model as a second known case.
+
+## Threats
+Binning/aggregation artefacts; non-stationarity (releases, freezes); unobserved parties (CI, Slack, Prow
+labels outside GH events — v11 caveat that reviews undercount approvals); bot accounts ambiguous;
+sparse states and prior sensitivity; Φ is a max over states (multiple comparisons); order-1 Markov
+misspecification; selection of repos. Ethics: public data, report repo/bot level only, no individual
+profiling.
+
+## Pilot already run (box, 30 Oct 2026)
+Daily bins on the in-repo v11 CSVs (22 days): B=1 on every day, W on 95%; 3 of 8 states visited
+(1, 6, 14 transitions). Posterior Φ median 0.48 (90% [0.25, 0.77]), but cores were single nodes in 34/40
+draws and a time-shuffle null gives ~0.22 — i.e. prior-dominated and uninformative. Conclusion: finer bins
+and a longer window are required (motivates H1).

org_frontier/recurrence/log_estimation/README.md

@@ -0,0 +1,7 @@
+# O13 — the verdict estimated from a real interaction log
+
+Claimed open question O13 (with O09 as its recurrence cross-check). This packet estimates a Boolean TPM from
+a logged worker / merge-bot / human-approver coordination in GitHub Archive event data, propagates the
+posterior uncertainty into exact IIT-4.0 Φ, and compares the estimated core with the elicited Prow model of
+[`bot_merged/`](../bot_merged/). Status: pre-registered, no data fetched. The predictions, encoding,
+decision rules, and instrument check are in [`HYPOTHESES.md`](HYPOTHESES.md).

org_frontier/recurrence/log_estimation/RESULTS.md

@@ -0,0 +1,79 @@
+# O13 results — hourly-bin run (30 Oct 2026)
+
+Run of the pre-registration in `../HYPOTHESES.md` (committed dd64c48a before any data fetch).
+Code: `analyze.py`, `fetch.sh`, `extract.py`. Full numbers: `results.json`, log `run.log`.
+
+## Data
+GH Archive hourly files, 90 days 2024-04-01 to 2024-06-29 (2,160 hours; every hour present). The files
+were streamed and filtered to five repositories, and no raw archive was kept. 2024 was chosen because
+2025–26 hours in the archive carry far fewer events per repository. Spot checks of kubernetes/kubernetes
+found 55 events in one 2024 hour against 1 in 2026 hours. The filtered events are 175,115 lines (~52 MB),
+kept on the box only and not committed.
+- Bot-merged: kubernetes/kubernetes (730 merges, 100% bot), rust-lang/rust (1,965 merges, 100% bot).
+- Human-merged comparison: django/django, facebook/react, microsoft/vscode (0% bot merges).
+- Nodes: W = PR opened/synchronize/reopened by a non-bot; M = PR merged; C = non-bot approving review or
+  `/lgtm`, `/approve`, `@bors r+` comment. Each node is 1 in a bin with ≥1 event. Lag 1. Beta(1,1) prior.
+  K=500 posterior draws and 200 circular-shift surrogates (≥24 h shift).
+
+## Instrument check (run first)
+- Controls pass. Decoupled → single-node complex only (Φ=1.0, core C). Coupled cycle → Φ=2.0, core WMC.
+- Blind recovery from simulated logs (T=2,160, ε=0.05, 100 replicates):
+  - v11 Prow projection (cycle): 100% → pass.
+  - **v9 human-merge (AND gate): 39% → FAIL** (rule: ≥80%).
+- Under the pre-registered rule, failing the instrument check means real-data verdicts are not read as
+  confirmatory. The real-data numbers below are reported as **exploratory**.
+
+## Real data, 1-hour bins (primary)
+| repo | merge | marginals W/M/C | states ≥30 | Φ median [90%] | P(multi-node core ⊇ M,C) | obs Φ (post-mean) | null 95th | rule label |
+|---|---|---|---|---|---|---|---|---|
+| kubernetes | bot | .34/.22/.28 | 8/8 | 0.167 [0.128, 0.211] | 0.000 | 0.152 (C) | 0.301 | dyadic |
+| rust | bot | .63/.35/.56 | 8/8 | 0.501 [0.457, 0.557] | 0.000 | 0.497 (M) | 0.497 | dyadic |
+| django | human | .11/.06/.08 | 6/8 | 0.245 [0.121, 0.483] | 0.000 | 0.224 (C) | 0.586 | dyadic |
+| react | human | .21/.13/.13 | 8/8 | 0.175 [0.114, 0.252] | 0.002 | 0.162 (W) | 0.526 | dyadic |
+| vscode | human | .48/.41/.44 | 7/8 | 0.167 [0.131, 0.225] | 0.086 | 0.157 (W) | 0.380 | dyadic |
+
+Core frequencies (kubernetes): C 302, W 196, M 2 of 500. Rust: M 500/500. Vscode: W 374, M 56, WMC 43, MW 26.
+Every repo's Φ is that of a **single-node self-loop complex** and sits at or below the surrogate null.
+Sensitivity to the Beta(0.5) prior leaves the medians essentially unchanged (kubernetes 0.170).
+Robustness: 15-min bins give the same reading (all single-node cores, Φ below the null). The exception is
+django, which fails H1 at 15 min. 4-hour bins fail H1 for every repo.
+
+## Hypotheses vs decision rules
+- **H1 coverage: supported.** All five repos reach ≥6/8 states with ≥30 visits at 1 h (kubernetes and
+  rust 8/8). Marginals are within [0.05, 0.95].
+- **H2 structure: not supported** (null holds). P(multi-node core ⊇ M,C) = 0.000 for both bot-merged
+  repos, against a required ≥0.8. The estimated arrangement does not recover the elicited v11 triadic core
+  (CMW / RBC), so it disagrees with the elicited model.
+- **H3 conduit: kubernetes −** collapsing M changes median Φ by +8.6% (<10%, formally "supported").
+  Rust changes by −84%, so H3 fails there. Because the full-system core is a single node, H3 does not
+  test the conduit claim it was written for.
+- **H4 bot vs human contrast: not supported.** Every repo is single-node and dyadic-labelled, and the
+  Φ>0 rate is identical. The only composition difference is that rust's core is M; that is not systematic
+  across bot repos (kubernetes's core is C/W).
+- **H5 CRQA C→M (O09): kubernetes supported.** The peak lag is +1 h (approval leads merge) with
+  prominence 0.088, above the 0.05 floor. The estimated C→M effect is also positive in the posterior-mean
+  TPM (see `results.json`). Rust has its peak at −5 h with low prominence, so H5 fails there.
+- **Verdict: inconclusive overall.** The instrument check failed on v9, so no confirmatory verdict can be
+  read. The exploratory reading on every repo is dyadic: there is no multi-party integration above
+  surrogate level. Behavioural coupling C→M is visible in kubernetes, but it does not produce Φ>0 for a
+  multi-node core.
+
+## Caveats
+- **Estimator weakness.** Blind recovery fails for AND-gate (v9) structures at this series length and
+  noise. The posterior-mean TPM with ε-noise smooths a conjunctive rule toward independence. The
+  pre-registered estimator is therefore underpowered for gate structures, which is a finding in itself.
+- **Encoding issues.**
+  - Activity-presence bits are not the elicited "state" bits. A PR approved and later merged spans many
+    hours, and the elicited model's bits are standing states (approved / merged), not event pulses.
+  - Hourly lag-1 also cuts across Prow's batching.
+- **Unobserved parties and drivers.** Unobserved CI and Prow labels are missing. Time-of-day common
+  drivers were not conditioned out (no clock node), and they push every node toward self-persistence.
+- **Pre-registered steps not run.** The BIC structural alternative, the order-2 check, and the
+  block-bootstrap variant were not run. The posterior-draw route was used, which the plan allows.
+
+## Suggested next step (not pre-registered here; needs a new pre-registration)
+- Encode standing states rather than event pulses: open-PR-awaiting-approval, approved-not-merged,
+  merged-in-last-h.
+- Add a clock node.
+- Use a rule-based (BIC) estimator, which should recover AND gates better.
+- Re-run the instrument check before any real data.