Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
1b337f8
Study 019: design scaffold — draft preregistration, panel-reviewed br…
kikashy Aug 15, 2026
29cf3f1
Study 019: record numeric outputs as a JPS roadmap item, and pin the …
kikashy Aug 15, 2026
a1458e5
Study 019: design-time toolchain resolution — OPA v1.19.0, jpack v0.1…
kikashy Aug 15, 2026
8fc1076
Study 019: contest policy v0 → panel → v0.1 — three verified inexpres…
kikashy Aug 15, 2026
873127e
Study 019: both references built and agreeing 2,540/2,540 — V6 settle…
kikashy Aug 15, 2026
a5bb49f
Study 019: gold suite v0 — 76 rows, checker green, floor gate passes …
kikashy Aug 15, 2026
04bd5f0
Study 019: clean-room oracle concurs — 76/76 on gold, 2,540/2,540 on …
kikashy Aug 15, 2026
3f73226
Study 019: drop a stray bytecode cache from the clean-room directory
kikashy Aug 15, 2026
bfaf149
Study 019: pilot materials — full-verbatim excerpts, contracts, and t…
kikashy Aug 15, 2026
3f382ef
Study 019: calibration pilot 01 — every arm at ceiling on E1; the var…
kikashy Aug 15, 2026
6ce5507
Study 019: primary endpoint pivots to E4 after the pilot ceiling — de…
kikashy Aug 15, 2026
2bddf58
Study 019: E4 scoring rules, run over the calibration pilot (NON-CITA…
kikashy Aug 15, 2026
793cbe9
Study 019: mutant sets committed + the E4 pilot read — the identity c…
kikashy Aug 15, 2026
019c95b
Study 019: full preregistration draft — R1 registered over E4, design…
kikashy Aug 15, 2026
e2279d6
Study 019: four pre-freeze gates land — adequacy satisfied, off-gold …
kikashy Aug 15, 2026
d66e748
Study 019: A1 closed — the risk-40 spend cliff is confirmed intended
kikashy Aug 15, 2026
7d551cf
Study 019: harness assembled and end-to-end smoked — 353 tests green,…
kikashy Aug 15, 2026
4168af8
Study 019: harness closeout — every stub landed, 387 tests green, the…
kikashy Aug 16, 2026
bf96915
Study 019: review round 1 opens — the prompt, committed before the re…
kikashy Aug 17, 2026
5527fe0
Study 019: review round 1 returns DO NOT FREEZE — 11 blockers, 8 majo…
kikashy Aug 17, 2026
f00d097
Study 019: round-1 response lands — X1 eliminated, all 20 findings fi…
kikashy Aug 18, 2026
5844037
Study 019: round-1 dispositions — twenty findings, each citing its en…
kikashy Aug 18, 2026
2c5f706
Study 019: review round 2 opens — disposition verification and the se…
kikashy Aug 18, 2026
e80ccea
Study 019: round 2 returns DO NOT FREEZE — 4 of 20 dispositions hold;…
kikashy Aug 18, 2026
7ce22ad
Study 019: round-2 response — all fourteen findings closed, suite 669…
kikashy Aug 19, 2026
cfd8eda
Study 019: review round 3 opens — disposition verification, the revie…
kikashy Aug 19, 2026
cf251c5
Study 019: round 3 returns DO NOT FREEZE — 7 of 14 hold; the reviewer…
kikashy Aug 19, 2026
c7ce326
Study 019: round-3 response — the manifest class closed at the root, …
kikashy Aug 19, 2026
80e54c8
Study 019: review round 4 opens — disposition verification, the two p…
kikashy Aug 19, 2026
cb1d1e8
Study 019: round 4 returns FREEZABLE AFTER LISTED FIXES — the first c…
kikashy Aug 19, 2026
fabe9a0
Study 019: round-4 response — every finding fixed at its generator, 7…
kikashy Aug 19, 2026
563ea8c
Study 019: review round 5 opens — disposition verification and the fi…
kikashy Aug 19, 2026
aab836f
Study 019: round 5 returns DO NOT FREEZE — the maintainer committed b…
kikashy Aug 19, 2026
33a3eed
Study 019: round-5 response — the bytecode class closed four ways, ev…
kikashy Aug 19, 2026
625c0cd
Study 019: review round 6 opens — disposition verification and the fi…
kikashy Aug 19, 2026
10b0f66
Study 019: round 6 returns DO NOT FREEZE — the straggler staged at la…
kikashy Aug 19, 2026
21c9545
Study 019: round-6 response — the open-round model, structured guards…
kikashy Aug 19, 2026
a910266
Study 019: review round 7 opens — the open-round model's live trial
kikashy Aug 19, 2026
f5132fd
Study 019: round-7 response — the descope executed, the ceremony made…
kikashy Aug 19, 2026
b5cb5ca
Study 019: review round 8 opens — the descope on trial, mechanically
kikashy Aug 19, 2026
c196276
Study 019: round-8 response and close — the structural replacement co…
kikashy Aug 19, 2026
b7e755a
Study 019: review round 9 opens — the convergence attempt
kikashy Aug 19, 2026
e66c8a8
Study 019: round-9 close under the ratified scope ruling — two regist…
kikashy Aug 19, 2026
7ab10a2
Study 019: round-10 close — R10-1 adopted in full depth, the freeze g…
kikashy Aug 19, 2026
282bb28
Study 019: round-11 close — one finding, two tests, each layer fails …
kikashy Aug 19, 2026
89a39c0
Study 019: round 12 returns FREEZABLE AS WRITTEN — the review regime …
kikashy Aug 20, 2026
7cb251d
Study 019: salvage the E1-line controls onto the canonical line
kikashy Aug 20, 2026
b8cefc0
Study 019: the freeze-fill's mechanical phase, F2 through F6
kikashy Aug 20, 2026
466516a
Study 019: the ceremony completes — every pin filled, REGISTERED, 102…
kikashy Aug 20, 2026
857e6e8
Merge remote-tracking branch 'origin/main' into study-019-scaffold
kikashy Aug 20, 2026
b73308e
Study 019: the freeze ceremony's artifact half — every registered doc…
kikashy Aug 20, 2026
7079d5e
Study 019: the two ceremony lines merge — corrections win, the fill s…
kikashy Aug 20, 2026
88ba872
Study 019: the preregistration pin binds the merged bytes
kikashy Aug 20, 2026
4bf27a0
DCO remediation for 49 commits authored as kikashy <kikashy@hotmail.com>
kikashy Aug 20, 2026
a1a6102
Merge remote-tracking branch 'origin/main' into study-019-scaffold
kikashy Aug 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
52 changes: 52 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,58 @@ jobs:
working-directory: studies/018-transition-rules
run: python -m pytest harness/tests -q

study-019-harness:
name: Study 019 · deterministic harness
runs-on: ubuntu-latest
steps:
- name: Check out source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
# The fixed CI runtime, and the honest statement of what it is
# (round-5 finding R5-5). harness/PINS.json registers the CPython
# 3.12 SERIES and integrity.verify_interpreter() enforces
# implementation and series; the patch level is deliberately
# reported at run time and not required (Study 012's round 3,
# finding 20). Naming 3.12.11 here fixes the runner so this job is
# reproducible, and it is not a claim that another patch would be
# refused.
python-version: "3.12.11"
- name: Install pytest
# Fully offline apparatus: the harness, the design generators and the
# controls are stdlib-only, so pytest is the whole install. No evaluator
# binary, no external clone.
run: python -m pip install --disable-pip-version-check pytest
- name: Verify the port chain, the pins and the study manifest
# `integrity.py` refuses to run without PYTHONSAFEPATH=1: invoking a
# script by path puts that script's own directory first on `sys.path`
# before any byte of the file runs, so the operator applies -P and this
# step establishes that they did.
working-directory: studies/019-authorship-across-representations
env:
PYTHONSAFEPATH: "1"
PYTHONDONTWRITEBYTECODE: "1"
run: python harness/integrity.py
- name: Run the harness, design and control suite
# Deterministic and offline: the registered partition and its per-code
# reachability, the transcript binding, the leak-token screen, the
# scorer's population and publication rules, the ports chain, the
# manifest's currency with the tree, and the currency assertions that
# recompute every count the preregistration states from the artifact it
# states it about. THE MATRIX ADJUDICATION NEVER RUNS HERE: it is an
# ATTEMPT, not a test (§7), and neither it nor anything that invokes
# `codex`, `jpack` or `opa` is part of this job. The engine-backed
# pipeline module skips itself unless the pinned binaries are present
# and hash to their pins, which in CI they are not.
# `PYTHONDONTWRITEBYTECODE` is not decoration: `integrity.verify()`
# refuses any `.pyc` the running interpreter did not produce, so a test
# run that writes bytecode would break the step above on the next run.
working-directory: studies/019-authorship-across-representations
env:
PYTHONDONTWRITEBYTECODE: "1"
run: python -m pytest harness/tests -q

python:
name: ${{ matrix.os }} · Python ${{ matrix.python }}
strategy:
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ records; it remains the canonical matrix, with the external source each study bu
| [016](studies/016-policy-currency-anchor/) | Interoperability | Frozen + run — R1 holds (both strata) | [`ANALYSIS.md`](studies/016-policy-currency-anchor/ANALYSIS.md), [`results/`](studies/016-policy-currency-anchor/results/) |
| [017](studies/017-witnessed-currency/) | Currency governance | Frozen + run — R1 holds (both strata) | [`ANALYSIS.md`](studies/017-witnessed-currency/ANALYSIS.md), [`results/`](studies/017-witnessed-currency/results/) |
| [018](studies/018-transition-rules/) | Currency governance | Frozen + run — R1 holds; reviewer holdout diverged on three preregistered cells | [`ANALYSIS.md`](studies/018-transition-rules/ANALYSIS.md), [`results/`](studies/018-transition-rules/results/) |
| [019](studies/019-authorship-across-representations/) | Blinded authorship | **Design draft — not preregistered** | [`PREREGISTRATION.md`](studies/019-authorship-across-representations/PREREGISTRATION.md) (draft), [`design/`](studies/019-authorship-across-representations/design/) |

This repository claims **no JPS conformance** for anything in it, and the table above adds no
aggregate headline: each study answers a different preregistered question and must be read with
Expand Down
2 changes: 2 additions & 0 deletions studies/019-authorship-across-representations/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
__pycache__/
.pytest_cache/
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Correction targets — Study 019 (registered before the freeze)

Preregistration §10 pins, before the freeze, where a correction must land and what it must
correct — verbatim wording, venue, URL, retrieval date — so that a failed, corrected, or
retracted claim has a pre-committed place to be corrected IN, with the same prominence as
the claim (the Study 012 `CLAIM.md` discipline; round-7 finding R7-9 is why this document
is a registered, freeze-gated artifact rather than a declared intention).

The study publishes in-repository. Every venue below is a file in
`Judgment-Pack/judgment-pack-evaluator-experiments`, referenced at `main`; URLs retrieved
2026-08-19. A correction to any target lands in the SAME file, at the SAME prominence
(head-of-file banner for documents; the row itself for index rows), and — for a corrected
or retracted R1 — additionally as a banner at the head of `ANALYSIS.md` and an entry in
`DEVIATIONS.md`, which is freeze-excluded precisely so it can receive one.

| # | Claim that may need correction | Venue | URL (retrieved 2026-08-19) |
|---|---|---|---|
| T1 | The R1 verdict sentence `ANALYSIS.md` will publish — one of the §5 decision table's registered rows, quoted verbatim from `RESULTS.json`'s `verdict` member at the primary attempt | `studies/019-authorship-across-representations/ANALYSIS.md` (does not exist until the attempt; the target binds the file by registered path) | https://github.com/Judgment-Pack/judgment-pack-evaluator-experiments/blob/main/studies/019-authorship-across-representations/ANALYSIS.md |
| T2 | The study-index row — Question, Theme, Status columns — whose Status cell will carry the R1 verdict | `studies/README.md`, the Study 019 row | https://github.com/Judgment-Pack/judgment-pack-evaluator-experiments/blob/main/studies/README.md |
| T3 | The repo-root index row for Study 019 (same Status discipline) | `README.md`, the Study 019 row | https://github.com/Judgment-Pack/judgment-pack-evaluator-experiments/blob/main/README.md |
| T4 | The study README's rendered round-status sentence and its verdict history, should any round's record require correction | `studies/019-authorship-across-representations/README.md` | https://github.com/Judgment-Pack/judgment-pack-evaluator-experiments/blob/main/studies/019-authorship-across-representations/README.md |
| T5 | The pilot readings quoted in the preregistration's Design provenance (A 0.878, B 0.897, C 0.806; high-kill 1/5, 0/5, 0/5) — non-citable, but published in a frozen document and correctable only by `DEVIATIONS.md` entry once frozen | `studies/019-authorship-across-representations/PREREGISTRATION.md`, Design provenance | https://github.com/Judgment-Pack/judgment-pack-evaluator-experiments/blob/main/studies/019-authorship-across-representations/PREREGISTRATION.md |

The verbatim wording of T1 cannot exist before the attempt; what is pinned is its closed
vocabulary — the §5 decision table's row strings as implemented by
`harness/e4lib/decision.py` and published by `harness/score.py`, which are freeze-covered
bytes — and the rule that ANALYSIS.md quotes the published string unedited. For T2–T4 the
current wording is whatever those files carry at the freeze commit; the correction
obligation attaches to the row/sentence, not to a snapshot of it. A correction is written
in every branch of §5's outcome space, including "no correction needed" being visibly
distinguishable from "correction owed and absent": if the primary attempt publishes
without incident, `CORRECTION.md` is still created, stating that no target required
correction — an absent file is a failure to publish, never an outcome.

## Target bound by the parallel ceremony line (merged 2026-08-19, both lines' work kept)

The parallel fill line searched for claims beyond the T1–T5 in-repo surfaces and bound one
this table lacked, kept here verbatim:

## Target 1 — the design-phase pilot's directional read

**Venue:** `design/mutants/E4-NOTES.md` in this repository — a committed design record on
`main` of `Judgment-Pack/judgment-pack-evaluator-experiments`, which is a published venue
in exactly the sense Study 012's first target was: public, attributable, and quotable by a
reader who never opens the harness.

**URL (commit-pinned):**
<https://github.com/Judgment-Pack/judgment-pack-evaluator-experiments/blob/c7ce3269c3d57d29da957b2c2e3556dcbf7e80df/studies/019-authorship-across-representations/design/mutants/E4-NOTES.md>

**Retrieved:** 2026-08-19, from the working tree at that commit.

**Verbatim wording:**

> **E4 discriminates, and in this pilot the direction is B/C above A.** The earlier surface
> read (35–49 authored rows vs 1–4 test rules) was misleading: the Rego test rules are
> table-driven and carry many assertions. Small N; non-citable; but the endpoint has
> headroom and variance, which is what the pivot needed.

**What would require correcting it:** the sentence already labels itself small-N and
non-citable, so a divergent registered result does not falsify it — but a registered result
in the SAME direction must not be reported as confirming a prediction, because the
preregistration (§11) states the anchor "no longer supports either way" after the reference
repair. The correction this file binds is therefore about *use*, in either direction: if
any post-freeze surface cites the pilot direction as evidence — for A or against A — the
correction lands at the head of `CORRECTION.md`, in this venue, quoting this wording.

## The recorded decision: no external venue carries a bound claim

Searched before the freeze: the maintainer's external posts on the program (dev.to,
retrieved 2026-08-19) concern the evaluator's authorship-coverage line (Studies 009–012)
and the trust-input line; none asserts a claim about what a representation's authored test
suite pins — this study's subject. Study 012's own published claim has its own correction
machinery (`studies/012-policy-perturbation/CORRECTION.md`) and is not re-bound here. If a
reviewer of the freeze PR knows an external claim this list misses, adding it is a one-line
edit to this file **before** the squash-merge; after the freeze, a missed target is a
DEVIATIONS.md entry, not a quiet edit.
6 changes: 6 additions & 0 deletions studies/019-authorship-across-representations/DEVIATIONS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Deviations — Study 019

Deviations from the frozen preregistration land here with a reason and a date — never by
editing the preregistration or any frozen artifact. Nothing is frozen yet.

(none)
Loading