Share the world, resample the consequences.
Branching agent RL re-runs the same prefix many times to compare actions. Your sandbox can roll back state. Nothing rolls back the world — so every branch gets its own clock, its own search ranking, its own latency spike. Those independent draws land straight in the advantage estimate, where they are not signal. They are variance.
Groundhog is the layer underneath a branching search. It records the exogenous channels once, then serves every branch of a fork the same exogenous noise, while re-rendering anything the branch actually changed.
import verifiers as vf
from groundhog import with_groundhog
env = with_groundhog(vf.load_environment("your-env"), tape="tapes/your-env")That is the whole integration. Everything below is the evidence that it matters.
On a demo environment with an exogenous reward shock 9× larger than the gap between the best and second-best action:
| (A) Naive-fork | (B) Full-replay | (C) Groundhog | |
|---|---|---|---|
| advantage-estimator variance | 4.29 | 0.0011 | 0.147 |
| bias vs the live world | none | −0.197 | none |
| rollouts to a stable correct answer | not within 800 | not within 800 | 182 |
| first "significant" answer | 38 — wrong sign | 10 — wrong sign | 178 — correct |
| converges to | verify (3/5 seeds) |
deep — wrong |
verify (5/5 seeds) |
| reward identical to the live world | is the live world | no | 100/100 exact |
| live API calls per group of 4 | 40 | 17 | 19 |
(B) wins on variance and is useless. It has the quietest estimator of the three and it converges, confidently, to the wrong policy — because it tells each branch that the write it just made never happened. That trap is the reason this repository exists: low variance is not the objective. Low variance on an unbiased estimator is.
And note the second row from the bottom. Condition (C) does not merely have less error than the live world — for a fixed world seed it reproduces a live rollout bit for bit, across 25 worlds × 4 actions, while spending 1–3 API calls instead of 6–14.
Both baselines will also hand you a confident wrong answer if you early-stop: (A) crosses significance at 38 rollouts with the sign inverted, purely from variance.
A group-relative advantage is a_i = r_i − mean_j(r_j). Write each reward as
r_i = q(action_i) + ε_exo,i + ε_endo,i. If the branches share their exogenous draw then
ε_exo,i = ε_exo for all i, and it cancels identically in r_i − mean(r). What is
left is the action quality you wanted plus the policy's own sampling noise. If the branches
do not share it — which is what happens when you snapshot state but re-query a live world —
then ε_exo survives into every advantage, and you pay for it at the rate
(σ_exo/gap)² in rollouts.
In the demo σ_exo/gap ≈ 9, so naive forking needs roughly two orders of magnitude more
rollouts to resolve the same comparison. That is not a property of the demo; it is the
arithmetic of common random numbers, and the demo is calibrated to make it visible.
flowchart TB
subgraph agent["agent rollout (branch root/b2)"]
A1["requests.get('/search?q=solar')"]
A2["time.time() · uuid4() · random.random()"]
A3["POST /docs (a mutation)"]
end
subgraph icept["① interceptors.py — close every exogenous channel"]
I1["HTTPAdapter.send / HTTPTransport.handle_request"]
I2["time · datetime · random · numpy.random · uuid"]
I3["os.environ · os.stat · getmtime"]
end
subgraph core["② noise_decomp.py — one resolver, three policies"]
D1{"causal fingerprint<br/>of this read"}
D2["fp == ''<br/>nothing I did affects this"]
D3["fp ≠ ''<br/>I changed a dependency"]
end
subgraph det["③ divergence.py"]
E1["resource versions<br/>+ dependency model"]
E2["which recorded reads<br/>did my write invalidate?"]
end
subgraph tape["④ tape.py + vclock.py"]
T1[("WorldTape<br/>JSONL + blobs<br/>keyed by (seed, branch, step)<br/>addressed by (key, occurrence)")]
T2["VirtualClock<br/>sleep() is free"]
end
W["live world<br/>(mock API / real service)"]
A1 --> I1
A2 --> I2
A3 --> I1
I1 --> D1
I2 --> T2
I3 --> T1
E1 --> D1
A3 -.->|"bump versions"| E1
E1 --> E2
D1 --> D2 --> T1
D1 --> D3
D3 -->|"re-render locally with the<br/>SAME shared noise key"| S["coherent synthesis<br/>(the coupled counterfactual)"]
D3 -->|"no local model of<br/>this endpoint"| W
D2 -.->|"identical bytes in every sibling<br/>= common random numbers"| V["variance cancels in<br/>r_i − mean(r)"]
T1 -.->|"record once"| W
style D2 fill:#1b9e8a,color:#fff
style D3 fill:#e8743b,color:#fff
style V fill:#1b9e8a,color:#fff
style S fill:#1b9e8a,color:#fff
Every observation factorises as response = render(world_state, exogenous_noise). State is
what the agent can change; noise is what the world does regardless. So the rule is not
"replay the response or don't" but:
keep the exogenous noise fixed across the fork group, and re-render it against each branch's own state.
Concretely, every exogenous value is addressed by a content key rather than stored as a stream:
value = F(world_seed, norm_key, occurrence, causal_fingerprint)
F is a keyed hash. Two branches that ask the same question get the same answer with no
coordination, whatever order they got there in. causal_fingerprint hashes only the
resource versions that branch itself mutated — so it is empty exactly when sharing is
sound, and changes exactly when it is not.
This is why derivation beats a recorded stream: a stream desynchronises the moment two branches read in different orders, and after a fork they always do.
The three benchmark conditions are three settings of one resolver:
| mode | exogenous noise | state after a mutation |
|---|---|---|
naive (A) |
fresh per branch | correct (it queries the live world) |
replay (B) |
shared | stale — the world lies |
groundhog (C) |
shared | re-rendered, or refetched and counted |
| module | what it does |
|---|---|
groundhog/tape.py |
Append-only JSONL + content-addressed blobs, keyed by (world_seed, branch_id, step). lookup() implements fork inheritance: an ancestor's event is inherited if it is in the shared prefix or classified action-independent. |
groundhog/noise_decomp.py |
The resolver. Splits each read into shared-or-resampled via the causal fingerprint, and counts everything the metrics need. |
groundhog/divergence.py |
Resource versioning + three dependency models. Answers "which recorded reads did this mutation invalidate?" and is scored on precision/recall against environment ground truth. |
groundhog/interceptors.py |
Closes the channels: requests/httpx at the transport layer, time/datetime, random/numpy.random, uuid, os.environ, file mtimes. Context-scoped, so the in-process mock world still sees a real clock. |
groundhog/vclock.py |
Virtual clock. sleep() advances the timeline and blocks for zero real time. |
groundhog/verifiers_adapter.py |
with_groundhog(env) and GroupRunner. Duck-typed, so verifiers is optional. |
groundhog/claims.py |
The four claims, what measures each, and a registry of the 92 symbols that carry them. groundhog info --claims audits that every one says so in its docstring; tests/test_claims.py enforces it. |
python -m benchmarks.run # ~3 min, writes figures/ and benchmark.json
python -m benchmarks.run --quick # ~15 s smoke test400 fork groups per condition, 4 branches each; bias reference from 1200 independent
live-world rollout pairs. Regenerate with python -m benchmarks.run.
| metric | (A) Naive-fork | (B) Full-replay | (C) Groundhog |
|---|---|---|---|
| (1) advantage contrast variance | 4.2867 | 0.0011 | 0.1466 |
| variance reduction vs (A) | 1.0x | 3929x | 29x |
| group advantage variance | 1.8126 | 0.0069 | 0.1451 |
| (2) rollouts to a stable correct answer | not within 800 | not within 800 | 182 |
| first significant answer (early-stop trap) | 38 (wrong sign!) | 10 (wrong sign!) | 178 |
| rollouts to 90% mass on best action | 1456 (3/5 seeds) | never | 1024 (5/5 seeds) |
| policy converged to | deep, verify |
deep, guess |
verify |
| (3) estimated contrast (truth +0.147+-0.017) | +0.107 | -0.050 | +0.127 |
| bias vs live world | -0.040 +-0.204 | -0.197 +-0.018 | -0.020 +-0.041 |
| verdict | unbiased | biased | unbiased |
| reward identical to live world, same seed | is the live world | no (see lie rate) | 100/100 exact |
| world lied (stale reads served) | 0.0% | 26.5% | 0.0% |
| agent blind to its own write | 0.0% | 100.0% | 0.0% |
| (4) reads shared across group | 0.0% | 100.0% | 71.4% |
| (5) live API calls / group of 4 | 40.0 | 17.3 | 19.3 |
| of which recording the prefix | 0.0 | 13.3 | 13.3 |
| wall seconds | 7.9 | 6.3 | 6.7 |
Read the table with metric (1) and metric (3) together. Separately, each one has a winner that is wrong.
Deliberately hostile to replay (envs/), with all four requested sources of
exogenous nondeterminism:
| source | how |
|---|---|
| time-dependent responses | GET /context returns the world's clock, day bucket, and a reader-attention shock |
| randomly-ranked search | GET /search reorders equally-relevant hits per world instance |
| writable filesystem | envs/vfs.py — the agent drafts to real files; mtimes come from the virtual clock, and each write is a mutation of fs:notes.md |
| latency jitter | lognormal per-call latency, entering reward as a cost |
It is a real ThreadingHTTPServer on an ephemeral port, not a stub — recording goes over
TCP, so the transport-level interception is genuinely exercised; replay never starts the
server at all. The agent picks one of four strategies:
| action | behaviour | true rank |
|---|---|---|
guess |
publish without reading | 4th |
skim |
search, read top-1, publish | 3rd |
deep |
search, read top-3, publish | 2nd |
verify |
search, read top-3, publish, then read the review of its own brief and revise | 1st |
verify wins by only ~0.16 reward against an exogenous shock with σ ≈ 1.4 — and it wins
only if it can see its own write. Under full replay that read returns the pre-publish
answer (exists: false), so the strategy's whole advantage evaporates and deep looks
better. Reward is computed only from what the agent observed, which is the mechanism by
which a lying replay produces a biased gradient rather than a cosmetic inconsistency.
(1) Variance — the variance across worlds of the verify − deep contrast, at one
group per world, i.e. equal budget for all three arms. Asserted with Levene's test, not by
eyeball (tests/test_variance.py).
(2) Sample efficiency — reported optimiser-free as the sample size at which the
contrast becomes significantly non-zero and stays that way. A policy-gradient loop's
time-to-target depends on the learning rate; at a large lr a high-variance estimator can
cross the target by random walk. The GRPO learning curves are reported too, over 5 seeds
with a sustained-attainment criterion, and they understate the estimator gap — most of
that learning problem is eliminating guess and skim, which is easy for every condition.
(3) Fidelity — three ways, weakest to strongest:
lie_rate: reads served from tape whose causal premise had already changed.bias: against a large-sample live-world reference, with a control variate on the observed attention draw (whose reward coefficient is known and whose mean is exactly zero) to sharpen the test by ~10×. The reference is allowed privileged knowledge; the three conditions under test are not, and none of them use it.- exact equivalence: for a fixed
world_seedandagent_seed, does condition (C) reproduce a live rollout bit-for-bit? Not on average — identically. 100/100 rollouts across 25 worlds × 4 actions, max |diff| = 0. Any non-zero difference would be a bug with a name: a stale share, a branch-keyed exogenous draw, or renderer drift.
(4) Reproducibility — record the same world twice, compare events.jsonl byte for byte.
The tape contains no host-derived value: no real timestamps, no uuid4, no host entropy.
20/20 byte-identical.
(5) API reduction — counted at the server, and reported as a curve against group size
rather than as a single percentage, because recording the prefix is a fixed cost that makes
(C) worse than (A) for a group of one. Per rollout the reduction is large (deep: 11
live calls → 1). Per group it depends on how many branches amortise the recording.
| model | precision | recall | how it fails |
|---|---|---|---|
naive_path |
1.00 | 0.46 | Watches phantom resources (corpus:index, corpus:stats) that nothing ever writes to, because search and aggregates are views over documents. So it never sees that a publication changed them: the world lies. |
causal_closure |
0.68 | 1.00 | Depends on every document plus set membership. Recall is perfect, but publishing a solar brief now invalidates the recorded climate search: wasted resampling and refetches. |
causal_relevance |
0.84 | 1.00 | Scopes dependencies to the query. Keeps recall, recovers most of the precision. Not all of it — whether a matching document reaches the top-k depends on the ranking, which no dependency model simulates. |
Ground truth needs no labels: every endpoint in envs/corpus.py is a pure
function, so "would this recorded read return something different now?" is answered exactly
by re-rendering it against the post-mutation state. That single design choice is what makes
metric (3) measurable instead of asserted.
pip install -e ".[all]"
groundhog record research --tape tapes/demo --world-seed 7 # spends 13 API calls
groundhog replay tapes/demo --action verify --teach-trends # spends 0, reproduces exactly
groundhog fork tapes/demo --branches 4 # 4 branches, one shared world
groundhog verify tapes/demo # byte-identical re-recording
groundhog inspect tapes/demo --events
python app.py # the replay viewergroundhog record is the only command that needs the world to exist. Everything else
runs from the tape directory alone — no server, no network, no credentials. The offline
replay test in tests/test_reproducibility.py shuts the
server down and points the environment at a closed port, so a stray socket raises instead of
quietly succeeding.
The hosted viewer is a
static Space: Hugging Face requires a PRO subscription to host interactive Gradio Spaces on
free CPU, so scripts/build_static_space.py precomputes 24
scenarios through the same app.render() the interactive version uses and ships them in a
single HTML file. python app.py gives you the live, fully interactive version locally.
from groundhog import GroupRunner
runner = GroupRunner(env_factory=make_env, world_seed=step, group_size=8, tape="tapes/w")
group = runner.run(lambda env, session, i: rollout(env, session, policy))
advantages = group.advantages # exogenous shock already cancelledGroupRunner forks group_size branches off one tape prefix. Exogenous draws are common to
the group; exploration noise is not — session.agent_rng() is keyed by branch on
purpose, because forcing every branch to sample the same action would destroy the very
exploration a branching search exists to do. Groundhog removes environment noise, not policy
noise, and the benchmark reports both so the distinction stays visible.
python scripts/record_tapes.py --out tapes/ # dry run
python scripts/record_tapes.py --push-to-hub USER/groundhog-world-tapesRefuses to publish a dataset that does not replay offline: it shuts the world down and
replays every tape against a closed port first. (That guard has already caught one real bug
in this repo.) --scrub drops the env channel before pushing.
Groundhog deliberately does not reimplement:
- sandbox snapshot / rollback (DeltaBox-style). World state is the sandbox's job.
Groundhog assumes it and controls only exogenous noise. In the demo, per-branch server
overlays plus
POST /_resetstand in for it. - branch rollout search (Branching Policy Optimization-style). Which branches to expand is the search's job. Groundhog is the layer beneath: it makes the branches comparable.
It is the thin layer those two need under them and neither provides.
- The variance gain is proportional to the exogenous share of reward variance. Where reward is dominated by the policy's own sampling, common random numbers buy little. The measured 29× sits below its 58× ceiling for a legitimate reason — part of the true contrast genuinely varies by world (reviewer strictness multiplies brief quality), and no coupling removes variation in the estimand itself.
- Synthesis needs a local model of the endpoint.
/trendsdeliberately has none, so Groundhog refetches it and says so (refetched), or approximates it and says that too (approximated). Pretending to synthesise an endpoint you do not understand is the failure mode this project criticises.--teach-trendsis the ablation. from time import timebefore installation keeps the real function. Module-attribute patching cannot reach an already-bound name. Import modules, not names, inside recorded code.- Only the curated
numpy.randomlegacy API is intercepted. Anything else keeps numpy's own behaviour and shows up as a missingrandomevent inreport()— visible rather than silent. - The prefix probe is a benchmark convenience. It reads each key as many times as the deepest branch will, which looks odd for an agent and is exactly right for a prefetcher. In production the group's first branch can record as it runs.
- The demo is calibrated to make the effect visible. σ_exo/gap ≈ 9 was chosen, not
discovered. The mechanism is general; the magnitude in your environment depends on your
noise decomposition, which
report()will tell you.
Everything here is recorded from the bundled mock world in envs/ — no real
third-party service is contacted, and no credentials or personal data are recorded.
Recording a real service with these interceptors is possible and is your responsibility. Check that provider's terms first. Do not publish tapes containing credentials, personal data, or content you have no right to redistribute. The interceptors capture HTTP response bodies verbatim.
pytest -q # 64 tests, ~9 s| file | asserts |
|---|---|
test_reproducibility.py |
byte-identical same-seed tapes; every declared channel actually intercepted; no host clock on the tape; offline replay with the server stopped; keyed-noise injectivity |
test_fork_sharing.py |
siblings observe identical action-independent reads; sharing survives divergent read order; the naive baseline genuinely does not share; endogenous noise stays per-branch |
test_divergence.py |
fingerprints empty until mutation and unaffected by irrelevant writes; (C) reproduces the live world exactly; (B) demonstrably lies; causal models reach recall 1.0 and relevance beats closure on precision |
test_variance.py |
(C)'s variance significantly below (A)'s (Levene, p < 0.01); the ratio does not exceed its endogenous ceiling — which would mean sharing had leaked; (B) confidently reaches the wrong sign |
test_adapter_and_cli.py |
with_groundhog on an environment that has never heard of Groundhog, sync and async; CLI record→replay→verify→fork; replay refuses to silently go online |
test_claims.py |
every load-bearing function's docstring names the claim it demonstrates, and every claim has demonstrators in more than one module — see groundhog/claims.py |
Apache-2.0. A research prototype: the mechanism is the contribution, the numbers are reproducible, and the limitations above are part of the result.