Skip to content

Make inference sampling independent of worker scheduling - #139

Open
Essoz wants to merge 1 commit into
daniel/parallel-inferfrom
codex/deterministic-inference
Open

Essoz wants to merge 1 commit into
daniel/parallel-inferfrom
codex/deterministic-inference

Conversation

@Essoz

@Essoz Essoz commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

Process-based inference previously shared or inherited mutable RNG state. Changing worker count or task scheduling could therefore change sampled evidence and support counts, even for the same saved traces. This gives generation, cross-trace collection, and precondition tasks deterministic RNG streams keyed by their semantic identity. Serial and process paths use the same scopes. Precondition downsampling keeps its existing seed-42 sample choices without reseeding the application's global RNG.

Targets daniel/parallel-infer so this PR contains only the sampling fix and its regression tests. Sampling rates and caps are unchanged; moving to task streams can change the selected subset once compared with the previous serial implementation.

Validation:

  • All 27 saved-trace runs agreed on full invariant and failed-hypothesis JSON within each workload: MNIST, Transformer, and DCGAN × 1/2/4 workers × three repetitions. Comparison includes support counts and duplicate multiplicity; only object-key and file-line order are ignored.
  • 11 Linux regression tests pass on the final PR files, including actual containment sampling with 1,005 nested calls per trace, fork and spawn, reversed/repeated dispatch, task/thread isolation, frozen exceptions, stable digest seeds, and exact legacy precondition samples above 5,000 examples.
  • Commit hooks for Ruff, Mypy, Black, and isort pass, as does git diff --check.
  • Old serial versus fixed: predicates and failed hypotheses are unchanged on all three workloads. One positive-support count changes from 20 to 21 in Transformer and one in DCGAN; each is consistent across all patched runs.

The real-workload matrix used Python 3.10.16, PyTorch 2.2.2, pandas 2.3.3, fork, and fixed PYTHONHASHSEED=0. This establishes agreement for the tested inputs and worker counts; it does not claim whole-engine determinism across Python versions, hash seeds, or reordered inputs. Existing set-population ordering and reservoir merge/cap defects remain separate. Two macOS subprocess cases hit OpenMP shared-memory initialization failures; the complete suite passed on Linux.

Reproduce the regression suite:

PYTHONDONTWRITEBYTECODE=1 OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 \
  python -m unittest discover -s tests -p test_inference_sampling.py -v

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant