test: Add pytest infrastructure and regression tests for the sklearn Ridge decoder - #6
Open
KenyaOtsuka wants to merge 2 commits into
Open
Conversation
…Ridge decoder The repository had no automated tests, which made it unsafe to change the decoding implementation. This adds the first test suite, pinning the current behavior of `train_decoder_sklearn_ridge.py` and `predict_feature.py`. - Add `scikit-learn` to the project dependencies. It is imported by `train_decoder_sklearn_ridge.py` but was never declared, so `uv sync` produced an environment that could not run the script. - Add a `dev` dependency group with pytest and `[tool.pytest.ini_options]`, including `pythonpath = ["."]` so the top-level analysis modules are importable from the tests without any `sys.path` manipulation. - `tests/helpers/synthetic.py`: build tiny `BData` / `Features` datasets on disk, with repeated stimuli (unbalanced repeats), two ROIs, a subject split over two `.h5` files, and both a 2-D `fc`-like and a 3-D `conv`-like feature layer so the chunking / `order='F'` path is exercised. - `tests/helpers/legacy_ridge.py`: a plain numpy + scikit-learn reference implementation of the direct brain -> feature Ridge decoder, mirroring bdpy's `ModelTraining` / `ModelTest` semantics exactly. - `tests/generate_golden.py` and `tests/data/golden/`: regression fixtures recording the pipeline output on a fixed-seed synthetic dataset. Run it as `python -m tests.generate_golden`. - Tests covering the artifact contract, decoded-feature output format, normalization parameters, label alignment, chunking, and the behavior of feature units with zero training variance. - Add a GitHub Actions workflow running the suite on push and pull requests. No change to the decoding implementation.
KenyaOtsuka
marked this pull request as ready for review
September 9, 2026 14:24
KenyaOtsuka
requested review from
HirokiYasuda03 and
ganow
and
a lite review from Copilot
September 9, 2026 14:24
There was a problem hiding this comment.
🟡 Changes recommended
A couple of tests load .npz goldens via np.load(...) without closing the returned NpzFile, which can leak file descriptors across the test run.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a first automated test/CI baseline for the scikit-learn Ridge decoding pipeline, including synthetic fixtures and golden regression outputs, so future refactors can be validated without changing the current decoder behavior.
Changes:
- Add
pytest+scikit-learndependencies and configure pytest import behavior viapythonpath. - Introduce a synthetic on-disk dataset generator plus a reference (legacy) Ridge implementation used to pin current semantics.
- Add GitHub Actions workflow to run the test suite on pushes and pull requests.
File summaries
| File | Description |
|---|---|
uv.lock |
Locks new runtime/dev dependencies (scikit-learn, pytest and transitive deps). |
pyproject.toml |
Adds scikit-learn, defines dev dependency group, and sets pytest configuration. |
.github/workflows/test.yml |
Runs uv sync --group dev and pytest on CI (Python 3.11). |
README.md |
Documents how to run tests and how to regenerate golden fixtures. |
.gitignore |
Ignores Python bytecode caches and pytest cache. |
tests/conftest.py |
Adds shared fixtures and ensures each test runs in an isolated working directory. |
tests/generate_golden.py |
Script to regenerate the committed golden .npz outputs. |
tests/helpers/synthetic.py |
Generates small synthetic BData + Features trees to exercise decoder edge cases. |
tests/helpers/pipeline.py |
Calls the real training/prediction scripts and provides helpers to read outputs. |
tests/helpers/legacy_ridge.py |
Reference numpy/sklearn implementation to match current scripts and support regression tests. |
tests/test_pipeline_golden.py |
End-to-end golden regression test for predictions + normalization artifacts. |
tests/test_legacy_reference.py |
Validates the reference implementation against scripts and goldens. |
tests/test_predict_feature.py |
Pins decoded-feature output layout and behavior for predict_feature. |
tests/test_train_decoder_sklearn_ridge.py |
Pins decoder artifact layout, normalization definitions, and rerun-skipping behavior. |
tests/test_chunking.py |
Pins chunking behavior and Fortran-order reshape semantics. |
tests/test_label_alignment.py |
Pins label alignment and feature expansion semantics used during training. |
tests/test_zero_variance_features.py |
Pins current failure mode for zero-variance feature units. |
Review details
- Files reviewed: 15/20 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The repository had no automated tests, making changes to the sklearn Ridge
decoder difficult to validate.
This PR adds a regression test suite that pins the current behavior before a
follow-up refactor of the decoder. The decoding implementation itself is
unchanged.
What
scikit-learndependency andpytestdevelopment setup.multiple ROIs, and 2-D / 3-D feature layers.
chunking / Fortran-order handling, output layout, and edge cases.
Details for reviewers
pythonpath = ["."]so the repository's top-level analysismodules are importable without modifying
sys.path.statistics are distinguishable.
.h5files to exercise the existingmulti-BData paths.
scikit-learn Ridge semantics, including normalization, label expansion,
chunking,
order='F'reshaping, and float32 behavior.slightly across BLAS implementations.
C-vs-Fortran reshape cannot pass accidentally.
evaluation.pyreadsnormalization files from that layout.
python -m tests.generate_golden, and only whenthe expected numerical output intentionally changes.
relative DistComp SQLite database under
./tmp/.Test
uv sync --group dev uv run pytest # 32 passed