Fetch the real-data test fixture from figshare instead of running fMRIPrep locally - #1
Merged
kencan7749 merged 1 commit intoAug 18, 2026
Conversation
Running tests/mri/fmriprep/test_fmriprep_real.py previously required each developer to install the raw OpenNeuro dataset with datalad, run FreeSurfer recon-all, and run fMRIPrep 1.2.1 in Docker, then build the golden master locally. This replaces that with downloading a small pre-processed fixture, as discussed in PR KamitaniLab#116. The fixture is published on figshare (doi:10.6084/m9.figshare.32857559.v1, CC BY 4.0) as ds006319_fmriprep-1.2.1_minimal.tar.gz: 1.77 GB, about 2.9 GB unpacked, derived from OpenNeuro ds006319 v1.0.1 (CC0). It contains only what the test reads, which is the T1w-space preproc BOLD and confounds for runs 02 and 03 of sub-S1/ses-SoundTest02, the raw events.tsv and bold.json, an empty ses-SoundTest01 that holds session index 0, the expected BData output, and a pinned stimulus_name label mapper. - Add scripts/real/step_1_figshare_download.sh. It downloads the archive, verifies its md5 before extracting so a truncated download fails loudly rather than as a confusing test failure, and extracts it under tests/data/mri/. FIGSHARE_LOCAL_TARBALL points it at a locally rebuilt archive instead. The download stays explicit: nothing is fetched during a normal pytest run, and the test still skips itself when the fixture is absent. - Rename scripts/real/step_5_run_test.sh to step_2_run_test.sh, so the real-data workflow is the same two steps as the mock one. - Move the pipeline scripts (step_1_download.sh, step_2_run_freesurfer.sh, step_3_run_fmriprep.sh, step_4_prepare_gm.sh, _fs_env.sh.example and its .gitignore) to scripts/fixture_generation/. They are no longer part of the test workflow but are still needed to reproduce the fixture, for example after a change to exclude, to the session layout, or to data_mode. - Add scripts/fixture_generation/step_5_build_minimal_bundle.sh, which cuts the minimal subset out of the full fMRIPrep outputs and writes the archive that is published. Without it the published fixture is not reproducible from this repository, only the 60 GB intermediate outputs are. It strips the BData header first: BData.save() records the call stack, including absolute paths, into /header, which must not be published. - Add scripts/fixture_generation/clean_bdata_header.py and gen_label_mapper.py supporting that step, and a README documenting requirements, the step order, and which files the subset keeps and why. - Update tests/mri/fmriprep/README.md: the Real-Data section is now two steps, with a fixture table giving the archive name, version, DOI, md5, size, source dataset, fMRIPrep and FreeSurfer versions, Docker image tag, generation command, and expected directory layout, as requested in the PR discussion. Update TEST_COVERAGE.md accordingly. - Ignore the downloaded fixture: tests/data/mri/ds006319/ and the label mapper next to the real golden master. Verification on the lab base env (Python 3.8.5): - step_1_figshare_download.sh against the published article: md5 matched and the archive extracted. - pytest tests/mri/fmriprep/test_fmriprep_real.py => 1 passed in 101.75s - pytest tests/mri/fmriprep/ -m "not real_data" => 36 passed, 1 deselected - git status stayed clean: the 478 MB dataset and 2.4 GB golden master are both ignored. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This targets
ks_update_fmriprepso it lands inside KamitaniLab#116 rather than as a separate PR. It implements the real-data fixture workflow discussed there: the fixture is now downloaded from figshare instead of being regenerated by each developer.I have uploaded the pre-processed data as offered in my earlier comment, and it is now published:
doi:10.6084/m9.figshare.32857559.v1 —
ds006319_fmriprep-1.2.1_minimal.tar.gz, 1,769,324,197 bytes, md57f4ec67fb6e73239d3e5dc1066ef8f55, CC BY 4.0.@kencan7749 If you have already started on this yourself, please feel free to close this and take only the parts that are useful. I did not want to hold you up with another round of questions.
What changes
scripts/real/step_1_figshare_download.sh. Downloads the archive, verifies its md5 before extracting, and places it undertests/data/mri/. A truncated download therefore fails loudly instead of surfacing as a confusing golden-master mismatch.FIGSHARE_LOCAL_TARBALL=/path/to.tar.gzpoints it at a locally rebuilt archive instead of the published one.scripts/real/step_5_run_test.shtostep_2_run_test.sh, so the real-data workflow is the same two steps as the mock one. Contents unchanged.scripts/fixture_generation/(step_1_download.sh,step_2_run_freesurfer.sh,step_3_run_fmriprep.sh,step_4_prepare_gm.sh,_fs_env.sh.exampleand its.gitignore). These are moves, not deletions: they are no longer part of the test workflow, but they are still needed to reproduce the fixture, and the README has to document the generation command.scripts/fixture_generation/step_5_build_minimal_bundle.shplusclean_bdata_header.pyandgen_label_mapper.py. Without these, what this repository can reproduce stops at the ~60 GB intermediate outputs, not the published archive. The build step also strips the BData header, becauseBData.save()records the call stack, including absolute paths, into/header, which must not be published.README.mdandTEST_COVERAGE.md, and ignore the downloaded fixture.Nothing in
test_fmriprep_real.pyortest_fmriprep_utils.pyis modified.RealDatasetMixinalready resolvestests/data/mri/ds006319, the golden master and the label mapper, and it still skips itself when the fixture is absent.Addressing the review comments
@micchu's comment:
real_datatestsstep_1_figshare_download.shis run manually. Nothing is fetched during a normalpytestrun, so the default suite stays offlineds006319/and the label mapper are now gitignoredtests/mri/fmriprep/README.mdds006319_fmriprep-1.2.1_minimal.tar.gz*_events.tsvand*_bold.jsonexcludeis exercisedsub-S1withses-SoundTest01(fully excluded) andses-SoundTest02(run-01 excluded, run-02 and run-03 loaded), matchingexclude={"session/run": [[1, 2, 3, 4, 5], [1]]}Two things I did not do, deliberately:
tar.gzrather thanzip. Happy to rebuild and republish aszipif you prefer.real_datamarkeraddoptspoint is left alone, since it belongs to this PR rather than to the fixture work.If you would rather have the download logic in Python than in a shell script, say so and I will convert it.
Fixture contents
Only what the test reads is included: 478 MB of inputs and a 2.4 GB golden master, down from the ~60 GB the pipeline produces. The pinned label mapper is what makes this possible; without it
RealDatasetMixinrebuilds the mapping at run time by scanning all 560 rawevents.tsvfiles, which would mean shipping the whole dataset. The source data are derived from OpenNeurods006319v1.0.1 (CC0), functional EPI only, with no anatomical images.Verification
On the lab base environment (Python 3.8.5), against the published article rather than a local copy:
git statusstays clean afterwards: the 478 MB dataset and the 2.4 GB golden master are both ignored.