Skip to content

Fetch the real-data test fixture from figshare instead of running fMRIPrep locally - #1

Merged
kencan7749 merged 1 commit into
kencan7749:ks_update_fmriprepfrom
izpyon:figshare-real-data-fixture
Aug 18, 2026
Merged

Fetch the real-data test fixture from figshare instead of running fMRIPrep locally#1
kencan7749 merged 1 commit into
kencan7749:ks_update_fmriprepfrom
izpyon:figshare-real-data-fixture

Conversation

@izpyon

@izpyon izpyon commented Aug 12, 2026

Copy link
Copy Markdown

This targets ks_update_fmriprep so it lands inside KamitaniLab#116 rather than as a separate PR. It implements the real-data fixture workflow discussed there: the fixture is now downloaded from figshare instead of being regenerated by each developer.

I have uploaded the pre-processed data as offered in my earlier comment, and it is now published:

doi:10.6084/m9.figshare.32857559.v1ds006319_fmriprep-1.2.1_minimal.tar.gz, 1,769,324,197 bytes, md5 7f4ec67fb6e73239d3e5dc1066ef8f55, CC BY 4.0.

@kencan7749 If you have already started on this yourself, please feel free to close this and take only the parts that are useful. I did not want to hold you up with another round of questions.

What changes

  • Add scripts/real/step_1_figshare_download.sh. Downloads the archive, verifies its md5 before extracting, and places it under tests/data/mri/. A truncated download therefore fails loudly instead of surfacing as a confusing golden-master mismatch. FIGSHARE_LOCAL_TARBALL=/path/to.tar.gz points it at a locally rebuilt archive instead of the published one.
  • Rename scripts/real/step_5_run_test.sh to step_2_run_test.sh, so the real-data workflow is the same two steps as the mock one. Contents unchanged.
  • Move the pipeline scripts to scripts/fixture_generation/ (step_1_download.sh, step_2_run_freesurfer.sh, step_3_run_fmriprep.sh, step_4_prepare_gm.sh, _fs_env.sh.example and its .gitignore). These are moves, not deletions: they are no longer part of the test workflow, but they are still needed to reproduce the fixture, and the README has to document the generation command.
  • Add scripts/fixture_generation/step_5_build_minimal_bundle.sh plus clean_bdata_header.py and gen_label_mapper.py. Without these, what this repository can reproduce stops at the ~60 GB intermediate outputs, not the published archive. The build step also strips the BData header, because BData.save() records the call stack, including absolute paths, into /header, which must not be published.
  • Update README.md and TEST_COVERAGE.md, and ignore the downloaded fixture.

Nothing in test_fmriprep_real.py or test_fmriprep_utils.py is modified. RealDatasetMixin already resolves tests/data/mri/ds006319, the golden master and the label mapper, and it still skips itself when the fixture is absent.

Addressing the review comments

@micchu's comment:

Request Where
Host the fixture externally and download it explicitly only for real_data tests figshare; step_1_figshare_download.sh is run manually. Nothing is fetched during a normal pytest run, so the default suite stays offline
Repository holds test code and download / verification scripts, not the binaries Only scripts are added; ds006319/ and the label mapper are now gitignored
Document fixture version, checksum, source dataset, fMRIPrep / FreeSurfer versions, Docker image tag, generation command, and expected directory structure Fixture table in tests/mri/fmriprep/README.md
Make the archive filename identify the fMRIPrep version ds006319_fmriprep-1.2.1_minimal.tar.gz
Include *_events.tsv and *_bold.json Included for runs 01–03
Single subject with multiple sessions / runs so exclude is exercised sub-S1 with ses-SoundTest01 (fully excluded) and ses-SoundTest02 (run-01 excluded, run-02 and run-03 loaded), matching exclude={"session/run": [[1, 2, 3, 4, 5], [1]]}
Workflow should only download, extract and verify the checksum, with as little shell as possible The real-data workflow went from five scripts to two, and the fMRIPrep execution phase is out of it

Two things I did not do, deliberately:

  • The archive is tar.gz rather than zip. Happy to rebuild and republish as zip if you prefer.
  • The real_data marker addopts point is left alone, since it belongs to this PR rather than to the fixture work.

If you would rather have the download logic in Python than in a shell script, say so and I will convert it.

Fixture contents

tests/data/mri/
  ds006319/
    sub-S1/ses-SoundTest02/func/          run-01..03 events.tsv and bold.json
    derivatives/fmriprep/fmriprep/sub-S1/
      ses-SoundTest01/func/               empty, holds session index 0
      ses-SoundTest02/func/               run-01 confounds; run-02/03 space-T1w_desc-preproc_bold and confounds
  golden_master/real/
    test_output_fmriprep_real_exclude.h5  expected output
    temp.tsv                              pinned stimulus_name label mapper

Only what the test reads is included: 478 MB of inputs and a 2.4 GB golden master, down from the ~60 GB the pipeline produces. The pinned label mapper is what makes this possible; without it RealDatasetMixin rebuilds the mapping at run time by scanning all 560 raw events.tsv files, which would mean shipping the whole dataset. The source data are derived from OpenNeuro ds006319 v1.0.1 (CC0), functional EPI only, with no anatomical images.

Verification

On the lab base environment (Python 3.8.5), against the published article rather than a local copy:

$ bash ./tests/mri/fmriprep/scripts/real/step_1_figshare_download.sh
md5 OK: 7f4ec67fb6e73239d3e5dc1066ef8f55

$ bash ./tests/mri/fmriprep/scripts/real/step_2_run_test.sh
tests/mri/fmriprep/test_fmriprep_real.py .                               [100%]
================== 1 passed, 43 warnings in 101.75s (0:01:41) ==================

$ python -m pytest tests/mri/fmriprep/ -m "not real_data" -q
36 passed, 1 deselected, 50 warnings in 3.84s

git status stays clean afterwards: the 478 MB dataset and the 2.4 GB golden master are both ignored.

Running tests/mri/fmriprep/test_fmriprep_real.py previously required each
developer to install the raw OpenNeuro dataset with datalad, run FreeSurfer
recon-all, and run fMRIPrep 1.2.1 in Docker, then build the golden master
locally. This replaces that with downloading a small pre-processed fixture,
as discussed in PR KamitaniLab#116.

The fixture is published on figshare (doi:10.6084/m9.figshare.32857559.v1,
CC BY 4.0) as ds006319_fmriprep-1.2.1_minimal.tar.gz: 1.77 GB, about 2.9 GB
unpacked, derived from OpenNeuro ds006319 v1.0.1 (CC0). It contains only what
the test reads, which is the T1w-space preproc BOLD and confounds for runs
02 and 03 of sub-S1/ses-SoundTest02, the raw events.tsv and bold.json, an
empty ses-SoundTest01 that holds session index 0, the expected BData output,
and a pinned stimulus_name label mapper.

- Add scripts/real/step_1_figshare_download.sh. It downloads the archive,
  verifies its md5 before extracting so a truncated download fails loudly
  rather than as a confusing test failure, and extracts it under
  tests/data/mri/. FIGSHARE_LOCAL_TARBALL points it at a locally rebuilt
  archive instead. The download stays explicit: nothing is fetched during a
  normal pytest run, and the test still skips itself when the fixture is
  absent.

- Rename scripts/real/step_5_run_test.sh to step_2_run_test.sh, so the
  real-data workflow is the same two steps as the mock one.

- Move the pipeline scripts (step_1_download.sh, step_2_run_freesurfer.sh,
  step_3_run_fmriprep.sh, step_4_prepare_gm.sh, _fs_env.sh.example and its
  .gitignore) to scripts/fixture_generation/. They are no longer part of the
  test workflow but are still needed to reproduce the fixture, for example
  after a change to exclude, to the session layout, or to data_mode.

- Add scripts/fixture_generation/step_5_build_minimal_bundle.sh, which cuts
  the minimal subset out of the full fMRIPrep outputs and writes the archive
  that is published. Without it the published fixture is not reproducible
  from this repository, only the 60 GB intermediate outputs are. It strips
  the BData header first: BData.save() records the call stack, including
  absolute paths, into /header, which must not be published.

- Add scripts/fixture_generation/clean_bdata_header.py and gen_label_mapper.py
  supporting that step, and a README documenting requirements, the step
  order, and which files the subset keeps and why.

- Update tests/mri/fmriprep/README.md: the Real-Data section is now two steps,
  with a fixture table giving the archive name, version, DOI, md5, size,
  source dataset, fMRIPrep and FreeSurfer versions, Docker image tag,
  generation command, and expected directory layout, as requested in the PR
  discussion. Update TEST_COVERAGE.md accordingly.

- Ignore the downloaded fixture: tests/data/mri/ds006319/ and the label
  mapper next to the real golden master.

Verification on the lab base env (Python 3.8.5):

- step_1_figshare_download.sh against the published article: md5 matched and
  the archive extracted.
- pytest tests/mri/fmriprep/test_fmriprep_real.py => 1 passed in 101.75s
- pytest tests/mri/fmriprep/ -m "not real_data" => 36 passed, 1 deselected
- git status stayed clean: the 478 MB dataset and 2.4 GB golden master are
  both ignored.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants