Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,11 @@ git submodule update --init --recursive
See [feature-extraction/atlasnet/README.md](feature-extraction/atlasnet/README.md) for instructions.

2. **Feature decoding** — Decode DNN features from fMRI data using linear decoders.
See [feature-decoding/README.md](feature-decoding/README.md) for instructions.

3. **Reconstruction** — Reconstruct 3D shapes from true or decoded features using AtlasNet.
See [reconstruction/atlasnet/README.md](reconstruction/atlasnet/README.md) for instructions.
This includes cross-validation training and prediction for estimating the
decoded-feature scaling factor. See
[feature-decoding/README.md](feature-decoding/README.md) for instructions.

3. **Feature scaling and reconstruction** — Scale decoded features using the
cross-validation estimate, then reconstruct 3D shapes from true or scaled
decoded features using AtlasNet. See
[reconstruction/atlasnet/README.md](reconstruction/atlasnet/README.md) for instructions.
26 changes: 26 additions & 0 deletions feature-decoding/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,32 @@ python scripts/predict_feature_fastl2lir.py train-3d-natural-objects_rep3_fmap_t

Output: decoded features saved to the path specified by `decoded_feature.path` in the config.

### Cross-validation for feature scaling

Cross-validation predictions on the training dataset are used to estimate the
standard deviation of decoded features for feature scaling.

#### 1. Train cross-validation decoders

```bash
uv run python scripts/cv_train_decoder_fastl2lir.py config/cv_train-3d-natural-objects-image_rep3_fmap_fmriprep_5000voxel_atlasnet.yaml
```

Output: cross-validation decoders are saved to the path specified by
`decoder.path` in the CV config.

#### 2. Decode cross-validation features

```bash
uv run python scripts/cv_predict_feature_fastl2lir.py config/cv_train-3d-natural-objects-image_rep3_fmap_fmriprep_5000voxel_atlasnet.yaml
```

Output: cross-validated training features are saved to the path specified by
`decoded_feature.path` in the CV config.

The cross-validated features are subsequently used for feature scaling. See
[the AtlasNet reconstruction instructions](../reconstruction/atlasnet/README.md#feature-scaling).

## Config files

Config files are located in `config/`. Each file specifies a combination of training and test datasets.
Expand Down
30 changes: 28 additions & 2 deletions reconstruction/atlasnet/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,29 @@ The AtlasNet model weights must be placed at:
data/models/atlasnet/network_crtd.pth
```

## Feature scaling

Before reconstructing shapes from decoded features, scale the decoded test
features using the standard deviation estimated from cross-validation predictions.

First, complete:

1. Regular feature decoder training and feature prediction.
2. Cross-validation decoder training and prediction described in
[feature-decoding/README.md](../../feature-decoding/README.md#cross-validation-for-feature-scaling).

Then run:

```bash
uv run feature_scaling.py
```

For every experiment listed in `feature_scaling.py`, the scaled features are saved to:

```text
data/decoded-features/{experiment}_scaled_traincvstd/atlasnet/
```

## Usage

```bash
Expand Down Expand Up @@ -51,9 +74,12 @@ true/

### From decoded features

Input: `data/decoded-features/{experiment}/atlasnet/`
Input: `data/decoded-features/{experiment}_scaled_traincvstd/atlasnet/`

Output: `data/reconstruction/atlasnet_encoder_bn5/decoded/{experiment}_scaled_traincvstd/{subject}/{roi}/`

Output: `data/reconstruction/atlasnet_encoder_bn5/decoded/{experiment}/{subject}/{roi}/`
`recon_from_features.py` reconstructs shapes only from the scaled decoded-feature
datasets listed in `decoded_datasets`.

```
decoded/
Expand Down
10 changes: 5 additions & 5 deletions reconstruction/atlasnet/recon_from_features.py
Original file line number Diff line number Diff line change
Expand Up @@ -152,11 +152,11 @@ def reconstruct_all(recon, features, output_dir, source_layer, subject=None, roi

# Decoded features
decoded_datasets = [
'train-3d-natural-objects_rep3_test-3d-natural-objects_rep8_fmap_fmriprep_5000voxel_fastl2lir_alpha5000',
'train-3d-natural-objects_rep3_test-3d-artificial-objects-image_rep8_fmap_fmriprep_5000voxel_fastl2lir_alpha5000',
'train-3d-natural-objects_rep3_test-3d-artificial-objects-rds_rep8_fmap_fmriprep_5000voxel_fastl2lir_alpha5000',
'train-3d-natural-objects_rep3_test-3d-contour-matched-rds-horizontal-shape-variants_rep8_fmap_fmriprep_5000voxel_fastl2lir_alpha5000',
'train-3d-natural-objects_rep3_test-3d-contour-matched-rds-thin-tilt-variants_rep8_fmap_fmriprep_5000voxel_fastl2lir_alpha5000',
'train-3d-natural-objects_rep3_test-3d-natural-objects_rep8_fmap_fmriprep_5000voxel_fastl2lir_alpha5000_scaled_traincvstd',
'train-3d-natural-objects_rep3_test-3d-artificial-objects-image_rep8_fmap_fmriprep_5000voxel_fastl2lir_alpha5000_scaled_traincvstd',
'train-3d-natural-objects_rep3_test-3d-artificial-objects-rds_rep8_fmap_fmriprep_5000voxel_fastl2lir_alpha5000_scaled_traincvstd',
'train-3d-natural-objects_rep3_test-3d-contour-matched-rds-horizontal-shape-variants_rep8_fmap_fmriprep_5000voxel_fastl2lir_alpha5000_scaled_traincvstd',
'train-3d-natural-objects_rep3_test-3d-contour-matched-rds-thin-tilt-variants_rep8_fmap_fmriprep_5000voxel_fastl2lir_alpha5000_scaled_traincvstd',
]

subjects = ['S1', 'S2', 'S3', 'S4', 'S5']
Expand Down