Skip to content

main branch for alignment training for S2BMS and Satbird - #157

Merged
vdplasthijs merged 129 commits into
developfrom
train/alignment
Oct 20, 2026
Merged

vdplasthijs merged 129 commits into
developfrom
train/alignment

Conversation

@gabrieletijunaityte

@gabrieletijunaityte gabrieletijunaityte commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Added:

  • viz_utils/py for figures
  • Add test concept implementation (train_alignment_model.py and
  • Remove stds from the loss. It was moved to dataset.
  • Add dimensions parameter to IdentityEncoder
  • Add plot method to dataset.
  • Add crop-padding method in the dataset for the vlm baselines
  • Add satbird caption builder

Small changes:

  • In soft_contrastive_vloss.py simplify loss to never use all aux values for the soft target value calculation.
  • In contrastive_validation.py safe guard for invalid k values.
  • train_alignment_model.py:
    • device management (safer pass to cpu and gpu).
    • Implement aux std values ( also in the dataset)
    • extract _on_x_start() (also in the base_model.py) to handle model setup on different model intialisations (e.g., if model is never trained but immediately evaluated (like vlm baselines)).
  • For the task wrapper make sure wandb gets 'crashed' flag if code breaks.
  • in evaluating.py removed use of average encoder class.
  • In train.py validation and testing flags are now independent of wandb logger.

Before submitting

  • Did you make sure title is self-explanatory and the description concisely explains the PR?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you test your PR locally with pytest command?

gabrieletijunaityte and others added 30 commits July 16, 2026 08:46
# Conflicts:
#	src/models/text_alignment_model.py
- Skip if exists
- Start > 0
- Satbird configuration
# Conflicts:
#	src/models/text_alignment_model.py
# Conflicts:
#	configs/experiment/s2bms_prediction/avr_aef_1_128_256_lin.yaml
#	configs/experiment/s2bms_prediction/avr_aef_1_128_256_mlp.yaml
#	configs/experiment/s2bms_prediction/avr_tessera_v11_1_128_256_deeper_mlp.yaml
#	configs/experiment/s2bms_prediction/avr_tessera_v11_1_128_256_lin.yaml
#	configs/experiment/s2bms_prediction/avr_tessera_v11_1_128_256_mlp.yaml
#	configs/experiment/s2bms_prediction/tessera_128_256_rn18_34_50_deeper_mlp.yaml
#	notebooks/19-GT-wandb_prediction_table.ipynb
# Conflicts:
#	notebooks/09-GT-aef-tessera-datasplit-filtering.ipynb
#	src/data/satbird_dataset.py
@gabrieletijunaityte gabrieletijunaityte changed the title Train/alignment main branch for alignment training for S2BMS and Satbird Oct 18, 2026
Comment thread src/utils/utils.py Resolved
Comment thread src/train.py Resolved
Comment thread src/eval.py Resolved
Comment thread src/models/base_model.py Resolved
Comment thread src/data/satbird_dataset.py Resolved
Comment thread src/data/satbird_dataset.py Resolved
Comment thread src/data/butterfly_dataset.py Resolved
Comment thread src/data/base_dataset.py Resolved
Comment thread src/data/base_datamodule.py Resolved
gabrieletijunaityte and others added 5 commits October 19, 2026 14:15
Only issue is I had to deselect these datasets:

def test_datasets_generic_properties(request, tmp_path, sample_csv):
    """This test checks that all datasets implement the basic properties and methods."""
    # list_datasets = [ButterflyDataset, SatBirdDataset, HeatGuatemalaDataset, YieldAfricaDataset]
    list_datasets = [ButterflyDataset]
@vdplasthijs
vdplasthijs marked this pull request as ready for review October 19, 2026 15:42
Comment thread src/models/text_alignment_model.py Resolved
@vdplasthijs
vdplasthijs merged commit 983470d into develop Oct 20, 2026
4 checks passed
@vdplasthijs vdplasthijs mentioned this pull request Oct 20, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants