Skip to content

feat: use Globus data transfers for results/output (SBP-535) - #139

Merged
marius-mather merged 26 commits into
devfrom
feat/output-transfers
Aug 21, 2026
Merged

feat: use Globus data transfers for results/output (SBP-535)#139
marius-mather merged 26 commits into
devfrom
feat/output-transfers

Conversation

@marius-mather

@marius-mather marius-mather commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Pull Request

Summary

SBP-535: use Globus data transfers to sync output. Set up data transfers once each run is completed on Seqera.

Changes

  • Update data transfer code to deal with outputs as well as inputs
  • Create data transfers when syncing workflows - data transfers get created after Seqera reports the job is complete
  • Add recursive column to DataTransfer so we can create folder transfers
  • Update job/result endpoints to account for result syncing - outputs need to be synced before results can be fetched.

How to Test

Run uv run pytest

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have added tests that prove my fix is effective or that my feature works
  • I have added or updated documentation where necessary
  • I have run linting and unit tests locally
  • The code follows the project's style guidelines

@vtnphan vtnphan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice and clean, just some small things could be improved as below

Comment thread app/services/globus_transfer.py
Comment thread alembic/versions/20260820_111241_recursive_data_transfer_4ac790a42b41.py Outdated
Comment thread app/db/models/core.py
Comment thread app/services/globus_transfer.py Outdated
@marius-mather
marius-mather requested a review from vtnphan August 21, 2026 03:35

@vtnphan vtnphan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty now and ready to shift!

@marius-mather
marius-mather merged commit a5fa2e7 into dev Aug 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants