You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ESMFold2: fold(lm_dropout=0) does not turn the LM dropout off #418
The fold() docstring says lm_dropout defaults to 0.3 and that 0/None disables it, but 0 leaves the checkpoint's own rate in place: _lm_dropout_context returns early for any falsy value. The released biohub/ESMFold2 and biohub/ESMFold2-Fast configs ship lm_encoder.lm_dropout = 0.25 with per_loop_lm_dropout = True, so a fold asked for no dropout still drops 25 % of the LM pair channel in every loop:
fromesm.models.esmfold2.processorimport_lm_dropout_contextwith_lm_dropout_context(model, 0):
model.config.lm_encoder.lm_dropout# 0.25, not 0
The helper's own docstring says "None/0 is a no-op", so the two docstrings disagree about what 0 should mean. For anyone using lm_dropout=0 as a control, the fold is not the dropout-free one it reads as.
What I would expect: 0 disables the dropout for the call and the config is restored afterwards, None keeps the checkpoint's setting, and a positive value overrides it, on both the release and the experimental architecture. If you would rather keep 0 as a no-op, the fold() docstring needs the fix instead.
I have the first version with tests on both architectures on a branch (diff) and can open it as a PR if that is the behaviour you want.
Activity