Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ Changelog
Unreleased
----------
- Loss aggregation uses the configured floating-point dtype in eager and compiled execution, including empty and zero-weight aggregates.
- Named broad structure-compatibility cases explicitly, moved extra datasets to the slow tier, and removed eager all-model loading and swallowed reader failures.
- Consolidated CIF/MTZ loading contracts, checked configured tensor placement, and replaced ModelFT smoke checks with exercised forward-cache behavior.
- Replaced local-arithmetic target tests with configured-device production-kernel checks on deposited coordinates and explicit least-squares expectations.
- Consolidated weighting tests by API ownership and strengthened Gaussian-likelihood, gradient-norm, and cached-loss assertions.
- Organized test fixtures into focused modules and reused module-scoped loaded objects for read-only functional checks while retaining fresh objects for mutation and loading tests.
- Rigid-body refinement stores its Euler angles pre-multiplied by the chain's radius of gyration, so a unit step in an angle and a unit step in a translation displace atoms comparably. In radians against Angstroms the rotation block of the Hessian carried 190-530x the curvature of the translation block on 1DAW and 3E98 -- the geometric ``Rg**2``, 411 and 442/516 -- putting ``cond(H)`` at 1e3-5e3, which is why six parameters needed ~250 L-BFGS iterations to place. Dividing the scale out in ``forward()`` brings the ratio to 0.4-1.3 and ``cond(H)`` to 3-18. Over ten structures the step then converges rather than exhausting its iteration budget, on about half the gradient evaluations, with R-free no worse anywhere. ``RigidXYZTensor.rotation_radians`` returns the physical angle, and setting ``angle_scale`` to ones restores the unscaled parametrization. Not a fix for the one or two negative Hessian eigenvalues at the finer cutoffs -- scaling a saddle leaves it a saddle -- and those counts are unchanged
- The rigid-body step no longer co-refines the scaler in the same L-BFGS as the rigid parameters. The body target centres on ``alpha*|F_calc|`` and ``alpha`` absorbs a rescaling of ``F_calc`` exactly, so the scale had a flat direction there; ``SCALE_TARGETS`` already excludes every alpha-centred row from the scale fit for this reason, and 0.6.2 fixed the same thing in the main driver. ``refine_scaler`` (objective ``ls``) owns the scale, between cutoffs
- Fixed ``refine_rigid_body`` leaving the caller's reflection data truncated. ``cut_res`` masks in place and returns ``self``, so each cutoff stamped its resolution mask on the caller's own object and the restore had nothing to restore to -- it only looked correct because the default schedule ends at the native limit. With ``--rigid-body-cutoffs 6,4`` on a 2.05 A dataset, 20138 of 23352 reflections stayed masked out for the rest of the run, R-factors included
Expand Down
26 changes: 17 additions & 9 deletions docs/user_guide/testing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,13 @@ PDB ID d_min (Å) Space group
``tests/files/`` also holds partial sets — ``1AK5_with_H.pdb`` + ``1AK5.mtz``
(no CIF), ``7L84.pdb`` + ``7L84-sf.cif`` (no MTZ), ``test_ihm_ensemble.cif`` —
so a test that globs one directory and assumes a matching file in another will
fail on those. Use ``sample_structure_pair`` / ``all_test_structures``.
fail on those. Use ``sample_structure_pair`` for the quick reference crystal,
or ``compatibility_structure_pair`` for named extended cases. The latter carries
the ``slow`` marker and selects paths without loading objects.

``tests/helpers/structure_cases.py`` assigns bundled CIF, MTZ and SF-CIF files to
the quick or extended compatibility panel. Additional files require an explicit
assignment; directory growth does not silently expand numerical test work.

Running Tests
-------------
Expand Down Expand Up @@ -99,11 +105,11 @@ The Amber stack, if you want it:
Fixtures
--------

Almost everything lives in the root ``tests/conftest.py`` and is therefore
available from every category — the ``integration/`` and ``functional/``
conftests are docstrings only. Mock data is the exception:
``tests/unit/conftest.py``. Read those two files for the authoritative list; the
ones you will reach for most:
Reusable setup lives in ``tests/fixtures/``. The root ``tests/conftest.py``
registers shared plugins and owns test-selection hooks. The unit conftest exposes
synthetic numerical factories; the functional conftest exposes the module-scoped,
read-only Fourier-model fixture. See ``tests/fixtures/README.md`` for ownership
and mutation rules. Common fixtures include:

- Paths (session-scoped): ``tests_root``, ``project_root``, ``test_files_dir``,
the per-format ``cif_dir``, ``mtz_dir``, ``pdb_dir``, ``cif_sf_dir``, and
Expand All @@ -118,9 +124,11 @@ ones you will reach for most:
``mock_aniso_u``, ``mock_scattering_factors``, ``mock_weights``.
- Real files: ``sample_cif_file``, ``sample_pdb_file``, ``sample_mtz_file``,
``sample_structure_factor_cif``, ``sample_structure_pair`` (matched model +
data), ``all_structure_pairs``, ``all_test_structures``.
- Loaded objects: ``loaded_model``, ``loaded_reflection_data``,
``model_and_data``, ``initialized_scaler``.
data), ``compatibility_structure_pair`` (one named slow crystal).
- Loaded objects: ``loaded_model``, ``loaded_model_ft``, ``loaded_reflection_data``,
``model_and_data``, ``initialized_scaler``. ``compatibility_model`` and
``compatibility_model_and_data`` load only the current slow case and remain
function-scoped to isolate mutations.

The mock-data fixtures yield a *factory* taking ``n_atoms`` / ``n_reflections``
and ``seed``; ``mock_cell`` and ``mock_cell_triclinic`` yield the tensor
Expand Down
33 changes: 31 additions & 2 deletions tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ This directory contains the complete test suite for torchref.

```
tests/
├── conftest.py # Root fixtures (paths, devices, skip decorators)
├── conftest.py # Fixture registration and test-selection hooks
├── fixtures/ # Shared setup, grouped by responsibility (see fixtures/README.md)
├── pytest.ini # Pytest configuration
├── __init__.py
├── files/ # Test data files (CIF, PDB, MTZ)
Expand All @@ -15,7 +16,7 @@ tests/
│ ├── mtz/ # Reflection MTZ files
│ └── cif_sf/ # Structure factor CIF files
├── unit/ # Unit tests (fast, no I/O)
│ ├── conftest.py # Unit test fixtures (mock data)
│ ├── conftest.py # Imports scoped numerical fixtures
│ ├── math_functions/ # Math module tests
│ ├── model/ # Model module tests
│ ├── refinement/ # Refinement module tests
Expand All @@ -38,6 +39,34 @@ tests/

## Running Tests

### Coverage ownership

| Contract | Owner |
|---|---|
| Loss weights, aggregation, cached loss reads | `unit/refinement/test_loss_state.py` |
| Refinement's default group weights | `unit/refinement/test_loss_weighting.py` |
| Gaussian amplitude-metric values and reductions | `unit/base/test_loss.py` |
| Restraint kernel values on deposited coordinates | `unit/base/test_target_values.py` |
| Gradient RMS norm | `unit/utils/test_gradnorm.py` |
| CIF atomic fields and crystal metadata | `integration/test_io_cif.py` |
| MTZ fields, resolution bins and model/data crystal agreement | `integration/test_io_reflections.py` |
| ModelFT forward cache and grid integration | `functional/test_model_ft_functional.py` |
| Extra deposited files and input inventory | `integration/test_structure_compatibility.py`, `helpers/structure_cases.py` |
| Numerical derivatives and backend parity | `unit/test_gradient_correctness.py`, `unit/structure_factor/` |

A production call must participate in the assertion: computing a formula only in
the test does not check its implementation. Kernel values, target registration,
device transitions, and default configuration are separate contracts even when
they exercise the same class. Keep mutation tests on fresh objects.

The quick reader contracts use 1DAW. Extended reader compatibility runs with
`pytest tests/integration/test_structure_compatibility.py --run-slow`; each file
is a separate case and must succeed. The manifest covers the bundled CIF, MTZ
and SF-CIF inputs, including the IHM fixture and reflection-only depositions.
Adding a data file requires an explicit coverage assignment in the manifest.
Extended scaler and restraint cases use 2DQ6 (trigonal) and 3A5V (body-centred
tetragonal), with fresh objects per case and `--run-slow` required.

### Quick Local Run (on login node, for small tests only)

```bash
Expand Down
21 changes: 11 additions & 10 deletions tests/RUNNING_TESTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ pytest tests/unit/refinement/ -v
pytest tests/unit/refinement/test_loss_weighting.py -v

# Target/loss functions
pytest tests/unit/refinement/test_targets.py -v
pytest tests/unit/base/test_target_values.py tests/unit/base/test_loss.py -v
```

### Scaling
Expand Down Expand Up @@ -176,17 +176,17 @@ pytest tests/unit/model/test_parameter_wrappers.py::TestMixedTensorOperations -v

#### Refinement Classes
```bash
# Fixed weighting
pytest tests/unit/refinement/test_loss_weighting.py::TestFixedWeighting -v
# Weight handling
pytest tests/unit/refinement/test_loss_state.py::TestWeightManagement -v

# Resolution-dependent weighting
pytest tests/unit/refinement/test_loss_weighting.py::TestResolutionDependentWeighting -v
# Default group weights
pytest tests/unit/refinement/test_loss_weighting.py::TestDefaultGroupWeights -v

# Gaussian NLL loss
pytest tests/unit/refinement/test_targets.py::TestGaussianNLL -v
pytest tests/unit/base/test_loss.py -v

# Least squares target
pytest tests/unit/refinement/test_targets.py::TestLeastSquaresTarget -v
pytest tests/unit/base/test_target_values.py -k least_squares -v
```

#### Symmetry Classes
Expand Down Expand Up @@ -361,13 +361,14 @@ pytest tests/unit --lf -v
| `math_functions/test_math_numpy.py` | `TestCoordinateTransformations`, `TestScatteringVectors`, `TestRFactorCalculations`, `TestRotation` |
| `model/test_model.py` | `TestModelInitialization`, `TestModelDeviceHandling` |
| `model/test_parameter_wrappers.py` | `TestMixedTensorInitialization`, `TestMixedTensorOperations`, `TestMixedTensorDeviceHandling`, `TestOccupancyTensor`, `TestPositiveMixedTensor` |
| `refinement/test_loss_weighting.py` | `TestFixedWeighting`, `TestResolutionDependentWeighting`, `TestLossWeightingModule` |
| `refinement/test_targets.py` | `TestTargetBase`, `TestGaussianNLL`, `TestLeastSquaresTarget`, `TestRiceNLL`, `TestTargetDeviceHandling`, `TestNumericStability` |
| `refinement/test_loss_weighting.py` | `TestDefaultGroupWeights` |
| `base/test_target_values.py` | Deposited-coordinate restraint values and least-squares weighting |
| `base/test_loss.py` | Gaussian NLL values and reductions |
| `scaling/test_scaler.py` | `TestScalerInitialization`, `TestScalerDeviceHandling`, `TestScalingCalculations`, `TestBFactorScaling`, `TestAnisotropicScaling` |
| `symmetrie/test_symmetrie.py` | `TestSymmetryInitialization`, `TestSymmetryMatrices`, `TestSymmetryApplication`, `TestSymmetryDeviceHandling`, `TestSpaceGroupMapping` |
| `io/test_data.py` | `TestReflectionDataInitialization`, `TestReflectionDataDeviceMovement`, `TestReflectionDataAttributes`, `TestReflectionDataProperties`, `TestMockReflectionData` |
| `restraints/test_restraints.py` | `TestRestraintsInitialization`, `TestBondRestraintCalculations`, `TestAngleRestraintCalculations`, `TestTorsionRestraintCalculations`, `TestRestraintDeviceHandling`, `TestRestraintNumericStability` |
| `utils/test_gradnorm.py` | `TestGradNorm` |
| `utils/test_gradnorm.py` | RMS norms for single/multiple parameters and zero gradients |
| `utils/test_utils.py` | `TestModuleReference`, `TestCIFReader` |

### Integration Tests (`tests/integration/`)
Expand Down
Loading