[flyingdoubleG]

Summary

LinearOperator.svd() is documented as computing A = U S V^T for a general A (M x N). The default _svd, however, goes through _symeig, so it is only valid for symmetric matrices. The code comment says so: "Will need to overwrite this function for non-psd LinearOperators". Two common non-symmetric operators do not override it, so svd() and torch.linalg.svd(op) silently return a wrong decomposition for them. A third operator has a separate sign bug.

operator	svd() reconstruction error on main
DenseLinearOperator(torch.randn(5, 5))	2.7 (some singular values come out as 0)
DenseLinearOperator(torch.randn(5, 3))	RuntimeError (eigh on a non-square matrix)
TriangularLinearOperator(...)	2.4
KroneckerProductLinearOperator of non-symmetric dense factors	0.9
KroneckerProductTriangularLinearOperator	0.9
DiagLinearOperator(torch.tensor([1., -2., 3.]))	4.0

For the diagonal, _svd builds V = evecs * sign(evals).unsqueeze(-1), where evecs is a ConstantDiagLinearOperator (the identity). That elementwise product keeps only the diagonal of the (N, 1) column, so the signs are lost and U S V^T = diag(|evals|).

Changes

• DenseLinearOperator._svd and TriangularLinearOperator._svd use torch.linalg.svd. For dense operators this is full_matrices=False, so rectangular operators work. The Kronecker _svd, which combines the factors' SVDs, is then correct for these factors as well.
• DiagLinearOperator._svd returns V = DiagLinearOperator(signs).
• The PSD-structured operators that rely on the default eigendecomposition path are unchanged.

Tests

• TestDenseLinearOperator.test_svd_not_symmetric: square, rectangular (both orientations) and batched non-symmetric matrices, through svd() and torch.linalg.svd.
• TestDiagLinearOperator.test_svd_negative_diagonal: inherited by the batch, multi-batch and Kronecker-diagonal test cases.
• TestTriangularLinearOperator.test_svd was overridden with pass. I removed the override, so the generic test_svd (forward and backward) now runs for lower and upper triangular operators.

On main, 7 of these tests fail. With this PR they pass. The full suite gives 5050 passed and 1 failed. The failure is test_psd_safe_cholesky_psd, which also fails on main without these changes. I ran ufmt and flake8 on the changed files.