Repository navigation
Conversation
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Concatenated operators currently inherit the generic
_bilinear_derivative, which reconstructs detached arguments and evaluatesautograd.gradwithout building a graph. WhenCatLinearOperatoris used inside another operator's matmul (for exampleBlockDiagLinearOperator), first derivatives can be numerically correct while covariance second derivatives are detached or incomplete.This appears in independent multitask Gaussian covariances: the squared dense covariance loss for two simple blocks yields correct first gradients with
create_graph=True, but those gradients haverequires_grad=False, so the next backward raises. In a two-output exact-GP posterior example the resulting mixed Hessian-vector error is 0.213, although values and all first derivatives agree with dense Gaussian conditioning to about 8e-15. Central differences of the unmodified first gradients converge to the dense result. This Cat-specific analytic derivative reduces that full model discrepancy to 2.3e-12.Delegate the bilinear VJP to each child with the corresponding row, column, or batch vector slice. Broadcast batches and move vectors to each child's device, retaining the incoming vector graphs without materializing the concatenated matrix. The forward implementation is unchanged. This is a separate missing analytic delegation from the triangular operator repair in #152; applying that repair alone does not fix this example.
Validation against main
1c5e24b:ufmt check,flake8, andgit diff --checkpass for the changed files.AI assistance: implemented and tested with OpenAI Codex. The numerical reproductions, regression runs, and formatting checks above were executed locally.