From 8d5f8c778843fcc973fa5467d7be9e4b8a2e818d Mon Sep 17 00:00:00 2001 From: Richard Cornelius Suwandi <59959022+richardcsuwandi@users.noreply.github.com> Date: Sun, 2 Aug 2026 00:42:27 +0800 Subject: [PATCH] Fix: `MultivariateQExponential.log_prob` ignores `power` when fast log-prob is off MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Problem When `gpytorch.settings.fast_computations.log_prob` is off, `MultivariateQExponential.log_prob` falls back to `torch.distributions.MultivariateNormal.log_prob`. That path is Gaussian-only, so it ignores `power` and always returns the `q = 2` log-density. This matters in practice because **BoTorch turns that setting off on import**, and QePyTorch reuses `gpytorch.settings`. So any BoTorch-style Exact QEP fit via ```python mll = ExactMarginalLogLikelihood(likelihood, model) loss = -mll(output, target) ``` quietly does ordinary Gaussian MLE, even when `power != 2`. I hit this with a BoTorch-style `SingleTaskQEP` wrapper: same data and init, `q ∈ {2, 1.5, 1, 0.5}`, but `output.log_prob(...)` and the MLL stayed identical for every `q` (matching the Gaussian value). `output.power` itself was set correctly; only the numerical log-prob / MLL was wrong. ## Why it happens `MultivariateQExponential.log_prob` currently does: ```python if settings.fast_computations.log_prob.off(): return super().log_prob(value) # Gaussian-only; ignores power ``` That early return is copied from GPyTorch's `MultivariateNormal`, where a Gaussian fallback is fine. Here `super()` is still `torch.distributions.MultivariateNormal`, which has no `power`, so the fallback is wrong for `q != 2`. The Q-Exponential formula in the rest of the method is correct. A plain `MultivariateQExponential(...).log_prob(y)` with fast log-prob left on is `q`-dependent as expected. The bug only appears once the fast-log-prob-off path is taken (e.g. after importing BoTorch). On a small ExactQEP / `SingleTaskQEP` example, the broken MLL was about `-12.25` for all four `q` values. Recomputing the QED formula from the same `loc`, covariance, and `power` gave the correct sequence: about `-12.25`, `-13.68`, `-15.65`, `-18.17`. ## Implications - Hyperparameter learning through `ExactMarginalLogLikelihood` under BoTorch (or any code with fast log-prob off) fits a GP, not a QEP, whenever `q != 2`. - Code that only reads `.power` (e.g. some acquisition logic) is unaffected. - The failure is silent: no error, just the wrong objective. ## Fix Only use the Gaussian `super().log_prob` fallback when `power == 2`. For `power != 2`, always evaluate the Q-Exponential log-density via `inv_quad_logdet`, even if `fast_computations.log_prob` is off. After the fix, the same BoTorch / `SingleTaskQEP` setup recovers the `q`-dependent MLL values above. ## Checklist (from CONTRIBUTING.md) - [x] Code change - [x] No new public API / docs objects required - [x] Ran `python -m unittest` locally before opening the PR --- qpytorch/distributions/multivariate_qexponential.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/qpytorch/distributions/multivariate_qexponential.py b/qpytorch/distributions/multivariate_qexponential.py index 5cd1ab8..6a55153 100644 --- a/qpytorch/distributions/multivariate_qexponential.py +++ b/qpytorch/distributions/multivariate_qexponential.py @@ -241,7 +241,12 @@ def log_prob(self, value: Tensor) -> Tensor: See :py:meth:`torch.distributions.Distribution.log_prob `. """ - if settings.fast_computations.log_prob.off(): + # The torch.distributions.MultivariateNormal fallback is Gaussian-only. + # Only use it when power == 2. When fast_computations.log_prob is off + # (e.g. BoTorch disables it on import; QePyTorch shares gpytorch.settings), + # falling back for power != 2 silently ignores `power` and returns the + # Gaussian log-density. + if settings.fast_computations.log_prob.off() and bool(torch.as_tensor(self.power == 2).all()): return super().log_prob(value) if self._validate_args: