Skip to content

fix(steamvr): let a streamed headset's poses through SteamVR unpredicted - #11

Merged
aaf2tbz merged 3 commits into
metalsharp:mainfrom
multision:fix/steamvr-head-prediction
Nov 20, 2026
Merged

aaf2tbz merged 3 commits into
metalsharp:mainfrom
multision:fix/steamvr-head-prediction

Conversation

@multision

@multision multision commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

SteamVR games streamed to a Quest over ALVR (Blade & Sorcery, Skyrim VR) swim and judder during head turns, most when the game runs below the headset's rate. OpenXR games on the same headset don't.

ALVR's client sends every head pose already predicted for the moment the frame drawn from it is shown. The SteamVR driver passed that pose to SteamVR together with the head's velocities, so SteamVR predicted it a second time, to its own display time, and rendered with a pose the headset never sent. Present then names the sample nearest SteamVR's mHmdPose (#8), which is off by the turn rate times SteamVR's prediction, and the headset reprojects from the wrong reference.

For the headsets that stream through ALVR (Quest family, Steam Frame), the head pose now goes to SteamVR without velocities. SteamVR's prediction then leaves it as it is, and Present finds the very sample. Controllers keep their velocities, which games read for throws. Headsets metalvrd drives itself (Rift, Lighthouse, PSVR2) are unchanged: their samples are for now, and SteamVR's prediction is wanted.

This carries #8's commit, rebased onto main. #8 conflicts with main since #9 added setViews at the same place, and this change needs #8's sample matching. The conflict is resolved here by keeping both. Merging this PR covers #8, which can then be closed; or merge #8 first and I'll rebase this onto it.

Testing

  • Built for macOS and for Windows (build/native, build/win64): not run, no CMake or Ninja on this Mac. driver_metalvr.dll built with x86_64-w64-mingw32-g++ 16.2 from the CMake target's sources, include paths and flags: 0 warnings.
  • scripts/test-native.sh and scripts/test-failures.sh: not run; they do not reach the SteamVR driver.
  • scripts/test-wine.sh: not run.
  • scripts/test-openvr.sh: not run (SteamVR driver, not the OpenComposite path).
  • On a headset: Meta Quest 3 over Wi-Fi with the ALVR client, ALVR server v20.14.1 in metalvrd, SteamVR 2.18.4 on MetalSharp Wine x86_64, Apple silicon Mac, macOS 27.2, Blade & Sorcery 1.3.2. The judder during head turns is clearly reduced compared with the same build without this commit (fix(steamvr): name the tracking sample each frame was rendered with #8, fix(steamvr): render with the headset's own fields of view and IPD #9, fix(steamvr): take SteamVR's own grip offset off Touch controllers #10 only). What remains is microstutter while the game runs at 60–70 fps on the 72 Hz headset; metalvrd's late-frame count rises exactly then, so that part follows the game's frame rate.

Checklist

  • The change is my own work, or open-source code whose license allows it here; nothing is taken from a closed-source product
  • Changes to ALVR or OpenComposite are patches under patches/, not edits in external/
  • New downloads are pinned to a commit
  • Documentation is updated where behaviour, options or the protocol change (no documented behaviour changes)
  • No Wine prefixes, logs with personal paths, or other personal data

The SteamVR driver's FRAME_END never set sample_id, and metalvrd's ALVR driver
drops a frame that names no tracking sample, so a Quest streaming SteamVR over
ALVR connected and tracked but received no video: every session ended with
"headset disconnected (0 frames sent, N dropped)".

SteamVR does not say which pose it rendered a frame with, only the pose itself
(SubmitLayer's mHmdPose). Keep the head orientations of the last 64 tracking
samples and send the one nearest the first projection layer's pose, as ALVR's
own SteamVR driver does (PoseHistory::GetBestPoseMatch). Newest samples win
ties, so a still head names the newest sample rather than an older one that
float noise happens to put closer. The headset then reprojects each frame from
the pose it was drawn with.
@multision
multision requested a review from aaf2tbz as a code owner November 19, 2026 17:05
@multision
multision force-pushed the fix/steamvr-head-prediction branch from 47384bf to bb5ad75 Compare November 19, 2026 18:45
ALVR's client sends every head pose already predicted for the moment the frame
drawn from it is shown, and Present names the tracking sample whose orientation
is nearest SteamVR's render pose (mHmdPose) so the headset reprojects from it.
The driver handed SteamVR the head's velocities as well, so SteamVR predicted
the pose a second time, to its own display time, and rendered with a pose the
headset never sent. The sample Present names is then off by the turn rate times
SteamVR's prediction, the headset reprojects from the wrong reference, and the
picture swims while the head turns: most when a game misses frames.

For the headsets that stream through ALVR (Quest family, Steam Frame) the head
pose goes to SteamVR without velocities, so its prediction leaves the pose as
it is and Present finds the very sample. Controllers keep their velocities,
which games read for throws. Headsets metalvrd drives itself (Rift, Lighthouse,
PSVR2) are unchanged: their samples are for now, and SteamVR predicts them.
@multision
multision force-pushed the fix/steamvr-head-prediction branch from bb5ad75 to b924191 Compare November 19, 2026 18:45
@aaf2tbz
aaf2tbz merged commit efd1583 into metalsharp:main Nov 20, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants