Skip to content

Improve simulator correctness and training reliability - #261

Merged
tesslerc merged 2 commits into
mainfrom
release/2026-10-15
Oct 16, 2026
Merged

tesslerc merged 2 commits into
mainfrom
release/2026-10-15

Conversation

@tesslerc

@tesslerc tesslerc commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Release summary

This update improves simulator correctness, evaluation reliability, and interactive inference. It also fixes observation-noise propagation and configuration errors, updates the BeyondMimic training recipe, and clarifies pretrained-model licensing.

Simulators and robot assets

  • MuJoCo: correct root angular-velocity conversion between body and world frames, and report body linear velocity at the link origin. Reset and projectile velocity writes use the same conventions.
  • MuJoCo actuators: normalize supported actuator gear ratios so control values and effort limits represent joint torque. Reject unsupported gear configurations instead of silently applying incorrect forces. Allow an explicit integrator override while preserving the MJCF choice by default.
  • Newton: repair explicit proportional and torque control.
  • IsaacLab scenes: use sequential cloning to preserve motion-to-object pairing for single-object scenes instead of randomly assigning objects. Multi-object scene assignment remains a limitation.
  • Massless marker links: give zero-mass bodies explicit 1 g masses and consistent inertias before simulator import, preventing unintended backend defaults. Physical-body masses and source MJCF files are preserved; IsaacLab conversion caches are refreshed.
  • SOMA retargeting: correct pelvis keypoint construction and document the mapping.

Training, observations, and evaluation

  • Preserve configured anchor-orientation and local-gyro noise when building observation views, rather than replacing those channels with noise-free derived values.
  • Restore model-owned rollout context after evaluation so training resumes with its saved context.
  • Improve evaluation parking/restoration and clear stale contact state before restoring simulation state.
  • Forward termination thresholds correctly through component bindings and validate conflicting component parameters.
  • Detect module producer/consumer ordering errors earlier and correct the MaskedMimic transformer module order.
  • Improve cluster retry handling and artifact copy-back reliability.

BeyondMimic recipe changes

  • Newly constructed BM controllers use q_target = q_default + 0.25 * effort_limit / stiffness * action, without clipping raw actions or changing PD gains.
  • The BM L2C2 example uses -0.1 * sum((action - previous_action)**2) on raw policy actions. This is separate from its L2C2 model loss; other recipes retain their reward bindings.
  • Existing saved configurations retain their original action scales and rewards. Start a new experiment to adopt the changed recipe.

Interactive inference and recording

  • MuJoCo receives keyboard input directly in the simulation viewer; the separate controls window is removed.
  • Press . to switch between ProtoMotions key bindings and MuJoCo's native viewer key bindings. Mouse orbit and zoom remain available in either mode.
  • MuJoCo recordings preserve the live viewer camera and visualization settings and keep frame dimensions fixed throughout a clip.

Models and documentation

  • Add full pretrained-model license texts, notices, model-card information, and automated license checks. Repository code remains Apache-2.0; model terms are documented separately.
  • Expand guidance on marker inertias, retargeting, BM action scaling and regularization, and pretrained-model licensing.
  • Add focused regression coverage for simulator state, actuators, evaluation lifecycle, observation noise, configuration wiring, and launch/retry behavior.

Compatibility and behavior changes

  • Pretrained weights and serialized configurations are unchanged. No model retraining or weight conversion is included.
  • Simulator fixes and restored observation noise can change behavior even when loading an unchanged configuration. These are correctness changes, not a guarantee of identical trajectories.
  • The BM recipe changes apply when building new configurations, not when resuming existing saved ones.
  • MuJoCo keyboard interaction changes from a separate window to the simulation viewer.

Validation

The release implementation was checked with:

  • CPU suite: 1,734 passed, 6 skipped.
  • LFS-data tests: 17 passed.
  • Newton explicit-control tests with the pinned dependency stack: 2 passed.
  • Release-fence checks: 20 passed, including serialized configuration strings.
  • Ruff on changed Python files, whitespace checks, and model-license checks.
  • Review of the projected file inventory, private references, and common credential patterns; no new private content was found. This does not certify the absence of every possible secret pattern.

Known limitations

  • Sequential scene assignment guarantees the intended mapping for single-object scenes; general multi-object assignment remains limited.
  • Newton tests ran on CPU. CUDA mesh-mesh contacts were not validated by those tests.
  • These regression results do not establish training quality or hardware performance.

Merge

Please use Rebase and merge to preserve the release author and sign-off. The release is a public-tree snapshot; development history is not included.

Correct MuJoCo velocity frames and actuator torque handling, integrate task
keyboard controls into its viewer, and improve video recording.

Improve IsaacLab scene cloning and evaluation lifecycle handling, restore model
rollout context after evaluation, and flush stale contacts before restoration.
Sequential scene assignment currently guarantees the intended mapping for
single-object scenes; general multi-object scene assignment remains limited.

Normalize massless robot links before simulator import and repair Newton's
explicit proportional and torque control paths. Correct SOMA pelvis keypoints
used by robot retargeting.

Improve configuration validation, termination threshold forwarding, observation
noise propagation, cluster retry handling, and artifact copy-back reliability.
Add regression tests and clarify pretrained model licensing and notices.

Align newly built BeyondMimic controller configs with quarter-effort action
scaling and use a squared-L2 raw-action-rate penalty in the BM L2C2 recipe.
Existing saved controller configs retain their original scales and bindings.

Pretrained model weights and serialized model configurations are unchanged.

Signed-off-by: protomotions <protomotions.nvidia@gmail.com>
Signed-off-by: protomotions <protomotions.nvidia@gmail.com>
@tesslerc
tesslerc requested a review from jyf588 October 16, 2026 19:45
@tesslerc
tesslerc merged commit f69fdca into main Oct 16, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants