Accelerate and strengthen asset tests - #7282
Conversation
Greptile SummaryThe PR substantially redesigns asset testing around shared contracts, focused backend units, and minimal local integration seams while reducing suite runtime. It also corrects PhysX COM/inertia TensorAPI layout handling, improves deformable classification, and enables hardware-aware OVPhysX manager reuse.
Confidence Score: 5/5The PR appears safe to merge, with no concrete changed-code defect identified. The production changes preserve documented asset buffer ordering and lifecycle boundaries, while the expanded focused tests cover the corresponding PhysX layouts, deformable classification, and OVPhysX mixed-device reuse. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart LR
Contracts["Shared asset contracts"] --> Newton["Newton focused units and local integration"]
Contracts --> PhysX["PhysX focused units and local integration"]
Contracts --> OVPhysX["OVPhysX focused units and local integration"]
PhysX --> Layout["COM and inertia TensorAPI layout fixes"]
PhysX --> Classification["Deformable view classification"]
OVPhysX --> Lifecycle["CPU/CUDA manager lifecycle reuse"]
Newton --> Results["Faster asset-test gates"]
Layout --> Results
Classification --> Results
Lifecycle --> Results
Reviews (1): Last reviewed commit: "Strengthen asset test contracts" | Re-trigger Greptile |
There was a problem hiding this comment.
Isaac Lab Review Bot
The asset-test redesign establishes shared contracts, backend-focused units, and small real-solver seams, with accompanying PhysX and OVPhysX fixes. Before merge, remove the committed benchmark/work-report artifact and eliminate the redundant, unenforced public-member snapshots.
- Design and architecture: The contract/unit/integration ownership split and explicit backend-capability table are coherent. However, the committed redesign-results document is a machine- and revision-specific work artifact prohibited by repository guidance, and
public_surface.pycontains unused duplicate member inventories that can silently diverge from the enforced mappings. - API: The touched production APIs retain their signatures: PhysX collection COM/inertia changes reshape only at the TensorAPI boundary, deformable type inference preserves material-before-topology precedence, and the OVPhysX device lifecycle change affects internal manager behavior. The public-surface audit is useful, but its unused duplicate snapshots should be removed or made authoritative.
- Implementation: The contract bootstrap, backend capability declarations, manager patch scoping, writer readbacks, fused-layout checks, and backend regression seams were traced. Required cleanup is limited to removing the generated benchmark report from the repository and consolidating the duplicated public-member inventory.
Minor fixes needed. Posted 2 actionable findings inline.
Automated review; human maintainers own approval decisions.
| - Times in the comparison are the repository orchestrator's aggregate pytest | ||
| and subprocess wall times, not the outer shell duration. | ||
|
|
||
| ## Copy-ready PR performance section |
There was a problem hiding this comment.
🟡 Warning · Design Architecture — Committed generated benchmark results artifact
This file is a generated work report: it contains a "Copy-ready PR performance section", a rebase-specific base SHA, a machine-specific environment (GPU, driver, wheel versions), sandbox paths such as /tmp/isaaclab-task8-warp, and a per-module disposition map. Repository guidelines prohibit committing generated plans, scratch files, or agent artifacts. It will go stale immediately after merge and carries no reader-facing documentation value. Keep these measurements in the pull-request description instead.
|
|
||
|
|
||
| _PUBLIC_MEMBER_SNAPSHOT = { | ||
| "AssetBase": """ |
There was a problem hiding this comment.
🔵 Suggestion · Design Architecture — Unused member snapshots duplicate enforced mapping
Only the BaseRigidObjectData, BaseRigidObjectCollectionData, and BaseArticulationData entries of _PUBLIC_MEMBER_SNAPSHOT feed PUBLIC_SURFACE_CONTRACTS; the AssetBase, BaseRigidObject, BaseRigidObjectCollection, and BaseArticulation entries are never referenced and respell the same member names that the _mapping(...) calls list inline. A future API change updates only the audited copy, leaving a silently diverging second inventory. Drop the unreferenced entries or build the mappings from them.
Remove Isaac Lab's process device lock and avoid enabling the wheel's sticky CPU-only mode for ordinary CPU scenes. Explicitly author CPU and GPU scene dynamics settings, and cover lifecycle reuse in one process.
Organize the shared asset contracts by API, data, and write behavior while keeping assertion definitions reusable. Declare backend capabilities explicitly and classify the public base surface so missing coverage cannot disappear silently.
Exercise installed PhysX through kitless boundary stubs and classify each contract family with its actual backend capability. Make public-surface coverage an exact, reasoned map to concrete contract targets.
Classify rigid-object and collection cache invalidation under the writes capability. Cover all three asset families in the selector regression.
Replace randomized size matrices and duplicated PhysX scenarios with literal unit coverage and one rotated force-at-position parity case.
Cover distinct global, torque, reset, and raw-buffer paths with literal tests. Make the retained PhysX parity geometry detect rotated induced torque.
Seed an inactive composer with test-only raw data so the merge guard remains observable.
Split backend-specific logic into fast Kitless units and retain a minimal set of local real-solver seams. Move actuator runtime and termination behavior to their owning suites, and fix collection COM and inertia TensorAPI layouts exposed by the new acceptance coverage.
Add literal backend readbacks and temporal actuator contracts that catch cached-only or stale-state regressions. Replace the skipped remote surface-gripper cases with local real and fast unit seams, and isolate Kitless imports from later real imports.
Restore every target-package module imported under lightweight stubs so transitive data classes cannot retain the fake manager. Keep the local SurfaceGripper integration visible to Isaac Sim's short-CI source selector.
Move helper and kernel coverage into fast unit modules and replace remote asset matrices with focused local solver seams. Cover fused collection layouts, native actuator adaptation, deformable lifecycle, and CPU staging while retaining mixed-device behavior.
Verify inertial and friction setters against raw backend bindings. Prove implicit and native actuator commands move the real solver, and own CPU warmup and selective reset behavior with focused tests.
Scope backend manager mocks to tests and classify real solver modules. Keep the fast Newton guard under target and document final speed-ups.
Keep internal design and execution notes out of the review while retaining the measured results artifact. Make the published gate commands self-contained with their cache and EULA prerequisites.
Make fast suites jointly collectable and marker-selectable, replace no-op writer seams with observable state checks, and declare backend limitations explicitly. Keep retained integration fixtures local and hardware-aware.
Reuse one backend scene per articulation module so broader real-solver coverage does not repeatedly rebuild simulation state. Qualify PhysX unit module names to keep mixed collection collision-free and refresh the measured performance report.
Retain the upstream link and center-of-mass alias checks in the reduced Newton integration seams. Refresh the benchmark report against the rebased develop head so the PR speed-up remains reproducible.
d815b16 to
a3d7711
Compare
Align the real articulation probes across Newton, PhysX, and OVPhysX while preserving backend-specific coverage. Exercise partial writes against two live environments in one cached scene so multi-environment behavior is proven without restoring repeated startup cost.
Description
This PR redesigns the asset tests around a clear ownership model:
contracttests (renamed from the oldifacesuite) cover only thepublic base-class API, data, selectors, writers, cache invalidation, and
backend parity;
ordering, target modes, cache behavior, TensorAPI layouts, and unsupported
capability declarations;
asset family;
test.
The articulation integration tests now use one persistent composite scene per
backend. Each module creates isolated two-environment actor islands, resets
once, and exposes the same four core pytest nodes:
Newton retains its model-change notification assertions in the property node.
PhysX adds one spatial-tendon node, while OVPhysX adds spatial-tendon and native
actuator-effort nodes, for totals of 4, 5, and 6 probes respectively. Partial
writes target environment 1 and prove environment 0 is preserved in real
backend state.
This grows real articulation coverage from 7 to 15 probes while reducing their
aggregate warmed wall time from 13.86s to 11.61s. The test functions remain
separate, but scene construction and shutdown are paid only once per module.
Newton's focused asset tests are Kit-free, use locally authored models, and do
not depend on Nucleus. The small composite Newton model runs on CPU because it
was faster locally; a duplicate CPU/GPU matrix is not required for this shared
contract. PhysX and OVPhysX retain GPU execution where it exercises meaningful
backend behavior. Cable and MPM are intentionally unchanged and excluded from
this work.
Assertions removed from the old large integration matrices have explicit new
owners: shared behavior moved to contract tests, backend branches moved to
unit/kernel tests, and real solver seams stayed in integration tests. The final
rebase also preserves upstream Newton root/body link and COM alias assertions
inside the reduced rigid-object integration probes.
The stronger tests exposed and fixed PhysX COM/inertia TensorAPI layout
handling and made OVPhysX device-manager reuse hardware-aware. Shared writer
contracts now verify selected and unselected backend state instead of merely
checking that a call does not raise.
No new required or optional dependency is introduced.
Performance
The five comparable CI-style scopes dropped from 19m57.60s to 2m14.11s
wall time: 8.93x faster overall. Before and after use the same repository
test orchestrator with one fresh subprocess per selected test file, a warmed
writable Warp cache, and the same machine/environment.
The final scopes take 34.06 seconds of aggregate pytest time and 134.11
seconds of subprocess wall time. Controller-owner and OV manager-lifecycle
checks are reported separately and excluded from both comparison denominators.
The committed results report contains the environment, exact commands,
per-suite pytest times, disposition map, and coverage-hole audit.
Type of change
Testing
./isaaclab.sh -f1,238 passed, 103 skippedin 17.92s subprocess walltime
59 passedin 50.57s subprocess wall time44 passedin 31.92s subprocess wall time60 passedin 28.80s subprocess wall time15 passedin 4.90s subprocess wall time15 passedin 11.61s aggregate walltime
Screenshots
Not applicable; this change affects tests and backend correctness.
Checklist
pre-commitchecks with./isaaclab.sh -fsource/<pkg>/changelog.d/for every touched packageCONTRIBUTORS.md