diff --git a/.gitattributes b/.gitattributes index ce629912cf..b40c667960 100644 --- a/.gitattributes +++ b/.gitattributes @@ -21,6 +21,10 @@ # Video assets are tracked by default. *.mp4 filter=lfs diff=lfs merge=lfs -text +# Audio assets are tracked by default. +*.wav filter=lfs diff=lfs merge=lfs -text +*.mp3 filter=lfs diff=lfs merge=lfs -text + # # Exceptions for small source SVGs (webpack-inlined; not binary assets). deps/cloudxr/webxr_client/src/icons/*.svg -filter -diff -merge text diff --git a/README.md b/README.md index 8f9da8f6b7..d6129dc322 100644 --- a/README.md +++ b/README.md @@ -69,6 +69,10 @@ Our [documentation page](https://nvidia.github.io/IsaacCapture) provides everyth - [Quick installation steps](https://nvidia.github.io/IsaacCapture/main/getting_started/quick_start.html) - [How to build from source](https://nvidia.github.io/IsaacCapture/main/getting_started/build_from_source.html) +### Check a Device Integration + +The CTS records one scripted take from a hand-tracking or full-body device and checks it against the data a retargeter needs; start with the [hand](src/python/isaaccapture/cts/hand/README.md) or [full-body](src/python/isaaccapture/cts/full_body/README.md) guide. + ### Install & Run Isaac Lab diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 93bb81be8f..3675231991 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -33,7 +33,9 @@ Images follow `.gitattributes` like everything else, Git LFS included. A checkou LFS holds pointer text and Sphinx copies it into the site verbatim, so a page that looks right locally can still ship broken artwork. Commit artwork together with the page or data that references it. Stage new artwork with a plain `git add` before running pre-commit: -`git add -N` bypasses the LFS filter, and `check-lfs-pointers` rejects the raw blob. +`git add -N` bypasses the LFS filter, and `check-lfs-pointers` rejects the raw blob. A clone +whose hooks pre-commit installed has no LFS pre-push hook, so GitHub declines a push that +adds artwork; run `git lfs push origin ` first. ## Device pages stand alone @@ -96,6 +98,14 @@ device page documents it, links belong in the Details panel, and the sentence it dropped. The build keeps `company` that narrow by rejecting every field but `name`, `logo`, and `logo_dark`. +## Conformance pages name no device maker + +`source/conformance/` and its media are vendor-neutral: no glove, suit or tracker company +or product appears in prose, cards, commands, take names or plugin names, because a named +example reads as an endorsement. Use a fictional device (Glove X with `glove_x_plugin`, +Suit X with `body.suit-x`) and say it is a placeholder. Headsets (PICO, Quest) are the +exception: every take runs on one. + ## Removing cache overrides Keep cache-removal documentation focused on migration impact. Omit asset-fetch diff --git a/docs/source/_static/css/custom.css b/docs/source/_static/css/custom.css index ebaec1fd17..ba0472c911 100644 --- a/docs/source/_static/css/custom.css +++ b/docs/source/_static/css/custom.css @@ -273,3 +273,25 @@ figure a.no-image-zoom:has(> img) { .trademark-notice { color: var(--pst-color-text-muted) !important; } + +/* Device Conformance pages (rst-class "cts-page"): long check names overflow the step cards, and + long literals overflow the page on a phone. Sphinx sets span.pre to nowrap; lifting it on desktop + body text would split options like --json at the hyphen, so outside cards it only applies below + 768px. Card literals are smaller so a 45-character check name fits a desktop card. */ +.cts-page .sd-card code.literal { + font-size: 0.8em; +} + +.cts-page .sd-card code.literal, +.cts-page .sd-card code.literal span.pre { + overflow-wrap: anywhere; + white-space: normal; +} + +@media (max-width: 767.98px) { + .cts-page code.literal, + .cts-page code.literal span.pre { + overflow-wrap: anywhere; + white-space: normal; + } +} diff --git a/docs/source/_static/cts/full-body-checker-panel.png b/docs/source/_static/cts/full-body-checker-panel.png new file mode 100644 index 0000000000..6889600d53 --- /dev/null +++ b/docs/source/_static/cts/full-body-checker-panel.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ab2662e91c43953441ec1ac2503ba49636081342e57fd785c1eed9f643d7e7bf +size 217862 diff --git a/docs/source/_static/cts/full-body-pacing.svg b/docs/source/_static/cts/full-body-pacing.svg new file mode 100644 index 0000000000..0810c22436 --- /dev/null +++ b/docs/source/_static/cts/full-body-pacing.svg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:09672abb426832f6577811b163dbd6454508295437849b3a6eeae19e807f8d7d +size 10292 diff --git a/docs/source/_static/cts/full_body/01-a_pose_still.gif b/docs/source/_static/cts/full_body/01-a_pose_still.gif new file mode 100644 index 0000000000..6525814bfb --- /dev/null +++ b/docs/source/_static/cts/full_body/01-a_pose_still.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e39cdded5d29a13568ea66f5d3d1e677a669d0a2e867deca655936f65104caed +size 136461 diff --git a/docs/source/_static/cts/full_body/02-t_pose_hold_open.gif b/docs/source/_static/cts/full_body/02-t_pose_hold_open.gif new file mode 100644 index 0000000000..9701e4a439 --- /dev/null +++ b/docs/source/_static/cts/full_body/02-t_pose_hold_open.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:21aa4b6afeec07ba8b4d48cf4d5632f4759c4905955faa3c746685fe8d0ff536 +size 136037 diff --git a/docs/source/_static/cts/full_body/03-neutral_stance.gif b/docs/source/_static/cts/full_body/03-neutral_stance.gif new file mode 100644 index 0000000000..cd362545ca --- /dev/null +++ b/docs/source/_static/cts/full_body/03-neutral_stance.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da241c1d0710328e6e599a65efd6ddff5d76f1b1090e9b7a580b4b9655004a49 +size 122321 diff --git a/docs/source/_static/cts/full_body/04-left_arm_raise.gif b/docs/source/_static/cts/full_body/04-left_arm_raise.gif new file mode 100644 index 0000000000..b1bd61ec0a --- /dev/null +++ b/docs/source/_static/cts/full_body/04-left_arm_raise.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:06554a922204a0a5bdf6fd32bf2863f1e6fc75455b5d5704ab759dc268b01fb9 +size 169180 diff --git a/docs/source/_static/cts/full_body/05-right_arm_raise.gif b/docs/source/_static/cts/full_body/05-right_arm_raise.gif new file mode 100644 index 0000000000..caf49540fc --- /dev/null +++ b/docs/source/_static/cts/full_body/05-right_arm_raise.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:38818f068779e30427922b9ffbb9d454f183b7fa50da210ce2324884bcc44e50 +size 174498 diff --git a/docs/source/_static/cts/full_body/06-left_leg_raise.gif b/docs/source/_static/cts/full_body/06-left_leg_raise.gif new file mode 100644 index 0000000000..d97cf5dc42 --- /dev/null +++ b/docs/source/_static/cts/full_body/06-left_leg_raise.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f5f4f2e60749d7501b405ad292b3fe39e0b98bc4e5c2da807abd7a59ae47141d +size 211280 diff --git a/docs/source/_static/cts/full_body/07-right_leg_raise.gif b/docs/source/_static/cts/full_body/07-right_leg_raise.gif new file mode 100644 index 0000000000..4c0493f7fe --- /dev/null +++ b/docs/source/_static/cts/full_body/07-right_leg_raise.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fdc856823a78ef570944887231bf9f673cda3e744659895272b9040956572f50 +size 205586 diff --git a/docs/source/_static/cts/full_body/08-squat_x2.gif b/docs/source/_static/cts/full_body/08-squat_x2.gif new file mode 100644 index 0000000000..b49fae1065 --- /dev/null +++ b/docs/source/_static/cts/full_body/08-squat_x2.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa6d7fb6b4429823e2c55092ef2122134992a07cf0a126b20e87268610eee87c +size 800322 diff --git a/docs/source/_static/cts/full_body/09-march_in_place.gif b/docs/source/_static/cts/full_body/09-march_in_place.gif new file mode 100644 index 0000000000..26bf639be3 --- /dev/null +++ b/docs/source/_static/cts/full_body/09-march_in_place.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2fd2dcba71bb1403855d1dfa0f688c44c540e004bb2f37f4606a241acdccca30 +size 293609 diff --git a/docs/source/_static/cts/full_body/10-t_pose_hold_close.gif b/docs/source/_static/cts/full_body/10-t_pose_hold_close.gif new file mode 100644 index 0000000000..26ab31555e --- /dev/null +++ b/docs/source/_static/cts/full_body/10-t_pose_hold_close.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c942c42a3ddbc419220f4de095d36f8fbba8368b85f0051e3b622a345a740314 +size 136102 diff --git a/docs/source/_static/cts/hand-checker-panel.png b/docs/source/_static/cts/hand-checker-panel.png new file mode 100644 index 0000000000..d87afed974 --- /dev/null +++ b/docs/source/_static/cts/hand-checker-panel.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6c741a345fc162f3a15f561750aa22b23d7d97b44f783e579691ffd5d1f24a1d +size 309480 diff --git a/docs/source/_static/cts/hand-pacing.svg b/docs/source/_static/cts/hand-pacing.svg new file mode 100644 index 0000000000..0e2d18d7d1 --- /dev/null +++ b/docs/source/_static/cts/hand-pacing.svg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7445bf9b6e8621d895563c8be87fc44c652a3b7472dc833283faf5638231967a +size 9931 diff --git a/docs/source/_static/cts/hand/01-flat_on_table_open.gif b/docs/source/_static/cts/hand/01-flat_on_table_open.gif new file mode 100644 index 0000000000..b3646c6274 --- /dev/null +++ b/docs/source/_static/cts/hand/01-flat_on_table_open.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5efd6322f6a0cc6b5aca8e8c03b7b8a9b61fba112c93f751b5c9b19c89023c4a +size 34098 diff --git a/docs/source/_static/cts/hand/02-fist.gif b/docs/source/_static/cts/hand/02-fist.gif new file mode 100644 index 0000000000..e5005e6c09 --- /dev/null +++ b/docs/source/_static/cts/hand/02-fist.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b65bbc021b3051f40c63a3bc38e49deb1e000eee2f8d2d95a689c8ecc8eed602 +size 477136 diff --git a/docs/source/_static/cts/hand/03-pinch_index.gif b/docs/source/_static/cts/hand/03-pinch_index.gif new file mode 100644 index 0000000000..89276b0e0a --- /dev/null +++ b/docs/source/_static/cts/hand/03-pinch_index.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c450a651e61d5d83a80f37d84ccf0e059a8830a08e2f06cab156c987efec7274 +size 182979 diff --git a/docs/source/_static/cts/hand/04-pinch_middle.gif b/docs/source/_static/cts/hand/04-pinch_middle.gif new file mode 100644 index 0000000000..95e0654978 --- /dev/null +++ b/docs/source/_static/cts/hand/04-pinch_middle.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:701762512cbef337e691f3d2c8265e8191514e1c60ef1e883ac8cd2e7ecd94c9 +size 182828 diff --git a/docs/source/_static/cts/hand/05-pinch_ring.gif b/docs/source/_static/cts/hand/05-pinch_ring.gif new file mode 100644 index 0000000000..2849e4d356 --- /dev/null +++ b/docs/source/_static/cts/hand/05-pinch_ring.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e3ba4d8c714a99c012a6a5fd02c51688f12a6e206b971c4cf7ac4376e6fcc35 +size 184029 diff --git a/docs/source/_static/cts/hand/06-pinch_little.gif b/docs/source/_static/cts/hand/06-pinch_little.gif new file mode 100644 index 0000000000..7ded49e9a8 --- /dev/null +++ b/docs/source/_static/cts/hand/06-pinch_little.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8afc21a9256d3cc8b3e3b0175ef0c027ec1965e8700f7910a57dadcead34e380 +size 189735 diff --git a/docs/source/_static/cts/hand/07-palms_together.gif b/docs/source/_static/cts/hand/07-palms_together.gif new file mode 100644 index 0000000000..8dd118bf22 --- /dev/null +++ b/docs/source/_static/cts/hand/07-palms_together.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:10cfc252efef9dac72f32250a178a50739c90e93f9a2c91db05e64d6a49721f7 +size 425270 diff --git a/docs/source/_static/cts/hand/08-tips_together.gif b/docs/source/_static/cts/hand/08-tips_together.gif new file mode 100644 index 0000000000..f309c946fa --- /dev/null +++ b/docs/source/_static/cts/hand/08-tips_together.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d799568ed691486daa5b4a3109008c9e19a54c823c15fb6b607db9be8fa4d0d7 +size 185944 diff --git a/docs/source/_static/cts/hand/09-right_tip_roll.gif b/docs/source/_static/cts/hand/09-right_tip_roll.gif new file mode 100644 index 0000000000..feb5ba042b --- /dev/null +++ b/docs/source/_static/cts/hand/09-right_tip_roll.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5279c9c91846d429710385e3673bfa36b12fa1a8f6792885e08ac0cb68744b1 +size 535406 diff --git a/docs/source/_static/cts/hand/10-left_tip_roll.gif b/docs/source/_static/cts/hand/10-left_tip_roll.gif new file mode 100644 index 0000000000..9f92f198bc --- /dev/null +++ b/docs/source/_static/cts/hand/10-left_tip_roll.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3d65c4bc49de43b658231668d9566ade89ff65e51648fd76a3382f3f83fcaac4 +size 529845 diff --git a/docs/source/_static/cts/hand/11-flat_on_table_close.gif b/docs/source/_static/cts/hand/11-flat_on_table_close.gif new file mode 100644 index 0000000000..b60373938c --- /dev/null +++ b/docs/source/_static/cts/hand/11-flat_on_table_close.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b475b8ffb30e6db4c25e46a7397e598ab23da944a0aca0a22afa7faa04532e1d +size 34059 diff --git a/docs/source/_static/cts/verdicts.svg b/docs/source/_static/cts/verdicts.svg new file mode 100644 index 0000000000..ef40332723 --- /dev/null +++ b/docs/source/_static/cts/verdicts.svg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a92c56efdced2c8925d88d2fc818176c95bf29228b460a888f0d83bd71ced6f4 +size 4266 diff --git a/docs/source/conformance/full_body.rst b/docs/source/conformance/full_body.rst new file mode 100644 index 0000000000..dcdc0e3f3c --- /dev/null +++ b/docs/source/conformance/full_body.rst @@ -0,0 +1,354 @@ +.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +.. SPDX-License-Identifier: Apache-2.0 + +.. _cts-full-body: + +.. rst-class:: cts-page + +Full-Body Suite +=============== + +The full-body suite records one take of ten standing poses and motions, then checks the +``core.FullBodyPoseRecord`` stream in it: the 24-joint skeleton of the OpenXR +``XR_BD_body_tracking`` extension, pelvis to hands, in the joint order listed in +:doc:`/device/body_tracking`. + +- **PICO body tracking** is the default source (``body.pico-xr``). Set up, calibrate and + activate the headset and Motion Trackers as described in :doc:`/device/body_tracking`. +- **Your own suit or mocap system** records through a vendor backend and its plugin + (:ref:`vendor-selection`). The backend must publish those 24 indices right-handed, +Y up, in metres, with the subject facing + −Z (:code-file:`oracle/README.md `). + A CloudXR headset must stay connected for the OpenXR session, but it needs no body tracking. + +.. list-table:: At a glance + :widths: 25 75 + + * - Steps + - 10, standing: four held poses and six motions + * - Time + - About 1.5 minutes, plus the time you take to get into each pose + +.. _cts-full-body-pacing: + +How each step runs +------------------ + +.. figure:: /_static/cts/full-body-pacing.svg + :alt: The ten timed steps in a row, and step 2 zoomed in: spoken cue, get into the pose with no time limit, press, beep, hold for the countdown, tick. + :width: 100% + +- **Audio is optional.** The capture panel shows each cue as text. To hear the cues, connect + speakers to the workstation; they do not play in the headset. +- Once the capture panel reads *ready when you are*, press the controller trigger, or space in + the capture panel's browser tab (not the terminal). +- A press at any other moment is ignored on purpose: no reaction, nothing queued. +- A press during the cue cuts the cue short. Press again when you are ready. +- Held poses (steps 1–3, 10): get into the pose, then press. Motions (4–9): press, then move. +- Held-pose measurements use the last 60% of the window, so settle early. + +.. _cts-full-body-steps: + +The ten steps +------------- + +In each motion step, the step's main joint must swing at least 15° +(``segmentation.labelled_step_actually_performed``). Checks marked *measured* only report a +number; they do not decide the verdict. + +.. grid:: 1 2 2 2 + :gutter: 3 + + .. grid-item-card:: 1 · A-pose, still + :img-top: /_static/cts/full_body/01-a_pose_still.gif + :img-alt: A figure standing still with straight arms angled down and out, in an A shape. + + :bdg-secondary:`hold 5 s` *"Arms down. Stand still."* + + Feet hip-width apart, arms straight and angled down away from your sides, eyes ahead. Stay + still until the tick. + + +++ + Not judged on its own; the signal and geometry checks cover the whole take, this window + included. + + .. grid-item-card:: 2 · T-pose + :img-top: /_static/cts/full_body/02-t_pose_hold_open.gif + :img-alt: A figure holding both arms straight out to the sides. + + :bdg-secondary:`hold 6 s` *"T pose. Hold."* + + Both arms straight out to the sides, level with the floor, palms down. Hold still. + + +++ + Measured: how far the arms sit below horizontal and how the two arms differ + (``posture.tpose_arm_droop``, ``posture.tpose_left_right_asymmetry``). + + .. grid-item-card:: 3 · Neutral stance + :img-top: /_static/cts/full_body/03-neutral_stance.gif + :img-alt: A figure standing relaxed with arms down. + + :bdg-secondary:`hold 7 s` *"Arms down. Relax."* + + Lower your arms from the T-pose and stand relaxed and still. + + +++ + Measured: the resting hip angles that the knee raises are compared with + (``posture.contralateral_crosstalk_single_leg_raise``). + + .. grid-item-card:: 4 · Left arm up + :img-top: /_static/cts/full_body/04-left_arm_raise.gif + :img-alt: A figure raising its left arm sideways to overhead and lowering it. + + :bdg-secondary:`move 5 s` *"Left arm up, overhead."* + + After the press, raise your straight left arm sideways to overhead and lower it. Stay upright. + + +++ + Checked: hand at least 60° above the shoulder (``posture.arm_raise_range_of_motion``), + torso lean 6° or less (``performance.arm_raise_torso_stability``), and no other arm or leg + swings 10° more (``segmentation.step_order_matches_labels``). + + .. grid-item-card:: 5 · Right arm up + :img-top: /_static/cts/full_body/05-right_arm_raise.gif + :img-alt: A figure raising its right arm sideways to overhead and lowering it. + + :bdg-secondary:`move 5 s` *"Right arm up, overhead."* + + After the press, raise your straight right arm sideways to overhead and lower it. Stay upright. + + +++ + Checked: hand at least 60° above the shoulder (``posture.arm_raise_range_of_motion``), + torso lean 6° or less (``performance.arm_raise_torso_stability``), and no other arm or leg + swings 10° more (``segmentation.step_order_matches_labels``). + + .. grid-item-card:: 6 · Left knee up + :img-top: /_static/cts/full_body/06-left_leg_raise.gif + :img-alt: A figure lifting its left knee to about hip height and putting it down. + + :bdg-secondary:`move 5 s` *"Left knee up."* + + After the press, lift your left knee to about hip height and lower it. Right foot stays put. + + +++ + Checked: the left hip swings at least 15° and no other arm or leg swings 10° more + (``segmentation.labelled_step_actually_performed``, ``segmentation.step_order_matches_labels``). + Measured: how much the still hip moves (``posture.contralateral_crosstalk_single_leg_raise``). + + .. grid-item-card:: 7 · Right knee up + :img-top: /_static/cts/full_body/07-right_leg_raise.gif + :img-alt: A figure lifting its right knee to about hip height and putting it down. + + :bdg-secondary:`move 5 s` *"Right knee up."* + + After the press, lift your right knee to about hip height and lower it. Left foot stays put. + + +++ + Checked: the right hip swings at least 15° and no other arm or leg swings 10° more + (``segmentation.labelled_step_actually_performed``, ``segmentation.step_order_matches_labels``). + Measured: how much the still hip moves (``posture.contralateral_crosstalk_single_leg_raise``). + + .. grid-item-card:: 8 · Squat twice + :img-top: /_static/cts/full_body/08-squat_x2.gif + :img-alt: A figure squatting slowly and standing up, twice. + + :bdg-secondary:`move 10 s` *"Squat twice, slowly."* + + After the press, squat slowly and stand fully up, twice, same depth, weight on both legs. + + +++ + Checked: hip flexion reaches at least 20° (``performance.squat_depth_sufficient``), the + two reps agree within 20% (``posture.squat_rep_repeatability``), and the knees agree + within 10° at the bottom (``posture.squat_knee_symmetry``). + + .. grid-item-card:: 9 · March in place + :img-top: /_static/cts/full_body/09-march_in_place.gif + :img-alt: A figure marching on the spot, lifting its knees in turn. + + :bdg-secondary:`move 6 s` *"March in place."* + + After the press, lift your knees in turn at a steady pace, without walking forward. + + +++ + Checked: the legs alternate (``posture.march_ankle_antiphase``), and the left knee lifts + at least three times with intervals varying 25% or less + (``performance.march_cadence_steady``). + + .. grid-item-card:: 10 · T-pose again + :img-top: /_static/cts/full_body/10-t_pose_hold_close.gif + :img-alt: A figure holding both arms straight out to the sides again. + + :bdg-secondary:`hold 6 s` *"T pose again. Hold."* + + Same as step 2: arms straight out, level with the floor, palms down. + + +++ + Measured: how far the same pose has drifted since step 2 + (``posture.cumulative_drift_between_tpose_windows``). + +.. _cts-full-body-rehearse: + +Rehearse +-------- + +- **Work in pairs.** One person performs; the other stands at the screen and presses space. + A suit without controllers in hand always needs the space bar. +- **Walk through the cards once** before recording. The figures face you, so their left arm + is on your right. +- **Clear the space** to raise both arms overhead, squat and march without hitting anything. +- **Practice takes cost nothing.** Every take gets its own directory; nothing is overwritten. + +.. _cts-full-body-record: + +Record +------ + +Set up the tools and connect the headset first (:doc:`setup`). Then, from the repository root: + +.. code-block:: bash + + src/python/isaaccapture/cts/full_body/capture/record.sh pico4u + +The first argument names the device and the take. Open the capture panel at http://localhost:8081 (or +``http://:8081`` from the headset; if the port was taken, use the address the run +prints) and press **Start recording** when the performer is standing ready. The script begins +at the first frame with a valid joint. After 10 s without one, a red box lists what to check; +the take keeps running. + +To record your own device, name the plugin, the vendor backend and its collection. The device +name must come first. ``suitx``, ``suit_x_plugin``, ``body.suit-x`` and ``suit_x`` are +placeholders for a fictional Suit X; replace each with your own: + +.. code-block:: bash + + src/python/isaaccapture/cts/full_body/capture/record.sh suitx \ + --plugin suit_x_plugin \ + --vendor body.suit-x \ + --vendor-param collection_id=suit_x + +- ``--plugin NAME`` is the ``name:`` in the plugin's ``plugin.yaml``, found under + ``install/plugins/`` once installed (``cmake --install build``). ``record.sh`` launches it + with that file's ``args:`` when you press **Start recording**, so do not start it yourself; + do start your suit's own streaming software first. +- ``--vendor`` is the id your backend registers. ``collection_id`` must match the + ``--collection-id=`` in the plugin's ``args:``; omit it if both use the backend's default. A + mismatch records an empty take that looks like a device that never connected. + +After step 10, *"Done. You can stop now."* plays and the capture panel shows a label report: +the first rows check the file; the step rows re-derive seven of the steps from the pelvis, +ankles and hands (A-pose, neutral stance and march have none). One ``BAD`` step row means that +step was mistimed or performed wrongly: record again. Most rows ``BAD`` with numbers like +``pelvis drops 0 cm`` means the joint stream is frozen: look at the device or plugin. A suit +that cannot track the hands flags ``LEFT_HAND`` and ``RIGHT_HAND`` invalid and gets one *all steps BAD* row, fewer than 24 valid joints and an ``n/a`` +arm-raise reach check instead: expected, not a fault. Press Ctrl+C when done. The take is in its +own directory: + +.. code-block:: text + + ~/isaaccapture-cts/pico4u_2026-03-04_101530/ + ├── pico4u_2026-03-04_101530-body.mcap the recording + ├── pico4u_2026-03-04_101530-body.labels.json the step windows + ├── pico4u_2026-03-04_101530-body.json device name, time, repo commit + └── pico4u_2026-03-04_101530-body.log the recorder's output + +Keep the four files together and never rename them: the labels cannot be regenerated. + +.. _cts-full-body-check: + +Check +----- + +Point the checker at the ``.mcap`` inside its take directory, so it finds the labels file +beside it (to check on another machine, copy the whole directory): + +.. code-block:: bash + + src/python/isaaccapture/cts/full_body/checker/.venv/bin/python -m full_body_cts.cli \ + ~/isaaccapture-cts//-body.mcap + +The command prints the verdict and one +line per check, and exits 0 ``pass``, 1 ``fail``, 2 ``retake`` or 3 ``insufficient_data``, or +4 when the check could not run; :ref:`cts-overview-verdicts` says what each means. Add ``--json`` for the machine-readable +report, or run ``--list-checks`` alone to see all 38 checks. The output for a synthetic +reference take, shortened: + +.. code-block:: text + :class: code-100col + + source full_body/oracle/fixtures/posture/posture_golden_full_script.mcap + frames 2050 + topic full_body/full_body schema core.FullBodyPoseRecord + verdict PASS + + [pass] schema.required_field_present.joints joints present on every pose + … + [pass] posture.arm_raise_range_of_motion both hands reach 89 deg above the shoulder + [pass] posture.march_ankle_antiphase the legs alternate (hip correlation -0.68) + [pass] posture.squat_knee_symmetry knees differ by 0.0 deg at depth + [pass] posture.squat_rep_repeatability the two reps agree to 0% of hip flexion + [pass] performance.squat_depth_sufficient the squat reaches 55 deg of hip flexion + [pass] performance.march_cadence_steady cadence holds at 0.90 Hz, step intervals varying 1% + [pass] performance.arm_raise_torso_stability the torso stays within 0.0 deg of upright + +.. _cts-full-body-view: + +View +---- + +The viewer needs the checker environment built with +``full_body/checker/setup_env.sh --panel`` (:doc:`setup`). + +.. code-block:: bash + + src/python/isaaccapture/cts/full_body/checker/.venv/bin/python -m full_body_cts.panel \ + ~/isaaccapture-cts//-body.mcap + +It prints the same report, then serves http://127.0.0.1:8080; ``--host 0.0.0.0`` opens it to +other machines. If 8080 is taken it moves to the next free port but still prints 8080, so pass a +free ``--port`` instead. The viewer replays the +skeleton (a joint that lost tracking turns red where it was last seen), plots frame rate and +valid joints over the take, and lists every check by group, with nine key signal and geometry +checks in a *Decides the take* folder on top. Its +**package for submission** button builds the archive you send: see :doc:`submit`. + +.. figure:: /_static/cts/full-body-checker-panel.png + :alt: The full-body viewer: a skeleton in a T-pose on a floor grid, and a sidebar with a + green PASS banner, playback controls, the package for submission button and the checks. + :width: 100% + + The viewer on a synthetic reference take, during the T-pose. + +Troubleshooting +--------------- + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - What you see + - What to do + * - **Start recording** hangs after ``waiting for a system...`` in the terminal + - No headset is connected to CloudXR; connect it (:doc:`setup`) and the take + goes on. Ctrl+C does not work here; close the terminal to get out. + * - *no joint has been valid yet*, then the red *No body data after 10 s* box + - PICO: check body tracking, the trackers and their calibration + (:doc:`/device/body_tracking`). Own device: check the plugin is running and + ``collection_id`` matches. + * - A press does nothing + - It came outside *ready when you are*, or the key went to another window. Click the + capture panel page, then press space. + * - The capture panel says **Spoken cues are off** + - ``aplay`` is missing. Install ``alsa-utils`` and restart the panel to hear the cues, or + record with the on-screen cues only. + * - An error naming a Git LFS pointer + - Run ``git lfs pull`` (:doc:`setup`), then record again. + * - *the session did not open: …*, on the capture panel + - Nothing was recorded. *Required plugin … was not discovered*: ``--plugin`` must match the + ``plugin.yaml`` ``name:`` of an installed plugin. *no live vendor* or *does not support + parameter*: the backend is not in the installed wheel, or the ``--vendor-param`` key is + wrong; rebuild and re-run ``capture/setup_env.sh``. Then run ``record.sh`` again. + * - Verdict ``pass``, posture checks ``n/a``, and the note *no motion-step labels beside the + recording* + - The checker found no labels. Run it on the recording in its take directory. + +More in :code-file:`full_body/README.md ` and +:code-file:`full_body/checker/README.md `. diff --git a/docs/source/conformance/hand-rehearsal.mp3 b/docs/source/conformance/hand-rehearsal.mp3 new file mode 100644 index 0000000000..1a51919835 --- /dev/null +++ b/docs/source/conformance/hand-rehearsal.mp3 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4a976f38e370e12c634ffba4e1d0d5377c2c8f086a2a034373de2eaa38ce4dde +size 941472 diff --git a/docs/source/conformance/hand.rst b/docs/source/conformance/hand.rst new file mode 100644 index 0000000000..47df9abc2f --- /dev/null +++ b/docs/source/conformance/hand.rst @@ -0,0 +1,376 @@ +.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +.. SPDX-License-Identifier: Apache-2.0 + +.. _cts-hand: + +.. rst-class:: cts-page + +Hand Suite +========== + +.. admonition:: Preview + :class: caution + + The hand tolerances are provisional and not yet calibrated against glove + recordings, so treat a result close to a limit as indicative. + +Run this suite if your device publishes hand poses: a glove through its plugin, or a +headset's own optical hand tracking. You record one take of eleven scripted gestures; the checker +reads the recording and its labels and returns a verdict. + +.. list-table:: At a glance + :widths: 20 80 + + * - Data + - ``core.HandPoseRecord``, left and right hand channels + * - Joints + - 26 OpenXR joints per hand, 21 judged. PALM and the four non-thumb metacarpals + are optional, so a 21-joint glove can pass. + * - Script + - 11 steps, seated at a table. About 2 minutes, paced by a timer: you press only + **Start recording**. + * - Checks + - 40, in four groups: signal, hand shape, placement, and the scripted steps + * - Your plugin + - Injects both hands with ``plugin_utils::HandInjector``: the suite records what Isaac Teleop's OpenXR hand tracker reads, not a channel of + your own. Joints in metres, placed in the room, at 30 Hz or more. A glove with no + wrist pose takes one from headset hand tracking or a controller + (``plugin_utils::WristPoseSource``). + +Finish :doc:`setup` first: tool environments, your plugin installed, and the headset +connected through CloudXR. + +.. _cts-hand-pacing: + +How each step runs +------------------ + +.. figure:: /_static/cts/hand-pacing.svg + :alt: Timeline of one take: eleven steps in a row, each a spoken cue, a + three-count count-in, a beep, a hold and a tick. One step is enlarged. + :width: 100% + + The cue is spoken, three counts lead in, a beep opens the hold, a tick + closes it. The capture panel shows **listen**, **get into position** (3, 2, 1), **hold**. + +- **Audio is optional.** The capture panel shows each cue as text, with the count. To hear + the cues, connect speakers to the workstation; they do not play in the headset. Without + speakers, keep the capture panel in the performer's view or have the helper read each cue + aloud; the beep is the moment the panel switches to **hold**. +- **Be in the pose by the beep, then keep still.** Move during the cue and count-in. +- **The first 40% of each hold is ignored**, so a slightly late arrival is fine. +- **The two rolls are measured whole, from beep to tick.** The cue's "at the tone" means + the beep, not the count-in: start turning then, at least 90° out, then back. + +Every step is also checked for the gross motion its name asks for +(``segmentation.labelled_step_actually_performed``); a missed step gives ``retake``. + +.. _cts-hand-steps: + +The eleven steps +---------------- + +Both hands perform every step together, except the two rolls. + +.. grid:: 1 2 2 2 + :gutter: 3 + + .. grid-item-card:: 1 · Flat on the table + :img-top: /_static/cts/hand/01-flat_on_table_open.gif + :img-alt: Both hands lie flat on the table, palms down, fingers together. + + :bdg-secondary:`hold 4 s` *"Rest both hands flat on the table, palms down, + fingers together."* + + Fingers straight and together, pressed onto the table. Do not hover. + +++ + Checked: fingers straight (``posture.flat_hand_extension``), joints in one plane + (``table.fingers_coplanar``), level (``table.level``), both hands at one height + (``table.hands_same_height``). + + .. grid-item-card:: 2 · Fists + :img-top: /_static/cts/hand/02-fist.gif + :img-alt: Two tight fists, thumbs over the fingers; the view circles them to show every side. + + :bdg-secondary:`hold 4 s` *"Make two tight fists."* + + Curl every finger fully, thumb wrapped over the fingers. + +++ + Checked: fingers curled (``posture.fist_closure``), more than in the flat steps + (``segmentation.step_order_matches_labels``). + + .. grid-item-card:: 3 · Pinch: index + :img-top: /_static/cts/hand/03-pinch_index.gif + :img-alt: On both hands the thumb closes onto the index fingertip over one second, then holds. + + :bdg-secondary:`hold 3 s` *"Thumb to index fingertip."* + + Pad to pad on both hands, the other fingers relaxed and open. + +++ + Checked: the tip nearest the thumb is the index (``pinch.finger_identity``). + Measured: thumb-to-tip distance (``pinch.contact_distance``). + + .. grid-item-card:: 4 · Pinch: middle + :img-top: /_static/cts/hand/04-pinch_middle.gif + :img-alt: On both hands the thumb closes onto the middle fingertip over one second, then holds. + + :bdg-secondary:`hold 3 s` *"Thumb to middle fingertip."* + + Pad to pad on both hands, the other fingers relaxed and open. + +++ + Checked: the tip nearest the thumb is the middle (``pinch.finger_identity``). + Measured: thumb-to-tip distance (``pinch.contact_distance``). + + .. grid-item-card:: 5 · Pinch: ring + :img-top: /_static/cts/hand/05-pinch_ring.gif + :img-alt: On both hands the thumb closes onto the ring fingertip over one second, then holds. + + :bdg-secondary:`hold 3 s` *"Thumb to ring fingertip."* + + Pad to pad on both hands, the other fingers relaxed and open. + +++ + Checked: the tip nearest the thumb is the ring (``pinch.finger_identity``). + Measured: thumb-to-tip distance (``pinch.contact_distance``). + + .. grid-item-card:: 6 · Pinch: little + :img-top: /_static/cts/hand/06-pinch_little.gif + :img-alt: On both hands the thumb closes onto the little fingertip over one second, then holds. + + :bdg-secondary:`hold 3 s` *"Thumb to little fingertip."* + + Pad to pad on both hands, the other fingers relaxed and open. + +++ + Checked: the tip nearest the thumb is the little (``pinch.finger_identity``). + Measured: thumb-to-tip distance (``pinch.contact_distance``). + + .. grid-item-card:: 7 · Palms together + :img-top: /_static/cts/hand/07-palms_together.gif + :img-alt: Palms pressed flat together, fingers up, seen first from your side and then from either side. + + :bdg-secondary:`hold 4 s` *"Press your palms together, fingers up."* + + Palms flat against each other, fingers up, thumbs closed alongside the index. + +++ + Checked: same-named fingertips of the two hands coincide + (``palms.fingertip_gap``). + + .. grid-item-card:: 8 · Fingertips together + :img-top: /_static/cts/hand/08-tips_together.gif + :img-alt: Hands raised, palms facing you, move together until the middle fingertips touch. + + :bdg-secondary:`hold 3 s` *"Raise both hands, palms facing you. Touch middle + fingertips."* + + Fingers point at each other; only the two middle fingertips touch. + +++ + Checked: the middle tips touch; this is the still reference for the rolls + (``tips.roll_contact``). + + .. grid-item-card:: 9 · Right tip roll + :img-top: /_static/cts/hand/09-right_tip_roll.gif + :img-alt: Tips touching, the right palm turns out and back; the left hand stays still. + + :bdg-secondary:`turn 6 s` *"Fingertips touching. At the tone, turn the right + palm out, then back."* + + Left hand still. At the beep, turn the right palm out at least 90° and back, + tips touching. + +++ + Checked: the turning tip stays on the still one (``tips.roll_contact``); the + wrist turns with the joints (``consistency.position_orientation_same_frame``). + + .. grid-item-card:: 10 · Left tip roll + :img-top: /_static/cts/hand/10-left_tip_roll.gif + :img-alt: Tips touching, the left palm turns out and back; the right hand stays still. + + :bdg-secondary:`turn 6 s` *"Fingertips touching. At the tone, turn the left + palm out, then back."* + + Right hand still. At the beep, turn the left palm out at least 90° and back, + tips touching. + +++ + Checked: the turning tip stays on the still one (``tips.roll_contact``); the + wrist turns with the joints (``consistency.position_orientation_same_frame``). + + .. grid-item-card:: 11 · Flat on the table again + :img-top: /_static/cts/hand/11-flat_on_table_close.gif + :img-alt: Both hands lie flat on the table again, palms down. + + :bdg-secondary:`hold 4 s` *"Hands flat on the table again."* + + Same as step 1: fingers straight and together, pressed onto the table. + +++ + Checked: the four step 1 checks. Measured: shape and height drift since step 1 + (``drift.flat_hand_between_windows``). + +A roll under 90°, or middle tips more than 8 cm apart in step 8, gives ``retake``. +How often each of the 26 joints is valid is measured (``coverage.all_26_valid_rate``), +not required. + +.. _cts-hand-rehearse: + +Rehearse +-------- + +Download :download:`hand-rehearsal.mp3 ` (about 2 minutes). It +plays the cues, counts, beeps and ticks at the real timing. Perform along with it before +your first take. + +- **Press and touch for real**: hands on the table, palms together, fingertips meeting. +- **You cannot pause or skip.** If a step goes wrong, let it finish and record again. +- **Practice takes cost nothing.** Takes are never overwritten; submit your best one. +- **Have a helper watch the capture panel** while you perform. + +.. _cts-hand-record: + +Record +------ + +Run ``record.sh`` from the repository root. The first argument is required and names +your device: free text that names the take directory and files and is stored as +``device`` in the provenance. The performer wears the connected headset; for headset +hands, or a glove wrist from hand tracking, it must see the hands. + +Headset optical hands, no plugin: + +.. code-block:: bash + + src/python/isaaccapture/cts/hand/capture/record.sh quest3 + +A glove, through its plugin. ``glovex`` and ``glove_x_plugin`` are placeholders for a +fictional Glove X: use your device's name and the ``name:`` from your plugin's +``plugin.yaml``. The plugin is loaded from ``install/plugins/`` (:doc:`setup`), and one that +fails to load stops the take. Environment variables your plugin reads, such as where it takes +the wrist pose from, go on the same command line. + +.. code-block:: bash + + src/python/isaaccapture/cts/hand/capture/record.sh glovex --plugin glove_x_plugin + +``--note TEXT``, ``--plugin-version NAME=VERSION`` and ``--wrist-source SOURCE`` (where a +glove's wrist pose comes from, such as ``hand_tracking`` or ``controller``) are stored with the +take. Then: + +#. Open the capture panel at http://localhost:8081, or ``http://:8081`` from the + headset's browser. If the port was taken, use the URL the run prints. +#. Press **Start recording**. The script starts once both hands have had a valid + joint. A red **No hand data after 10 s** box lists causes; the take keeps running. +#. Perform the eleven steps. After *"Done. You can stop now."* the capture panel writes the + labels and audits them, row by row, ``ok`` or ``BAD``: + + - ``BAD`` on ``hand channels`` or ``segmentation.label_alignment``: one hand sent + no records, or the step windows have no frames. Look at the data stream. + - ``BAD`` on ``segmentation.labelled_step_actually_performed``: a step was missed + (the detail names it). Record another take. + +#. Press Ctrl+C when you have read it. + +.. code-block:: text + + ~/isaaccapture-cts/glovex_2026-03-04_101530/ + ├── glovex_2026-03-04_101530-hand.mcap the recording + ├── glovex_2026-03-04_101530-hand.labels.json one window per step + ├── glovex_2026-03-04_101530-hand.json provenance: device, plugins, commit + └── glovex_2026-03-04_101530-hand.log capture panel output and label audit + +Keep the four files together and never rename them. The labels are stamped during the +take and cannot be regenerated. + +.. _cts-hand-check: + +Check +----- + +Point the checker at the ``.mcap`` inside its take directory, so it finds the labels file +beside it (to check on another machine, copy the whole directory): + +.. code-block:: bash + + src/python/isaaccapture/cts/hand/checker/.venv/bin/python -m hand_cts.cli \ + ~/isaaccapture-cts//-hand.mcap + +It prints the verdict and one line per check, and exits 0 ``pass``, 1 ``fail``, 2 ``retake`` +or 3 ``insufficient_data``, or 4 when the check could not run; :ref:`cts-overview-verdicts` +says what each means and what to do. +Each ``[FAIL]`` line names the check and what it measured; +:code-file:`hand/checker/README.md ` +explains every check. The output for a synthetic reference take, shortened: + +.. code-block:: text + :class: code-100col + + source hand/oracle/fixtures/golden_full_script.mcap + frames 8100 + topic hands/left_hand schema core.HandPoseRecord + verdict PASS + + … + [pass] rate.minimum_rate left: 60.0 Hz; right: 60.0 Hz + [pass] rate.frame_gaps left: no gaps; right: no gaps + [pass] continuity.max_joint_velocity left: peak 1.5 m/s; right: peak 1.5 m/s + … + [pass] posture.fist_closure left: straightness 0.39; right: straightness 0.39 + … + [pass] palms.fingertip_gap index 6 mm, middle 9 mm, ring 6 mm, little 1 mm + … + +If the report says ``no motion-step labels beside the recording``, fix the path before +trusting a ``pass``. ``--json`` prints the machine-readable report, ``--list-checks`` lists +every check without a recording, and ``--check NAME`` runs only that check. + +.. _cts-hand-view: + +View +---- + +The viewer needs the checker environment set up with ``--panel`` (:doc:`setup`): + +.. code-block:: bash + + src/python/isaaccapture/cts/hand/checker/.venv/bin/python -m hand_cts.panel \ + ~/isaaccapture-cts//-hand.mcap + +It prints the same report and the address it serves, http://127.0.0.1:8080 by default +(``--host 0.0.0.0`` to reach it from another machine). The viewer replays both hands with the +current step and lists the checks, the deciding ones first; invalid joints show red. Its +**package for submission** button builds the archive you send: see :doc:`submit`. + +.. figure:: /_static/cts/hand-checker-panel.png + :alt: The hand viewer: both hands lying flat on a table grid, and a sidebar with a green + PASS banner, playback controls, a Submission folder with the package for submission + button, and the checks that decide the take. + :width: 100% + + The viewer on a synthetic reference take, during step 1. + +Troubleshooting +--------------- + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - What you see + - What to do + * - **Start recording** hangs after ``waiting for a system...`` in the terminal + - No headset is connected to CloudXR; connect it (:doc:`setup`) and the take + goes on. Ctrl+C does not work here; close the terminal to get out. + * - **No hand data after 10 s** + - Plugin: it exited, the glove is off, unpaired or uncalibrated, or no wrist + source is tracked. Headset: hand tracking is off or the hands are out of view. + * - The capture panel shows ``21 / 26 valid`` in red + - Expected for a 21-joint glove; only the 21 required joints are judged. + * - The capture panel says **Spoken cues are off** + - ``aplay`` is missing. Install ``alsa-utils`` and restart the panel to hear the cues, or + record with the on-screen cues only. + * - ``… is a Git LFS pointer, not audio`` at startup + - Install ``git-lfs`` and run ``git lfs pull``. + * - *the session did not open: … Required plugin … was not discovered*, or ``--plugin given + but no plugins/ or install/plugins/ in the repo`` + - ``--plugin`` must match the ``name:`` in an installed ``plugin.yaml``. Run + ``cmake --install build`` (:doc:`setup`), then record again. + +More in :code-file:`hand/README.md ` +(recording) and +:code-file:`hand/checker/README.md ` +(every check and the report format). diff --git a/docs/source/conformance/overview.rst b/docs/source/conformance/overview.rst new file mode 100644 index 0000000000..58990f22a5 --- /dev/null +++ b/docs/source/conformance/overview.rst @@ -0,0 +1,194 @@ +.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +.. SPDX-License-Identifier: Apache-2.0 + +.. _cts-overview: + +.. rst-class:: cts-page + +Device Conformance Testing +========================== + +You have integrated a device that publishes hand or full-body poses into Isaac Teleop. The +Conformance Test Suite (CTS) produces the evidence that the integration works: you record one +scripted take, a checker reads only that recording and returns a verdict, and you send the take +through the submission form for your pull request. This page explains the process, the two suites +and what a verdict means. + +How it works +------------ + +.. grid:: 1 2 3 3 + :gutter: 3 + + .. grid-item-card:: 1 · Integrate + :link: /device/add_device + :link-type: doc + + Your device feeds Isaac Teleop's hand or body tracker, through your plugin or the + headset's own tracking. + +++ + Add a New Device + + .. grid-item-card:: 2 · Set up + :link: setup + :link-type: doc + + Build with Python 3.12, create the tool environments, connect the headset. + +++ + Set Up the Tools + + .. grid-item-card:: 3 · Rehearse + + Practice the script until the performer knows every step. + +++ + :ref:`Hand ` · :ref:`Full body ` + + .. grid-item-card:: 4 · Record + + Record one scripted take in the capture panel in your browser. + +++ + :ref:`Hand ` · :ref:`Full body ` + + .. grid-item-card:: 5 · Check + + The checker reads the take and returns one verdict. + +++ + :ref:`Hand ` · :ref:`Full body ` + + .. grid-item-card:: 6 · Submit + :link: submit + :link-type: doc + + Package the take with its report and send it through the submission form. + +++ + Submit Your Results + +.. _cts-overview-suites: + +Pick your suite +--------------- + +Run the suite for the data your device produces. A device that produces both runs both. + +.. grid:: 1 1 2 2 + :gutter: 3 + + .. grid-item-card:: Hand :bdg-warning:`Preview` + :link: hand + :link-type: doc + + - **For:** gloves, or a headset's own optical hand tracking + - **Records:** ``core.HandPoseRecord``, 26 OpenXR joints per hand, 21 of them judged + - **Script:** 11 gestures, seated at a table, about 2 minutes + - **Pacing:** automatic; a spoken cue, a count-in, a beep, then a timed hold + +++ + Hand Suite + + .. grid-item-card:: Full body + :link: full_body + :link-type: doc + + - **For:** headset body tracking, or a mocap suit through a vendor backend + - **Records:** ``core.FullBodyPoseRecord``, 24 joints + - **Script:** 10 poses and motions, standing + - **Pacing:** self-paced; you press when ready, then hold the pose or do the motion + +++ + Full-Body Suite + +What the checker looks at +------------------------- + +.. list-table:: + :header-rows: 1 + :widths: 22 30 48 + + * - Area + - What it asks + - Examples + * - Signal and timing + - Is the stream well formed and steady? + - Finite values, unit quaternions, monotonic timestamps, frame rate and gaps, joint speed + * - Skeleton geometry + - Is it a plausible human skeleton, in metres? + - Units, bone lengths that stay constant, human proportions, joints in their named slots + and on their named sides, handedness + * - Placement and frames + - Are the joints where the body is, in the frame they claim? + - Not parked at the origin, wrist not frozen, quaternion component order, positions and + orientations that agree, +Y up + * - Poses over the scripted steps + - Did each step happen, in order, as labelled? + - Hands flat on a level table, pinch finger identity, arm raised overhead, squat depth, + march rhythm + +The checker needs only the ``mcap`` and ``flatbuffers`` Python packages. It is deterministic, so +we re-run it on your take and get the same verdict. Some checks are only measured and reported; +the suite pages say which. + +.. _cts-overview-verdicts: + +Verdicts +-------- + +.. figure:: /_static/cts/verdicts.svg + :alt: Verdict ladder: a device failure gives fail, else a performance failure gives retake, else + an unanswered required check or no answer at all gives insufficient_data, else pass. + :width: 420px + :align: center + +.. list-table:: + :header-rows: 1 + :widths: 18 10 40 32 + + * - Verdict + - Exit status + - Meaning + - What to do + * - ``pass`` + - 0 + - Nothing in the recording argues against the integration. + - Submit it. + * - ``fail`` + - 1 + - A failure blamed on the device or its plugin. + - Fix the integration and record again, or submit with an explanation. + * - ``retake`` + - 2 + - The take is unusable because of how it was performed. The device is not implicated. + - Rehearse the failed step and record again. + * - ``insufficient_data`` + - 3 + - The recording cannot answer the question: too short, no channel, or a required check could + not run. + - Check that the device streamed on its channel for the whole take, then record again. + * - (none) + - 4 + - The check could not run: a usage error, an unknown check, or a recording or labels file + that cannot be read. + - Fix the command or the file named in the error, then run the check again. + +A take checked without its labels file still gets a verdict, even ``pass``, but every step check +reads ``n/a``: keep the labels file beside the recording. You can submit any verdict with its +report. + +.. dropdown:: Marks in the report + + - ``pass``: answered, and the answer is acceptable. + - ``FAIL``: answered, and the answer is not. Whom the check blames decides ``fail`` or ``retake``. + - ``meas``: measured and reported, not judged. Read the value; it is not an approval. + - ``n/a``: not answered, because data was missing or a check it depends on failed. Never a pass. + - ``note``: an advisory check failed. Reported; it does not move the verdict. + +What you need +------------- + +- A Linux workstation with the repository built for Python 3.12. +- An XR headset connected through CloudXR, even when your device produces the joints. +- Your device's plugin built and installed, if it has one. +- ``aplay`` (``alsa-utils``) for the spoken cues. Without it, or without speakers, the capture + panel's on-screen cues still guide every step. +- Room for the script (a table and chair for hand, standing space for full body), and ideally a + second person at the screen. +- A Google account to sign in to the submission form. + +Start with :doc:`setup`. diff --git a/docs/source/conformance/setup.rst b/docs/source/conformance/setup.rst new file mode 100644 index 0000000000..a265da6e8a --- /dev/null +++ b/docs/source/conformance/setup.rst @@ -0,0 +1,180 @@ +.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +.. SPDX-License-Identifier: Apache-2.0 + +.. _cts-setup: + +.. rst-class:: cts-page + +Set Up the Tools +================ + +This page gets a workstation ready to record: the repository built, the CTS tool environments +created and a headset streaming through CloudXR. Do it once per clone; both suites share every step. +Run the commands from the repository root. + +.. contents:: Steps + :local: + :depth: 1 + :backlinks: none + +1. Check the requirements +------------------------- + +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Requirement + - Details + * - Linux workstation + - Recording runs on Linux. The checker and the viewer also run on macOS, so you can check a + take on another machine. + * - Source clone with ``git-lfs`` + - The CTS runs from a source clone; the ``isaaccapture`` package does not include it. The + spoken cues are Git LFS audio, so install ``git-lfs`` before cloning, or run + ``git lfs pull`` in an existing clone. + * - The repository built for **Python 3.12** + - The capture environment installs the cp312 ``isaaccapture`` wheel. The default build makes + 3.11. See step 2. + * - ``uv``, ``curl`` and ``unzip`` + - The setup scripts use them to build the environments and fetch ``flatc``. + * - ``aplay`` (``alsa-utils``) + - The capture panel plays the spoken cues through it. Without it the panel says so and the + take runs with on-screen cues only. Install with ``sudo apt install alsa-utils``. Speakers + are optional: the cues play on the workstation, not the headset, and the capture panel + also shows each one as text. + * - An XR headset and CloudXR + - Needed even when your device produces the joints: the capture's OpenXR session comes from + CloudXR. See step 4. + * - Your plugin, installed + - ``cmake --install build`` installs every built plugin into ``install/plugins/``; add + ``--component `` to install one, if its plugin declares a component. Headset hand + tracking and PICO body tracking need no plugin. + * - Space and a helper + - Hand: a table and a chair. Full body: room to stand, raise both arms and squat. A second + person at the screen makes either suite easier. + +2. Build the repository +----------------------- + +Configure with Python 3.12 from the start. A build tree keeps the Python version it was configured +with, so delete an existing ``build/`` before switching. + +.. code-block:: bash + + cmake -B build -DISAAC_TELEOP_PYTHON_VERSION=3.12 + cmake --build build --parallel + cmake --install build + +A plugin behind a ``BUILD_PLUGIN_`` option builds only when you add +``-DBUILD_PLUGIN_=ON`` to the first line. + +:doc:`/getting_started/build_from_source/index` lists the build prerequisites, and +:doc:`/references/build` explains per-Python-version build trees. + +3. Create the tool environments +------------------------------- + +Each suite has two environments: the checker's and the capture panel's. Create the checker's first; +it generates the bindings the capture panel reads the recording back with. Run only the suites you +need: + +.. code-block:: bash + + # Hand suite + src/python/isaaccapture/cts/hand/checker/setup_env.sh --panel + src/python/isaaccapture/cts/hand/capture/setup_env.sh + + # Full-body suite + src/python/isaaccapture/cts/full_body/checker/setup_env.sh --panel + src/python/isaaccapture/cts/full_body/capture/setup_env.sh + +``--panel`` adds the viewer that replays a take and packages it for submission. +To only check takes on another machine, such as a Mac, run just the checker script there. The +scripts are idempotent and write only inside their own directory; re-run them after pulling or +rebuilding. + +.. _cts-setup-headset: + +4. Connect the headset +---------------------- + +CloudXR is the OpenXR runtime the capture runs on. **Start recording** waits until a headset has +connected, even when your device produces the joints. Use the capture interpreter of either suite, +and set ``PY`` in each terminal you open: + +.. code-block:: bash + + PY=$PWD/src/python/isaaccapture/cts/hand/capture/.venv/bin/python + # or …/cts/full_body/capture/.venv/bin/python + +#. Start the CloudXR service in a terminal of its own and leave it open. The first time, it shows a + link to the CloudXR licence and asks you to accept it; your answer is remembered. + + .. code-block:: bash + + $PY -m isaaccapture.cloudxr.service run + + It prints where the runtime and proxy logs are written; a headset that will not connect says + why there. Run one service per workstation. + +#. In a second terminal, set ``PY`` again and print the client URL: + + .. code-block:: bash + + $PY -m isaaccapture.cloudxr.webclient --print-only + + It prints the streaming target on its ``stream`` row (``:48322``), then the client URL. + +#. Put the headset on the same network as the workstation. In the headset browser, open + ``https://:48322/`` **first** and accept the self-signed certificate: **Advanced**, then + **Proceed**. Until you do, the client loads but CONNECT fails silently. + +#. Open the client URL and press **CONNECT**. + +.. dropdown:: What the certificate warning looks like + + .. figure:: /_static/cloudxr_accept_cert_not_private.png + :alt: Browser warning that the connection to port 48322 is not private + :width: 420px + + Choose Advanced, then Proceed. The next page reads Certificate Accepted. + +The workstation firewall must allow the CloudXR ports (:ref:`whitelist-firewall-ports`). For +device profiles, a detached service or opening the client over USB, see +:doc:`/references/cloudxr`. For PICO body tracking, set up the trackers first: +:doc:`/device/body_tracking`. + +Troubleshooting +--------------- + +.. list-table:: + :header-rows: 1 + :widths: 45 55 + + * - What you see + - What to do + * - ``no isaaccapture wheel in …/{install,build}/wheels; build the repo first`` + - Build the repository (step 2), then re-run the capture setup script. + * - The wheel will not install into the capture environment + - The wheel is not cp312. Delete ``build/``, rebuild with Python 3.12 (step 2) and re-run + the capture setup script. + * - ``run …/checker/setup_env.sh first; the panel decodes the take it writes`` + - Run the checker's setup script before the capture one. + * - ``bfbs differs from the repo golden`` + - The schema in your checkout no longer matches the one the CTS judges. Restore + ``src/core/schema/`` to the commit you are testing. + * - ``… is a Git LFS pointer, not audio`` + - Install ``git-lfs`` and run ``git lfs pull`` in the repository. + * - The client loads but CONNECT does nothing + - Accept the certificate at ``https://:48322/`` first. Check that the headset is on the + same network and the firewall allows the CloudXR ports. + * - ``No module named 'cryptography'`` when the CloudXR service starts + - The capture environment lacks the CloudXR extras that create the certificate. Run + ``uv pip install --python $PY 'cryptography>=3.0' 'websockets>=14'``, then start the + service again. + * - A second CloudXR service will not start + - One is already running. ``$PY -m isaaccapture.cloudxr.service status`` reports it; + ``service stop`` ends a detached one, Ctrl+C a foreground one. + +Next, record a take: :doc:`hand` or :doc:`full_body`. diff --git a/docs/source/conformance/submit.rst b/docs/source/conformance/submit.rst new file mode 100644 index 0000000000..0f719bf6b2 --- /dev/null +++ b/docs/source/conformance/submit.rst @@ -0,0 +1,103 @@ +.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +.. SPDX-License-Identifier: Apache-2.0 + +.. _cts-submit: + +.. rst-class:: cts-page + +Submit Your Results +=================== + +This page shows how to package a checked take and send it through the submission form for your +integration pull request. You can submit any verdict, as long as its report goes with it. + +.. contents:: Steps + :local: + :depth: 1 + :backlinks: none + +1. Open your integration pull request +------------------------------------- + +Open the pull request that adds your device (:doc:`/device/add_device`, +:doc:`/getting_started/contributing`). Each form submission refers to that pull request. + +2. Record and check one take per suite +-------------------------------------- + +Record and check one take for each suite your device supports: :doc:`hand`, :doc:`full_body`, or +both. Takes are never overwritten, so submit the one you want reviewed. Record from a commit you +have pushed to the pull request, with nothing under ``src/python/isaaccapture/cts/`` changed: the +take and its report record the commit and whether the tree was modified, and we re-run the checker +at that commit. + +3. Package the take +------------------- + +Keep the four files of the take together and never rename them. Open the take in the viewer of +its suite: + +.. code-block:: bash + + # Hand + src/python/isaaccapture/cts/hand/checker/.venv/bin/python -m hand_cts.panel \ + ~/isaaccapture-cts//-hand.mcap + # Full body + src/python/isaaccapture/cts/full_body/checker/.venv/bin/python -m full_body_cts.panel \ + ~/isaaccapture-cts//-body.mcap + +If something else uses port 8080, add ``--port`` with a free port. Open the viewer +(http://127.0.0.1:8080 by default) and press **package for submission** in the **Submission** +folder. When the archive is ready, a notification in the viewer offers +``-hand..zip`` or ``-body..zip`` with its size. Click it to save the +archive on the computer running the browser, or right-click it and choose **Save link as** to pick +where. Under the button, the viewer shows the archive's name, size and the first 12 characters of +its SHA-256; you quote them in step 5. + +.. code-block:: text + + glovex_2026-03-04_101530-hand.pass.zip + └── glovex_2026-03-04_101530-hand.pass/ + ├── report.json verdict, checks, hashes, commit + ├── report.txt the same report as text + ├── glovex_2026-03-04_101530-hand.mcap the recording + ├── glovex_2026-03-04_101530-hand.labels.json the step windows + ├── glovex_2026-03-04_101530-hand.json capture provenance + └── glovex_2026-03-04_101530-hand.log recorder log + +4. Submit the form +------------------ + +Fill in the submission form with the link to your pull request and upload the archive from step 3. +Submit the form once for each archive. The form asks you to sign in with a Google account. + +.. button-link:: https://forms.gle/hLqAhNkae4bhAUX79 + :color: primary + + Open the submission form + +5. Describe the take in your pull request +----------------------------------------- + +Add one block per suite to the pull request description: + +.. code-block:: text + + Device conformance + Suite: hand + Take: glovex_2026-03-04_101530 + Verdict: pass + Archive: glovex_2026-03-04_101530-hand.pass.zip, sha256 + Notes: + +6. What happens next +-------------------- + +We re-run the checker on your take at the commit recorded with it, review the replay in the +viewer, and reply on your pull request. + +.. note:: + + The archive holds the performer's recorded motion and the recorder log, which shows file paths + and other details of your workstation. The form sends it to us only; keep it out of the pull + request. diff --git a/docs/source/device/add_device.rst b/docs/source/device/add_device.rst index e3c008fe9b..ef40120ad9 100644 --- a/docs/source/device/add_device.rst +++ b/docs/source/device/add_device.rst @@ -284,3 +284,7 @@ example; ``full_body_printer`` is covered in :doc:`body_tracking/index`. For the shared library build and contribution workflow, see :doc:`/development/build_from_source/index` and :doc:`/development/contributing`. + +.. seealso:: + + Once your device publishes hand or full-body poses, prove it with :doc:`/conformance/overview`. diff --git a/docs/source/index.rst b/docs/source/index.rst index 9fe57ba79f..785a44487d 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -85,6 +85,16 @@ Table of Contents Spatial trackers device/add_device +.. toctree:: + :maxdepth: 1 + :caption: Device Conformance + + Overview + conformance/setup + conformance/hand + conformance/full_body + conformance/submit + .. toctree:: :maxdepth: 2 :caption: References diff --git a/docs/tools/cts_media/README.md b/docs/tools/cts_media/README.md new file mode 100644 index 0000000000..85d3327477 --- /dev/null +++ b/docs/tools/cts_media/README.md @@ -0,0 +1,79 @@ + + +# CTS docs media + +Generates the step GIFs and the hand rehearsal track used by the Device Conformance +pages (`docs/source/conformance/`). The outputs are committed; nothing here runs in CI. +Re-run it, and commit the outputs with the change, when a suite's `capture/session.py` +(`STEPS`, `COUNTDOWN_BEATS`, `BEAT_S`), the hand `capture_panel.py` `TAIL_S`, the hand +cue WAVs or an oracle's motion changes. + +| Output | Made by | +|---|---| +| `docs/source/_static/cts/hand/NN-