Add tactile schemas, trackers, bindings and sources

docs/source/device/add_device.rst

@@ -122,6 +122,9 @@ seven-layer tracker stack. Instead:
    :code-file:`src/core/codegen/generate_trackers.py` and wires generated sources into
    ``deviceio_trackers``, ``live_trackers``, and ``replay_trackers``.
 
+A device with a table that is fixed per producer, such as a sensor layout, declares it with
+``metadata``; see :ref:`Trackers with metadata <generated-trackers-metadata>`.
+
 Use ``direction = "push"`` when Teleop **pushes** a typed table to a plugin (e.g.
 ``HapticCommandPushTracker``). Cross-process **consumers** that bucket multiple
 ``HapticCommand.endpoint`` samples on one collection (e.g. ``HapticCommandReaderTracker`` for Manus

docs/source/references/generated_trackers.rst

@@ -73,6 +73,28 @@ Python edit at all.
 ``.pyi`` stubs need no work either: :code-file:`src/core/python/generate_stubs.py` derives them from
 the built module.
 
+.. _generated-trackers-metadata:
+
+Trackers with metadata
+----------------------
+
+Set ``metadata = "<Table>"`` on a ``pull`` entry when the device also has a table that is fixed per
+producer and too large, or too static, to repeat in every frame (a sensor layout, a calibration).
+The table lives in its own ``.fbs`` with that table as ``root_type`` and no ``Record`` wrapper;
+``metadata_schema`` defaults to the table name in ``snake_case``.
+
+- **Transport.** The producer encodes the table once and pushes it with ``SchemaMetadataPusher``
+  (:code-file:`src/core/pusherio/cpp/inc/pusherio/schema_metadata_pusher.hpp`) as a loop of
+  ``TrackerMetadataChunk`` samples on ``<collection_id>/metadata``, a few per frame, for as long as
+  the data collection exists. :code-file:`src/core/schema/cpp/inc/schema/tracker_metadata.hpp` holds
+  the chunking and reassembly rules.
+- **Consumer.** The facade gains ``get_metadata()``. The tracker adopts the first complete metadata
+  and keeps it for the session, and ``get_data()`` stays empty unless the producer's metadata is the
+  adopted one, so a frame that reaches a consumer can always be read with ``get_metadata()``.
+- **Recording.** The metadata is written once to ``<channel>_metadata``, before any data it lets
+  through; data is recorded only while it would be published. Replay reads the metadata when the
+  session opens.
+
 Generated today
 ---------------
 

examples/CMakeLists.txt

@@ -19,6 +19,7 @@ if(BUILD_EXAMPLES)
     add_subdirectory(deviceio_live_view)
     add_subdirectory(soma_body_publisher)
     add_subdirectory(soma_hand_publisher)
+    add_subdirectory(tactile)
     add_subdirectory(haptic_feedback)
     add_subdirectory(sharpa_avatar)
     add_subdirectory(vehicle_teleop)

examples/tactile/CMakeLists.txt

@@ -0,0 +1,7 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+cmake_minimum_required(VERSION 3.20)
+
+include(${CMAKE_SOURCE_DIR}/cmake/InstallPythonExample.cmake)
+install_python_example(DESTINATION examples/tactile)

examples/tactile/README.md

@@ -0,0 +1,49 @@
+<!--
+SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+SPDX-License-Identifier: Apache-2.0
+-->
+
+# Tactile
+
+A mock tactile device and a consumer for exercising the tactile tracker without hardware.
+
+- `feeder` publishes a 26-joint hand with 200 taxels by default. Each frame carries one
+  `TactileData` and the next chunk of the `TactileShape` metadata. It publishes through
+  `TensorPushTracker` on a `DeviceIOSession`, as a tactile plugin would through `SchemaPusher`
+  and `SchemaMetadataPusher`.
+- `printer` reads one `TactileTracker` directly. It discovers the joint and taxel counts from
+  `TactileShape`, prints the first and last taxel forces, and records the session to MCAP.
+
+```text
+feeder -> "tactile"          (TactileData, every frame)
+       -> "tactile/metadata" (TactileShape chunks, in a loop)
+            -> TactileTracker -> printer, and tactile/* MCAP channels
+```
+
+Install the example:
+
+```bash
+uv pip install -e ./examples/tactile
+```
+
+Start the printer first; it starts the CloudXR runtime, and no headset needs to connect:
+
+```bash
+python -m isaaccapture_examples.tactile.printer --accept-eula --duration 0
+```
+
+Then, in another terminal, start the feeder. It attaches to the same runtime:
+
+```bash
+python -m isaaccapture_examples.tactile.feeder
+```
+
+Stop the printer with `Ctrl+C`; it writes `./recordings/tactile_<timestamp>.mcap`. Print a
+recording with no runtime at all:
+
+```bash
+python -m isaaccapture_examples.tactile.printer --replay recordings/tactile_<timestamp>.mcap
+```
+
+Use feeder options `--joints` and `--taxels` to change the shape; the printer discovers both
+counts from metadata. The two processes need only agree on `--collection-id`.

examples/tactile/pyproject.toml

@@ -0,0 +1,23 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "isaaccapture-examples-tactile"
+version = "0.0.0"
+description = "Isaac Teleop tactile feeder and printer"
+requires-python = ">=3.11,<3.14"
+dependencies = [
+    "isaaccapture[cloudxr]",
+    "numpy>=1.23.0",
+]
+
+[tool.hatch.build.targets.wheel]
+only-include = ["python/isaaccapture_examples/tactile"]
+sources = ["python"]
+
+[tool.uv.sources]
+isaaccapture = { path = "../..", editable = true }

examples/tactile/python/isaaccapture_examples/tactile/__init__.py

@@ -0,0 +1,2 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0

examples/tactile/python/isaaccapture_examples/tactile/common.py

@@ -0,0 +1,14 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+"""Values the feeder and the printer must agree on."""
+
+DEFAULT_COLLECTION_ID = "tactile"
+
+MIN_NUM_JOINTS = 2
+MAX_NUM_JOINTS = 26
+DEFAULT_NUM_JOINTS = MAX_NUM_JOINTS
+DEFAULT_NUM_TAXELS = 200
+
+# TactileTracker's default max_flatbuffer_size: the reader rejects a larger collection.
+TACTILE_SAMPLE_SIZE = 16384

examples/tactile/python/isaaccapture_examples/tactile/feeder.py

@@ -0,0 +1,197 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+"""Publish mock tactile frames and their TactileShape metadata, as a tactile device would.
+
+Each frame pushes one TactileData sample on the data collection and the next chunk(s) of the
+TactileShape on "<collection_id>/metadata", looping over the chunks for as long as it runs.
+Pair it with ``python -m isaaccapture_examples.tactile.printer``.
+"""
+
+import argparse
+import logging
+import math
+import sys
+import time
+from contextlib import ExitStack
+
+import numpy as np
+
+from isaaccapture.cloudxr import CloudXRLauncher
+from isaaccapture.deviceio_session import DeviceIOSession
+from isaaccapture.deviceio_trackers import TensorPushTracker
+from isaaccapture.oxr import OpenXRSession
+from isaaccapture.schema import (
+    TRACKER_METADATA_CHUNK_SIZE,
+    TRACKER_METADATA_TENSOR_IDENTIFIER,
+    TactileData,
+    TactileShape,
+    split_tracker_metadata,
+    tracker_metadata_collection_id,
+)
+
+from .common import (
+    DEFAULT_COLLECTION_ID,
+    DEFAULT_NUM_JOINTS,
+    DEFAULT_NUM_TAXELS,
+    MAX_NUM_JOINTS,
+    MIN_NUM_JOINTS,
+    TACTILE_SAMPLE_SIZE,
+)
+
+logger = logging.getLogger("isaaccapture.examples.tactile.feeder")
+
+WRIST = 1
+# First and distal joint of each finger, thumb to little, as XrHandJointEXT ordinals.
+FINGER_FIRST_JOINTS = (2, 6, 11, 16, 21)
+FINGER_DISTAL_JOINTS = (4, 9, 14, 19, 24)
+
+
+def mock_shape(num_joints: int, num_taxels: int) -> TactileShape:
+    """Build a flat hand in hand-root space with taxels on its distal segments."""
+    positions = np.zeros((num_joints, 3), np.float32)
+    parents = np.full(num_joints, WRIST, np.uint32)
+    parents[WRIST] = TactileShape.ROOT_PARENT
+    positions[0] = (0.0, 0.05, 0.0)
+
+    taxel_joints = []
+    for finger, first in enumerate(FINGER_FIRST_JOINTS):
+        if first >= num_joints:
+            break
+        finger_end = (
+            FINGER_FIRST_JOINTS[finger + 1] - 1 if finger < 4 else MAX_NUM_JOINTS - 1
+        )
+        last = min(finger_end, num_joints - 1)
+        for joint in range(first, last + 1):
+            positions[joint] = (
+                -0.04 + 0.02 * finger,
+                0.02 + 0.025 * (joint - first),
+                0.0,
+            )
+            if joint > first:
+                parents[joint] = joint - 1
+        taxel_joints.append(min(FINGER_DISTAL_JOINTS[finger], last))
+
+    if not taxel_joints:
+        taxel_joints.append(WRIST)
+
+    taxel_positions = np.zeros((num_taxels, 3), np.float32)
+    joint_indices = np.zeros((num_taxels, 4), np.uint32)
+    joint_weights = np.zeros((num_taxels, 4), np.float32)
+    per_segment = math.ceil(num_taxels / len(taxel_joints))
+    for taxel in range(num_taxels):
+        segment, slot = taxel % len(taxel_joints), taxel // len(taxel_joints)
+        joint = taxel_joints[segment]
+        taxel_positions[taxel] = positions[joint] + (
+            0.0,
+            0.02 * slot / per_segment,
+            -0.005,
+        )
+        joint_indices[taxel] = joint
+        joint_weights[taxel, 0] = 1.0
+
+    return TactileShape(
+        joint_bind_positions=positions,
+        joint_bind_orientations=np.tile([0.0, 0.0, 0.0, 1.0], (num_joints, 1)),
+        joint_parents=parents,
+        taxel_positions=taxel_positions,
+        taxel_normals=np.tile([0.0, 0.0, -1.0], (num_taxels, 1)),
+        taxel_tangents=np.tile([1.0, 0.0, 0.0], (num_taxels, 1)),
+        taxel_joint_indices=joint_indices,
+        taxel_joint_weights=joint_weights,
+    )
+
+
+def mock_forces(num_taxels: int, t: float) -> np.ndarray:
+    """Generate a positive 0.5 Hz Z-force wave, phase-shifted per finger."""
+    finger = np.arange(num_taxels) % 5
+    phase = 2.0 * math.pi * 0.5 * t - 1.2 * finger
+    forces = np.zeros((num_taxels, 3), np.float32)
+    forces[:, 2] = 1.25 + 0.75 * np.sin(phase)
+    return forces
+
+
+def main(argv: list[str]) -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--collection-id", default=DEFAULT_COLLECTION_ID)
+    parser.add_argument("--joints", type=int, default=DEFAULT_NUM_JOINTS)
+    parser.add_argument("--taxels", type=int, default=DEFAULT_NUM_TAXELS)
+    parser.add_argument("--rate", type=float, default=60.0, help="Frames per second")
+    parser.add_argument(
+        "--chunks-per-frame",
+        type=int,
+        default=1,
+        help="TactileShape chunks pushed with each frame",
+    )
+    parser.add_argument(
+        "--duration",
+        type=float,
+        default=0.0,
+        help="Seconds to run; 0 runs until Ctrl+C",
+    )
+    CloudXRLauncher.add_launcher_arguments(parser)
+    args = parser.parse_args(argv[1:])
+    if not MIN_NUM_JOINTS <= args.joints <= MAX_NUM_JOINTS:
+        parser.error(f"--joints must be between {MIN_NUM_JOINTS} and {MAX_NUM_JOINTS}")
+    if args.taxels <= 0 or args.chunks_per_frame <= 0:
+        parser.error("--taxels and --chunks-per-frame must be positive")
+    if not math.isfinite(args.rate) or args.rate <= 0:
+        parser.error("--rate must be finite and positive")
+
+    chunks = split_tracker_metadata(mock_shape(args.joints, args.taxels).to_bytes())
+    logger.info(
+        "Publishing %d joints and %d taxels on '%s'; TactileShape in %d chunk(s)",
+        args.joints,
+        args.taxels,
+        args.collection_id,
+        len(chunks),
+    )
+
+    frames = 0
+    pushed_chunks = 0
+    with ExitStack() as stack:
+        stack.enter_context(CloudXRLauncher.launch_context(args))
+        data_tracker = TensorPushTracker(
+            args.collection_id, "tactile", TACTILE_SAMPLE_SIZE
+        )
+        metadata_tracker = TensorPushTracker(
+            tracker_metadata_collection_id(args.collection_id),
+            TRACKER_METADATA_TENSOR_IDENTIFIER,
+            TRACKER_METADATA_CHUNK_SIZE,
+        )
+        trackers = [data_tracker, metadata_tracker]
+        oxr_session = stack.enter_context(
+            OpenXRSession(
+                "TactileFeeder", DeviceIOSession.get_required_extensions(trackers)
+            )
+        )
+        session = stack.enter_context(
+            DeviceIOSession.run(trackers, oxr_session.get_handles())
+        )
+
+        start = time.monotonic()
+        try:
+            while args.duration <= 0 or time.monotonic() - start < args.duration:
+                remaining = start + frames / args.rate - time.monotonic()
+                if remaining > 0:
+                    time.sleep(remaining)
+                session.update()
+                t = time.monotonic() - start
+                data_tracker.push(
+                    session, TactileData(mock_forces(args.taxels, t)).to_bytes()
+                )
+                for _ in range(args.chunks_per_frame):
+                    metadata_tracker.push(session, chunks[pushed_chunks % len(chunks)])
+                    pushed_chunks += 1
+                frames += 1
+                if frames % max(1, int(args.rate * 5)) == 0:
+                    logger.info("Published %d frames", frames)
+        except KeyboardInterrupt:
+            pass
+
+    logger.info("Published %d frames in total", frames)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))