diff --git a/.claude/skills/cmake-structure/SKILL.md b/.claude/skills/cmake-structure/SKILL.md index e7ebf66abf..d995d2c162 100644 --- a/.claude/skills/cmake-structure/SKILL.md +++ b/.claude/skills/cmake-structure/SKILL.md @@ -1,7 +1,7 @@ --- name: cmake-structure description: >- - IsaacTeleop CMake target layout and #include conventions. Use whenever you + Isaac Capture CMake target layout and #include conventions. Use whenever you create or edit a CMakeLists.txt, add or move a module/library/executable/test target, set up target_include_directories or target_link_libraries, decide where a header file goes (private vs public inc/), write or reorder #include @@ -15,7 +15,7 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. SPDX-License-Identifier: Apache-2.0 --> -# IsaacTeleop CMake & include structure +# Isaac Capture CMake & include structure The rules, recommended structure, repo conventions, and review checklist live in the repo's single source of truth: diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 8900a6d8ce..52cc010d76 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -26,7 +26,7 @@ A clear and concise description of what you expected to happen. If applicable, add screenshots to help explain your problem. **Platforms** - - Isaac Teleop Version: [e.g. 1.0.23] + - Isaac Capture Version: [e.g. 1.0.23] - OS: [e.g. Ubuntu X86, ARM64] - Integrations: [e.g. Isaac Lab, Isaac Sim, ROS2] - XR Device: [e.g. Quest 3, Pico 4 Ultra] diff --git a/AGENTS.md b/AGENTS.md index 5d42231f8f..da4db0255f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -22,24 +22,29 @@ SPDX-License-Identifier: Apache-2.0 Repository owners: keep this block at the top of this file; do not delete it. --> -# IsaacTeleop — agent notes +# Isaac Capture — agent notes + +## Product naming + +Isaac Capture names the overall library and repository. Isaac Teleop remains its +teleoperation component; preserve that brand and existing technical identifiers. ## CRITICAL — mandatory preflight (do not skip) -**Hard requirement:** Do **not** edit or add code under `IsaacTeleop/` until you have **read** (e.g. with the Read tool) **every** relevant `AGENTS.md` file as defined below. This is **not** optional, **not** "only for large tasks," and **not** satisfiable by inferring from prior conversation. If you have not read those files in **this** session for **this** task, stop and read them first. +**Hard requirement:** Do **not** edit or add code under `IsaacCapture/` until you have **read** (e.g. with the Read tool) **every** relevant `AGENTS.md` file as defined below. This is **not** optional, **not** "only for large tasks," and **not** satisfiable by inferring from prior conversation. If you have not read those files in **this** session for **this** task, stop and read them first. **You must:** 1. List the **directories** you expect to edit or create files under (e.g. `src/core/live_trackers/cpp/`, `src/core/deviceio_base/cpp/inc/…`). 2. For **each** such directory, **read** **`AGENTS.md` in that directory** if it exists. -3. Walk **up** toward the **IsaacTeleop repo root** and **read** **`AGENTS.md` in every ancestor directory** that has one (e.g. `live_trackers/` → `src/core/` → **`IsaacTeleop/AGENTS.md` (this file)**). +3. Walk **up** toward the **IsaacCapture repo root** and **read** **`AGENTS.md` in every ancestor directory** that has one (e.g. `live_trackers/` → `src/core/` → **`IsaacCapture/AGENTS.md` (this file)**). **You must not** assume that reading the single "closest" `AGENTS.md` is enough. **Multiple** `AGENTS.md` files can apply to one change (e.g. both `live_trackers` and `deviceio_base`). **Listing every `AGENTS.md` in this repo** (no curated table here—add new files under the tree without editing this document): ```bash -# Run from the IsaacTeleop repository root (the directory that contains this file): +# Run from the IsaacCapture repository root (the directory that contains this file): find . -name AGENTS.md ! -path '*/.git/*' | sort ``` @@ -216,7 +221,7 @@ pre-commit install --hook-type commit-msg ## Pre-commit — match CI before you stop -- From the **IsaacTeleop repo root** (this directory), run pre-commit and **fix all failures** before you treat a change as finished (do not only rely on “should pass” reasoning). +- From the **IsaacCapture repo root** (this directory), run pre-commit and **fix all failures** before you treat a change as finished (do not only rely on “should pass” reasoning). - **Use the same hook set as GitHub Actions:** `.github/workflows/pre-commit.yaml` runs pre-commit with **`SKIP=check-copyright-year`**. Mirror that locally: ```bash diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index f6ca355dae..0d46a094f7 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -2,7 +2,7 @@ ## Overview -Define the code of conduct followed and enforced for Isaac Teleop +Define the code of conduct followed and enforced for Isaac Capture ### Intended audience diff --git a/README.md b/README.md index 90a0379079..8cd98527ea 100644 --- a/README.md +++ b/README.md @@ -3,13 +3,11 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. SPDX-License-Identifier: Apache-2.0 --> -![Isaac Teleop](docs/source/_static/isaac-teleop-hero.jpg) - -# Isaac Teleop +# NVIDIA Isaac Capture
-**The unified framework for high-fidelity ego-centric and robotics data collection.** +**Shared capture infrastructure for teleoperation and egocentric data.** [![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/) [![Isaac Lab](https://img.shields.io/badge/Isaac%20Lab-3.0.0-orange.svg)](https://isaac-sim.github.io/IsaacLab/develop) @@ -21,11 +19,20 @@ SPDX-License-Identifier: Apache-2.0 --- + +![Isaac Capture architecture: Isaac Teleop and Ego Pipeline share device plugins, retargeters, and data utilities, with integrations into simulation and real-robot environments](docs/source/_static/isaac-capture-architecture.png) + ## Overview -**Isaac Teleop**: The unified standard for high-fidelity egocentric and robot data collection. -It is designed to address the data bottleneck in robot learning by streamlining device integration; -standardizing high-fidelity human demo data collection; and foster device & data interoperability. +**Isaac Capture** provides shared infrastructure for collecting human and robot demonstration data. +It brings together two components: + +- **Isaac Teleop** supports robot-in-the-loop capture through teleoperation, including Teleop Session and Remote Assist. +- **Ego Pipeline** supports human and egocentric data capture through Ego Capture and Ego Post Processing. + +Both components use a shared core of device plugins, retargeters, and data utilities for recording, +replaying, and converting data. Isaac Teleop remains a component of Isaac Capture, with its existing +functionality preserved. ## Key Features @@ -33,20 +40,25 @@ standardizing high-fidelity human demo data collection; and foster device & data - Standardized device interface - Flexible retargeting framework -### Teleoperation Use Cases +### Isaac Teleop - Currently supported use cases - Use XR headsets for gripper / tri-finger hand manipulation - Use XR headsets with gloves for dex-hand manipulation - Seated full body loco-manipulation (Homie) - Tracking based full body loco-manipulation with GR00T-WholeBodyControl (SONIC) - - Egocentric data collection (aka “no-robot”) - Upcoming use cases - Teleoperate using only non-XR devices (e.g. gamepad, Gello, haply, etc.) - Teleoperate cloud based robotics simulations - Remote teleoperation with camera streaming to the desktop - Remote teleoperation with immersive camera streaming to XR headsets +### Ego Pipeline + +- Capture egocentric data without a robot in the loop. +- Reconstruct hand and camera poses from egocentric video with the + [Ego Post Processing pipeline](https://nvidia.github.io/IsaacCapture/main/references/egocentric_hand_reconstruction.html). + ## Quick Start ### Documentation @@ -60,6 +72,6 @@ Our [documentation page](https://nvidia.github.io/IsaacCapture) provides everyth ### Install & Run Isaac Lab -Isaac Teleop is design to work side by side with [NVIDIA Isaac Lab](https://github.com/isaac-sim/IsaacLab) starting with Isaac Lab [3.0 Beta release](https://github.com/isaac-sim/IsaacLab/releases/tag/v3.0.0-beta). +The latest [NVIDIA Isaac Lab](https://github.com/isaac-sim/IsaacLab) release, [3.0.0 EA](https://github.com/isaac-sim/IsaacLab/releases/tag/v3.0.0-EA), includes Isaac Capture, previously released as Isaac Teleop. To get started, please refer to Isaac Lab's [Installation](https://isaac-sim.github.io/IsaacLab/develop/source/setup/installation/index.html) guide for more details. Then follow the [CloudXR teleoperation in Isaac Lab](https://isaac-sim.github.io/IsaacLab/develop/source/how-to/cloudxr_teleoperation.html) to get started with Teleoperation in Sim. diff --git a/cmake/cmake-structure.md b/cmake/cmake-structure.md index b9b68c3dca..e54255b823 100644 --- a/cmake/cmake-structure.md +++ b/cmake/cmake-structure.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. SPDX-License-Identifier: Apache-2.0 --> -# IsaacTeleop CMake & include structure +# Isaac Capture CMake & include structure Single source of truth for how CMake targets, directories, and `#include` paths are organized in this repo. These rules apply to **all** coding agents and AI diff --git a/deps/README.md b/deps/README.md index aa8ddf9042..0e03882854 100644 --- a/deps/README.md +++ b/deps/README.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. SPDX-License-Identifier: Apache-2.0 --> -# Isaac Teleop Dependencies +# Isaac Capture Dependencies This directory contains all project dependencies, organized by type. @@ -91,7 +91,7 @@ machine and are located via `find_package`. ### pybind11 - **Source**: https://github.com/pybind/pybind11.git - **Version**: a2e59f0e7065404b44dfe92a28aca47ba1378dc4 (v2.11.0-182) -- **Purpose**: C++/Python bindings for Isaac Teleop Python API +- **Purpose**: C++/Python bindings for Isaac Capture Python API - **Build**: Header-only library - **License**: BSD-style diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 769b9578dd..92916db93a 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -65,7 +65,7 @@ for non-HTML builders. ## The external marker follows the target, not the record -The docs site and the IsaacTeleop repository are both this project, so a link into either +The docs site and the IsaacCapture repository are both this project, so a link into either renders with no arrow and no screen-reader "(external)" — tracking links included. Only third-party domains get the marker, and other GitHub organizations (`isaac-sim`) count as third-party. `ecosystem_grid.PROJECT_REPO` is the one place that boundary is written down. diff --git a/docs/source/_static/isaac-capture-architecture.png b/docs/source/_static/isaac-capture-architecture.png new file mode 100644 index 0000000000..2ca7a82c56 --- /dev/null +++ b/docs/source/_static/isaac-capture-architecture.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc5599d0cb0c7fe2ec6d05aee817e07583aa41ec5f6dd2bcfb18dcb429ffa4e2 +size 742022 diff --git a/docs/source/_static/isaac-capture-architecture.png.license b/docs/source/_static/isaac-capture-architecture.png.license new file mode 100644 index 0000000000..817df24424 --- /dev/null +++ b/docs/source/_static/isaac-capture-architecture.png.license @@ -0,0 +1,2 @@ +SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 diff --git a/examples/README.md b/examples/README.md index 4d79155afc..f3d5245ce2 100644 --- a/examples/README.md +++ b/examples/README.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Isaac Teleop examples +# Isaac Capture examples Each subdirectory is a self-contained example. Read one, run it, or copy it into your own project — the last of those is the reason for the layout below. diff --git a/src/core/AGENTS.md b/src/core/AGENTS.md index 706cd668d3..7008bd42e7 100644 --- a/src/core/AGENTS.md +++ b/src/core/AGENTS.md @@ -3,11 +3,11 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. SPDX-License-Identifier: Apache-2.0 --> -# Agent notes — IsaacTeleop `src/core` (index) +# Agent notes — Isaac Capture `src/core` (index) **CRITICAL:** The mandatory multi-file `AGENTS.md` preflight in **[`../../AGENTS.md`](../../AGENTS.md)** applies here too. Before changing anything under `src/core/`, you must have read **this file**, **the repo root `AGENTS.md`**, and **every other `AGENTS.md` on the directory paths you will touch**. Do not skip any of them. -To see **all** `AGENTS.md` files in the IsaacTeleop repo, use the **`find` command (or `**/AGENTS.md` glob) documented in the repo root [`AGENTS.md`](../../AGENTS.md)**—do not rely on a hand-maintained list in this file. +To see **all** `AGENTS.md` files in the Isaac Capture repo, use the **`find` command (or `**/AGENTS.md` glob) documented in the repo root [`AGENTS.md`](../../AGENTS.md)**—do not rely on a hand-maintained list in this file. If work under **`src/core/`** went wrong—**user** correction, **pre-commit/CI** failure, or **repeated** same-class mistakes—you **must** follow the repo root **[`AGENTS.md`](../../AGENTS.md)** **Mandatory learning loop**: distill a short rule and **update** the **nearest** relevant `AGENTS.md` (this file or a package file) or **source comments** in the same session (including **delta vs `main`** scope).