From 83449f0db8c4d5c97beafd828d17f32e121b7d99 Mon Sep 17 00:00:00 2001 From: Jiwen Cai Date: Wed, 14 Oct 2026 02:21:27 +0000 Subject: [PATCH] docs: align Markdown documentation with Isaac Capture Present Isaac Capture as the shared library while retaining Isaac Teleop as its teleoperation component. Include the architecture diagram and updated Isaac Lab reference, and align shared project guidance with the umbrella name. Signed-off-by: Jiwen Cai --- .claude/skills/cmake-structure/SKILL.md | 4 +-- .github/ISSUE_TEMPLATE/bug_report.md | 2 +- AGENTS.md | 15 ++++++--- CODE_OF_CONDUCT.md | 2 +- README.md | 32 +++++++++++++------ cmake/cmake-structure.md | 2 +- deps/README.md | 4 +-- docs/AGENTS.md | 2 +- .../_static/isaac-capture-architecture.png | 3 ++ .../isaac-capture-architecture.png.license | 2 ++ examples/README.md | 2 +- src/core/AGENTS.md | 4 +-- 12 files changed, 48 insertions(+), 26 deletions(-) create mode 100644 docs/source/_static/isaac-capture-architecture.png create mode 100644 docs/source/_static/isaac-capture-architecture.png.license diff --git a/.claude/skills/cmake-structure/SKILL.md b/.claude/skills/cmake-structure/SKILL.md index e7ebf66abf..d995d2c162 100644 --- a/.claude/skills/cmake-structure/SKILL.md +++ b/.claude/skills/cmake-structure/SKILL.md @@ -1,7 +1,7 @@ --- name: cmake-structure description: >- - IsaacTeleop CMake target layout and #include conventions. Use whenever you + Isaac Capture CMake target layout and #include conventions. Use whenever you create or edit a CMakeLists.txt, add or move a module/library/executable/test target, set up target_include_directories or target_link_libraries, decide where a header file goes (private vs public inc/), write or reorder #include @@ -15,7 +15,7 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. SPDX-License-Identifier: Apache-2.0 --> -# IsaacTeleop CMake & include structure +# Isaac Capture CMake & include structure The rules, recommended structure, repo conventions, and review checklist live in the repo's single source of truth: diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 8900a6d8ce..52cc010d76 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -26,7 +26,7 @@ A clear and concise description of what you expected to happen. If applicable, add screenshots to help explain your problem. **Platforms** - - Isaac Teleop Version: [e.g. 1.0.23] + - Isaac Capture Version: [e.g. 1.0.23] - OS: [e.g. Ubuntu X86, ARM64] - Integrations: [e.g. Isaac Lab, Isaac Sim, ROS2] - XR Device: [e.g. Quest 3, Pico 4 Ultra] diff --git a/AGENTS.md b/AGENTS.md index 5d42231f8f..da4db0255f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -22,24 +22,29 @@ SPDX-License-Identifier: Apache-2.0 Repository owners: keep this block at the top of this file; do not delete it. --> -# IsaacTeleop — agent notes +# Isaac Capture — agent notes + +## Product naming + +Isaac Capture names the overall library and repository. Isaac Teleop remains its +teleoperation component; preserve that brand and existing technical identifiers. ## CRITICAL — mandatory preflight (do not skip) -**Hard requirement:** Do **not** edit or add code under `IsaacTeleop/` until you have **read** (e.g. with the Read tool) **every** relevant `AGENTS.md` file as defined below. This is **not** optional, **not** "only for large tasks," and **not** satisfiable by inferring from prior conversation. If you have not read those files in **this** session for **this** task, stop and read them first. +**Hard requirement:** Do **not** edit or add code under `IsaacCapture/` until you have **read** (e.g. with the Read tool) **every** relevant `AGENTS.md` file as defined below. This is **not** optional, **not** "only for large tasks," and **not** satisfiable by inferring from prior conversation. If you have not read those files in **this** session for **this** task, stop and read them first. **You must:** 1. List the **directories** you expect to edit or create files under (e.g. `src/core/live_trackers/cpp/`, `src/core/deviceio_base/cpp/inc/…`). 2. For **each** such directory, **read** **`AGENTS.md` in that directory** if it exists. -3. Walk **up** toward the **IsaacTeleop repo root** and **read** **`AGENTS.md` in every ancestor directory** that has one (e.g. `live_trackers/` → `src/core/` → **`IsaacTeleop/AGENTS.md` (this file)**). +3. Walk **up** toward the **IsaacCapture repo root** and **read** **`AGENTS.md` in every ancestor directory** that has one (e.g. `live_trackers/` → `src/core/` → **`IsaacCapture/AGENTS.md` (this file)**). **You must not** assume that reading the single "closest" `AGENTS.md` is enough. **Multiple** `AGENTS.md` files can apply to one change (e.g. both `live_trackers` and `deviceio_base`). **Listing every `AGENTS.md` in this repo** (no curated table here—add new files under the tree without editing this document): ```bash -# Run from the IsaacTeleop repository root (the directory that contains this file): +# Run from the IsaacCapture repository root (the directory that contains this file): find . -name AGENTS.md ! -path '*/.git/*' | sort ``` @@ -216,7 +221,7 @@ pre-commit install --hook-type commit-msg ## Pre-commit — match CI before you stop -- From the **IsaacTeleop repo root** (this directory), run pre-commit and **fix all failures** before you treat a change as finished (do not only rely on “should pass” reasoning). +- From the **IsaacCapture repo root** (this directory), run pre-commit and **fix all failures** before you treat a change as finished (do not only rely on “should pass” reasoning). - **Use the same hook set as GitHub Actions:** `.github/workflows/pre-commit.yaml` runs pre-commit with **`SKIP=check-copyright-year`**. Mirror that locally: ```bash diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index f6ca355dae..0d46a094f7 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -2,7 +2,7 @@ ## Overview -Define the code of conduct followed and enforced for Isaac Teleop +Define the code of conduct followed and enforced for Isaac Capture ### Intended audience diff --git a/README.md b/README.md index 90a0379079..8cd98527ea 100644 --- a/README.md +++ b/README.md @@ -3,13 +3,11 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. SPDX-License-Identifier: Apache-2.0 --> -![Isaac Teleop](docs/source/_static/isaac-teleop-hero.jpg) - -# Isaac Teleop +# NVIDIA Isaac Capture
-**The unified framework for high-fidelity ego-centric and robotics data collection.** +**Shared capture infrastructure for teleoperation and egocentric data.** [![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/) [![Isaac Lab](https://img.shields.io/badge/Isaac%20Lab-3.0.0-orange.svg)](https://isaac-sim.github.io/IsaacLab/develop) @@ -21,11 +19,20 @@ SPDX-License-Identifier: Apache-2.0 --- + +![Isaac Capture architecture: Isaac Teleop and Ego Pipeline share device plugins, retargeters, and data utilities, with integrations into simulation and real-robot environments](docs/source/_static/isaac-capture-architecture.png) + ## Overview -**Isaac Teleop**: The unified standard for high-fidelity egocentric and robot data collection. -It is designed to address the data bottleneck in robot learning by streamlining device integration; -standardizing high-fidelity human demo data collection; and foster device & data interoperability. +**Isaac Capture** provides shared infrastructure for collecting human and robot demonstration data. +It brings together two components: + +- **Isaac Teleop** supports robot-in-the-loop capture through teleoperation, including Teleop Session and Remote Assist. +- **Ego Pipeline** supports human and egocentric data capture through Ego Capture and Ego Post Processing. + +Both components use a shared core of device plugins, retargeters, and data utilities for recording, +replaying, and converting data. Isaac Teleop remains a component of Isaac Capture, with its existing +functionality preserved. ## Key Features @@ -33,20 +40,25 @@ standardizing high-fidelity human demo data collection; and foster device & data - Standardized device interface - Flexible retargeting framework -### Teleoperation Use Cases +### Isaac Teleop - Currently supported use cases - Use XR headsets for gripper / tri-finger hand manipulation - Use XR headsets with gloves for dex-hand manipulation - Seated full body loco-manipulation (Homie) - Tracking based full body loco-manipulation with GR00T-WholeBodyControl (SONIC) - - Egocentric data collection (aka “no-robot”) - Upcoming use cases - Teleoperate using only non-XR devices (e.g. gamepad, Gello, haply, etc.) - Teleoperate cloud based robotics simulations - Remote teleoperation with camera streaming to the desktop - Remote teleoperation with immersive camera streaming to XR headsets +### Ego Pipeline + +- Capture egocentric data without a robot in the loop. +- Reconstruct hand and camera poses from egocentric video with the + [Ego Post Processing pipeline](https://nvidia.github.io/IsaacCapture/main/references/egocentric_hand_reconstruction.html). + ## Quick Start ### Documentation @@ -60,6 +72,6 @@ Our [documentation page](https://nvidia.github.io/IsaacCapture) provides everyth ### Install & Run Isaac Lab -Isaac Teleop is design to work side by side with [NVIDIA Isaac Lab](https://github.com/isaac-sim/IsaacLab) starting with Isaac Lab [3.0 Beta release](https://github.com/isaac-sim/IsaacLab/releases/tag/v3.0.0-beta). +The latest [NVIDIA Isaac Lab](https://github.com/isaac-sim/IsaacLab) release, [3.0.0 EA](https://github.com/isaac-sim/IsaacLab/releases/tag/v3.0.0-EA), includes Isaac Capture, previously released as Isaac Teleop. To get started, please refer to Isaac Lab's [Installation](https://isaac-sim.github.io/IsaacLab/develop/source/setup/installation/index.html) guide for more details. Then follow the [CloudXR teleoperation in Isaac Lab](https://isaac-sim.github.io/IsaacLab/develop/source/how-to/cloudxr_teleoperation.html) to get started with Teleoperation in Sim. diff --git a/cmake/cmake-structure.md b/cmake/cmake-structure.md index b9b68c3dca..e54255b823 100644 --- a/cmake/cmake-structure.md +++ b/cmake/cmake-structure.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. SPDX-License-Identifier: Apache-2.0 --> -# IsaacTeleop CMake & include structure +# Isaac Capture CMake & include structure Single source of truth for how CMake targets, directories, and `#include` paths are organized in this repo. These rules apply to **all** coding agents and AI diff --git a/deps/README.md b/deps/README.md index aa8ddf9042..0e03882854 100644 --- a/deps/README.md +++ b/deps/README.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. SPDX-License-Identifier: Apache-2.0 --> -# Isaac Teleop Dependencies +# Isaac Capture Dependencies This directory contains all project dependencies, organized by type. @@ -91,7 +91,7 @@ machine and are located via `find_package`. ### pybind11 - **Source**: https://github.com/pybind/pybind11.git - **Version**: a2e59f0e7065404b44dfe92a28aca47ba1378dc4 (v2.11.0-182) -- **Purpose**: C++/Python bindings for Isaac Teleop Python API +- **Purpose**: C++/Python bindings for Isaac Capture Python API - **Build**: Header-only library - **License**: BSD-style diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 769b9578dd..92916db93a 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -65,7 +65,7 @@ for non-HTML builders. ## The external marker follows the target, not the record -The docs site and the IsaacTeleop repository are both this project, so a link into either +The docs site and the IsaacCapture repository are both this project, so a link into either renders with no arrow and no screen-reader "(external)" — tracking links included. Only third-party domains get the marker, and other GitHub organizations (`isaac-sim`) count as third-party. `ecosystem_grid.PROJECT_REPO` is the one place that boundary is written down. diff --git a/docs/source/_static/isaac-capture-architecture.png b/docs/source/_static/isaac-capture-architecture.png new file mode 100644 index 0000000000..2ca7a82c56 --- /dev/null +++ b/docs/source/_static/isaac-capture-architecture.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc5599d0cb0c7fe2ec6d05aee817e07583aa41ec5f6dd2bcfb18dcb429ffa4e2 +size 742022 diff --git a/docs/source/_static/isaac-capture-architecture.png.license b/docs/source/_static/isaac-capture-architecture.png.license new file mode 100644 index 0000000000..817df24424 --- /dev/null +++ b/docs/source/_static/isaac-capture-architecture.png.license @@ -0,0 +1,2 @@ +SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 diff --git a/examples/README.md b/examples/README.md index 4d79155afc..f3d5245ce2 100644 --- a/examples/README.md +++ b/examples/README.md @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All SPDX-License-Identifier: Apache-2.0 --> -# Isaac Teleop examples +# Isaac Capture examples Each subdirectory is a self-contained example. Read one, run it, or copy it into your own project — the last of those is the reason for the layout below. diff --git a/src/core/AGENTS.md b/src/core/AGENTS.md index 706cd668d3..7008bd42e7 100644 --- a/src/core/AGENTS.md +++ b/src/core/AGENTS.md @@ -3,11 +3,11 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. SPDX-License-Identifier: Apache-2.0 --> -# Agent notes — IsaacTeleop `src/core` (index) +# Agent notes — Isaac Capture `src/core` (index) **CRITICAL:** The mandatory multi-file `AGENTS.md` preflight in **[`../../AGENTS.md`](../../AGENTS.md)** applies here too. Before changing anything under `src/core/`, you must have read **this file**, **the repo root `AGENTS.md`**, and **every other `AGENTS.md` on the directory paths you will touch**. Do not skip any of them. -To see **all** `AGENTS.md` files in the IsaacTeleop repo, use the **`find` command (or `**/AGENTS.md` glob) documented in the repo root [`AGENTS.md`](../../AGENTS.md)**—do not rely on a hand-maintained list in this file. +To see **all** `AGENTS.md` files in the Isaac Capture repo, use the **`find` command (or `**/AGENTS.md` glob) documented in the repo root [`AGENTS.md`](../../AGENTS.md)**—do not rely on a hand-maintained list in this file. If work under **`src/core/`** went wrong—**user** correction, **pre-commit/CI** failure, or **repeated** same-class mistakes—you **must** follow the repo root **[`AGENTS.md`](../../AGENTS.md)** **Mandatory learning loop**: distill a short rule and **update** the **nearest** relevant `AGENTS.md` (this file or a package file) or **source comments** in the same session (including **delta vs `main`** scope).