Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude/skills/cmake-structure/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
name: cmake-structure
description: >-
IsaacTeleop CMake target layout and #include conventions. Use whenever you
Isaac Capture CMake target layout and #include conventions. Use whenever you
create or edit a CMakeLists.txt, add or move a module/library/executable/test
target, set up target_include_directories or target_link_libraries, decide
where a header file goes (private vs public inc/), write or reorder #include
Expand All @@ -15,7 +15,7 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES.
SPDX-License-Identifier: Apache-2.0
-->

# IsaacTeleop CMake & include structure
# Isaac Capture CMake & include structure

The rules, recommended structure, repo conventions, and review checklist live in
the repo's single source of truth:
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ A clear and concise description of what you expected to happen.
If applicable, add screenshots to help explain your problem.

**Platforms**
- Isaac Teleop Version: [e.g. 1.0.23]
- Isaac Capture Version: [e.g. 1.0.23]
- OS: [e.g. Ubuntu X86, ARM64]
- Integrations: [e.g. Isaac Lab, Isaac Sim, ROS2]
- XR Device: [e.g. Quest 3, Pico 4 Ultra]
Expand Down
15 changes: 10 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,24 +22,29 @@ SPDX-License-Identifier: Apache-2.0
Repository owners: keep this block at the top of this file; do not delete it.
-->

# IsaacTeleop — agent notes
# Isaac Capture — agent notes

## Product naming

Isaac Capture names the overall library and repository. Isaac Teleop remains its
teleoperation component; preserve that brand and existing technical identifiers.

## CRITICAL — mandatory preflight (do not skip)

**Hard requirement:** Do **not** edit or add code under `IsaacTeleop/` until you have **read** (e.g. with the Read tool) **every** relevant `AGENTS.md` file as defined below. This is **not** optional, **not** "only for large tasks," and **not** satisfiable by inferring from prior conversation. If you have not read those files in **this** session for **this** task, stop and read them first.
**Hard requirement:** Do **not** edit or add code under `IsaacCapture/` until you have **read** (e.g. with the Read tool) **every** relevant `AGENTS.md` file as defined below. This is **not** optional, **not** "only for large tasks," and **not** satisfiable by inferring from prior conversation. If you have not read those files in **this** session for **this** task, stop and read them first.

**You must:**

1. List the **directories** you expect to edit or create files under (e.g. `src/core/live_trackers/cpp/`, `src/core/deviceio_base/cpp/inc/…`).
2. For **each** such directory, **read** **`AGENTS.md` in that directory** if it exists.
3. Walk **up** toward the **IsaacTeleop repo root** and **read** **`AGENTS.md` in every ancestor directory** that has one (e.g. `live_trackers/` → `src/core/` → **`IsaacTeleop/AGENTS.md` (this file)**).
3. Walk **up** toward the **IsaacCapture repo root** and **read** **`AGENTS.md` in every ancestor directory** that has one (e.g. `live_trackers/` → `src/core/` → **`IsaacCapture/AGENTS.md` (this file)**).

**You must not** assume that reading the single "closest" `AGENTS.md` is enough. **Multiple** `AGENTS.md` files can apply to one change (e.g. both `live_trackers` and `deviceio_base`).

**Listing every `AGENTS.md` in this repo** (no curated table here—add new files under the tree without editing this document):

```bash
# Run from the IsaacTeleop repository root (the directory that contains this file):
# Run from the IsaacCapture repository root (the directory that contains this file):
find . -name AGENTS.md ! -path '*/.git/*' | sort
```

Expand Down Expand Up @@ -215,7 +220,7 @@ pre-commit install --hook-type commit-msg

## Pre-commit — match CI before you stop

- From the **IsaacTeleop repo root** (this directory), run pre-commit and **fix all failures** before you treat a change as finished (do not only rely on “should pass” reasoning).
- From the **IsaacCapture repo root** (this directory), run pre-commit and **fix all failures** before you treat a change as finished (do not only rely on “should pass” reasoning).
- **Use the same hook set as GitHub Actions:** `.github/workflows/pre-commit.yaml` runs pre-commit with **`SKIP=check-copyright-year`**. Mirror that locally:

```bash
Expand Down
2 changes: 1 addition & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Overview

Define the code of conduct followed and enforced for Isaac Teleop
Define the code of conduct followed and enforced for Isaac Capture

### Intended audience

Expand Down
32 changes: 22 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,11 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES.
SPDX-License-Identifier: Apache-2.0
-->

![Isaac Teleop](docs/source/_static/isaac-teleop-hero.jpg)

# Isaac Teleop
# NVIDIA Isaac Capture

<div align="center">

**The unified framework for high-fidelity ego-centric and robotics data collection.**
**Shared capture infrastructure for teleoperation and egocentric data.**

[![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/)
[![Isaac Lab](https://img.shields.io/badge/Isaac%20Lab-3.0.0-orange.svg)](https://isaac-sim.github.io/IsaacLab/develop)
Expand All @@ -21,32 +19,46 @@ SPDX-License-Identifier: Apache-2.0

---

<!-- Keep one overview image, below the title and badges and immediately before Overview. -->
![Isaac Capture architecture: Isaac Teleop and Ego Pipeline share device plugins, retargeters, and data utilities, with integrations into simulation and real-robot environments](docs/source/_static/isaac-capture-architecture.png)

## Overview

**Isaac Teleop**: The unified standard for high-fidelity egocentric and robot data collection.
It is designed to address the data bottleneck in robot learning by streamlining device integration;
standardizing high-fidelity human demo data collection; and foster device & data interoperability.
**Isaac Capture** provides shared infrastructure for collecting human and robot demonstration data.
It brings together two components:

- **Isaac Teleop** supports robot-in-the-loop capture through teleoperation, including Teleop Session and Remote Assist.
- **Ego Pipeline** supports human and egocentric data capture through Ego Capture and Ego Post Processing.

Both components use a shared core of device plugins, retargeters, and data utilities for recording,
replaying, and converting data. Isaac Teleop remains a component of Isaac Capture, with its existing
functionality preserved.

## Key Features

- Unified stack for sim & real teleoperation
- Standardized device interface
- Flexible retargeting framework

### Teleoperation Use Cases
### Isaac Teleop

- Currently supported use cases
- Use XR headsets for gripper / tri-finger hand manipulation
- Use XR headsets with gloves for dex-hand manipulation
- Seated full body loco-manipulation (Homie)
- Tracking based full body loco-manipulation with GR00T-WholeBodyControl (SONIC)
- Egocentric data collection (aka “no-robot”)
- Upcoming use cases
- Teleoperate using only non-XR devices (e.g. gamepad, Gello, haply, etc.)
- Teleoperate cloud based robotics simulations
- Remote teleoperation with camera streaming to the desktop
- Remote teleoperation with immersive camera streaming to XR headsets

### Ego Pipeline

- Capture egocentric data without a robot in the loop.
- Reconstruct hand and camera poses from egocentric video with the
[Ego Post Processing pipeline](https://nvidia.github.io/IsaacCapture/main/references/egocentric_hand_reconstruction.html).

## Quick Start

### Documentation
Expand All @@ -60,6 +72,6 @@ Our [documentation page](https://nvidia.github.io/IsaacCapture) provides everyth

### Install & Run Isaac Lab

Isaac Teleop is design to work side by side with [NVIDIA Isaac Lab](https://github.com/isaac-sim/IsaacLab) starting with Isaac Lab [3.0 Beta release](https://github.com/isaac-sim/IsaacLab/releases/tag/v3.0.0-beta).
The latest [NVIDIA Isaac Lab](https://github.com/isaac-sim/IsaacLab) release, [3.0.0 EA](https://github.com/isaac-sim/IsaacLab/releases/tag/v3.0.0-EA), includes Isaac Capture, previously released as Isaac Teleop.

To get started, please refer to Isaac Lab's [Installation](https://isaac-sim.github.io/IsaacLab/develop/source/setup/installation/index.html) guide for more details. Then follow the [CloudXR teleoperation in Isaac Lab](https://isaac-sim.github.io/IsaacLab/develop/source/how-to/cloudxr_teleoperation.html) to get started with Teleoperation in Sim.
2 changes: 1 addition & 1 deletion cmake/cmake-structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES.
SPDX-License-Identifier: Apache-2.0
-->

# IsaacTeleop CMake & include structure
# Isaac Capture CMake & include structure

Single source of truth for how CMake targets, directories, and `#include` paths
are organized in this repo. These rules apply to **all** coding agents and AI
Expand Down
4 changes: 2 additions & 2 deletions deps/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES.
SPDX-License-Identifier: Apache-2.0
-->

# Isaac Teleop Dependencies
# Isaac Capture Dependencies

This directory contains all project dependencies, organized by type.

Expand Down Expand Up @@ -91,7 +91,7 @@ machine and are located via `find_package`.
### pybind11
- **Source**: https://github.com/pybind/pybind11.git
- **Version**: a2e59f0e7065404b44dfe92a28aca47ba1378dc4 (v2.11.0-182)
- **Purpose**: C++/Python bindings for Isaac Teleop Python API
- **Purpose**: C++/Python bindings for Isaac Capture Python API
- **Build**: Header-only library
- **License**: BSD-style

Expand Down
2 changes: 1 addition & 1 deletion docs/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ for non-HTML builders.

## The external marker follows the target, not the record

The docs site and the IsaacTeleop repository are both this project, so a link into either
The docs site and the IsaacCapture repository are both this project, so a link into either
renders with no arrow and no screen-reader "(external)" — tracking links included. Only
third-party domains get the marker, and other GitHub organizations (`isaac-sim`) count as
third-party. `ecosystem_grid.PROJECT_REPO` is the one place that boundary is written down.
Expand Down
3 changes: 3 additions & 0 deletions docs/source/_static/isaac-capture-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions docs/source/_static/isaac-capture-architecture.png.license
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
2 changes: 1 addition & 1 deletion examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All
SPDX-License-Identifier: Apache-2.0
-->

# Isaac Teleop examples
# Isaac Capture examples

Each subdirectory is a self-contained example. Read one, run it, or copy it into
your own project — the last of those is the reason for the layout below.
Expand Down
4 changes: 2 additions & 2 deletions src/core/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES.
SPDX-License-Identifier: Apache-2.0
-->

# Agent notes — IsaacTeleop `src/core` (index)
# Agent notes — Isaac Capture `src/core` (index)

**CRITICAL:** The mandatory multi-file `AGENTS.md` preflight in **[`../../AGENTS.md`](../../AGENTS.md)** applies here too. Before changing anything under `src/core/`, you must have read **this file**, **the repo root `AGENTS.md`**, and **every other `AGENTS.md` on the directory paths you will touch**. Do not skip any of them.

To see **all** `AGENTS.md` files in the IsaacTeleop repo, use the **`find` command (or `**/AGENTS.md` glob) documented in the repo root [`AGENTS.md`](../../AGENTS.md)**—do not rely on a hand-maintained list in this file.
To see **all** `AGENTS.md` files in the Isaac Capture repo, use the **`find` command (or `**/AGENTS.md` glob) documented in the repo root [`AGENTS.md`](../../AGENTS.md)**—do not rely on a hand-maintained list in this file.

If work under **`src/core/`** went wrong—**user** correction, **pre-commit/CI** failure, or **repeated** same-class mistakes—you **must** follow the repo root **[`AGENTS.md`](../../AGENTS.md)** **Mandatory learning loop**: distill a short rule and **update** the **nearest** relevant `AGENTS.md` (this file or a package file) or **source comments** in the same session (including **delta vs `main`** scope).

Expand Down
Loading