Quick Start

This guide walks through running a teleoperation session with an XR headset using CloudXR. By the end you will have the retargeting pipeline processing live hand/controller data and printing gripper commands to the terminal.

1. Check out code base (Optional)

Clone the repository and enter the project directory:

git clone https://github.com/NVIDIA/IsaacTeleop.git
cd IsaacTeleop

As a quick start guide, we don’t need to build the code base from source. However, we still need to clone the repository for a couple quick samples to run.

2. Install the isaacteleop pip package

In a new terminal, activate your preferred virtual or conda environment, then install the package from PyPI (or from a local wheel if you built from source):

# From PyPI
pip install 'isaacteleop[cloudxr,retargeters]~=1.0.0' --extra-index-url https://pypi.nvidia.com

Instead of installing the package from PyPI, you can build from source and install the local wheel. See Build from Source for more details.

ARM64 / aarch64 only (e.g. NVIDIA DGX Spark)

PyPI does not publish pre-built nlopt wheels for ARM64, so the retargeters extra cannot be installed directly from PyPI (see issue #452). Follow the aarch64 nlopt wheel build steps from the build-from-source guide first, then install isaacteleop with an additional --find-links:

pip install 'isaacteleop[cloudxr,retargeters]~=1.0.0' \
    --extra-index-url https://pypi.nvidia.com \
    --find-links=/tmp/nlopt-wheels/

3. Run CloudXR Server

Start the CloudXR runtime. The first run downloads the CloudXR Web Client SDK and asks you to review and accept the EULA:

python -m isaacteleop.cloudxr

To bypass the interactive EULA prompt (e.g. for CI or headless runs), pass the flag:

python -m isaacteleop.cloudxr --accept-eula

Optional launch modes

The launcher supports three optional flags that can be combined to control how the headset connects and how the web client is delivered.

Command	What it does
python -m isaacteleop.cloudxr	Plain: headset navigates to GitHub Pages URL over WiFi.
python -m isaacteleop.cloudxr --host-client	Serves the web client at https://<ip>:48322/client/ via the WSS proxy. No separate port, no USB or TURN relay required. Useful when GitHub Pages is unreachable.
python -m isaacteleop.cloudxr --setup-oob	OOB hub + CDP automation: opens the browser on the headset and auto-clicks CONNECT over USB adb. Client URL is GitHub Pages.
python -m isaacteleop.cloudxr --setup-oob --host-client	OOB hub + CDP with client at /client/ on the WSS proxy (air-gapped / proxy use).
python -m isaacteleop.cloudxr --setup-oob --usb-local	All traffic over USB: adb-reverse + coturn TURN relay + loopback HTTPS. Requires coturn and a WiFi-associated headset.

--usb-local requires --setup-oob. See Out-of-Band Teleop Control for full OOB documentation.

You should see output similar to:

Figure: CloudXR run output

Important

Keep this terminal open — CloudXR must stay running for the duration of the session. Open a new terminal for the remaining steps.

Also take note of the source /home/dev/.cloudxr/run/cloudxr.env path it mentioned in the output. You will need to source it in step 6. Load CloudXR environment variables.

CloudXR configurations (optional)

The CloudXR runtime uses the auto-webrtc device profile by default (Pico & Quest). For Apple Vision Pro it defaults to auto-native.

To inspect the active settings after startup:

cat ~/.cloudxr/run/cloudxr.env

To override settings, create an env file and pass it at startup:

echo 'NV_DEVICE_PROFILE=auto-native' > custom.env
python -m isaacteleop.cloudxr --cloudxr-env-config=./custom.env

Environment variables

Variable	Default	Description	Values
NV_DEVICE_PROFILE	auto-webrtc	Device profile	auto-webrtc, auto-native, Quest3, AppleVisionPro
NV_CXR_ENABLE_PUSH_DEVICES	true	Push device overseer for hand tracking	true, false
NV_CXR_FILE_LOGGING	true	File-based logging (disable to print to console)	true, false

4. Whitelist ports for Firewall

CloudXR requires certain network ports to be open. Depending on your firewall configuration, you might need to whitelist them manually.

Meta Quest and Pico headsets

For Quest and Pico headsets (web client), at the minimum, you need to whitelist the ports for the CloudXR runtime and wss proxy:

sudo ufw allow 47998/udp
sudo ufw allow 49100,48322/tcp

If you are running the web client from source (dev server), open both ports:

sudo ufw allow 8080,8443/tcp

Vision Pro client

For Vision Pro client, you need to whitelist the ports for the CloudXR runtime and wss proxy:

sudo ufw allow 48010,48322/tcp
sudo ufw allow 47998:48000,48005,48008,48012/udp

Please see the CloudXR network setup for more details for other network configurations (such as running the CloudXR runtime and wss proxy in containerized environment; or using Vision Pro client).

5. Connect an XR headset

Meta Quest, PICO headset, or desktop browser

No physical headset required for a quick test: open https://nvidia.github.io/IsaacTeleop/client in a desktop browser — IWER (Immersive Web Emulator Runtime) loads automatically and emulates a Meta Quest 3 headset.

For a real headset, open the same URL in your Meta Quest or PICO browser.

Important

If using a physical headset, make sure it is updated to the latest firmware before connecting. Older firmware may ship an outdated WebXR runtime that fails to connect or streams with reduced functionality.

Note

If GitHub Pages is unreachable (corporate network, air-gapped machine), start the server with --host-client in step 3. Run CloudXR Server and open https://<your-ip>:48322/client/ instead of the GitHub Pages URL. Port 48322 is already whitelisted in step 4. Whitelist ports for Firewall.

CloudXR web client

Figure: CloudXR web client usage instruction

Privacy warning

Figure: Browser privacy warning for self-signed certificate

Certificate accepted

Figure: Certificate accepted page

As illustrated in the figure above, there are 3 steps to connect to your headset:

1. Enter the IP address of the workstation running CloudXR

2. Accept the self-signed SSL certificate, which was created automatically during 3. Run CloudXR Server:

   • Click the Click https://<ip>:48322/ to accept cert link that appears on the page.

   • In the new tab, you will see a “Your connection is not private” warning. Click Advanced, then Proceed to <ip> (unsafe).

   • Once accepted, the page will show Certificate Accepted. Navigate back to the CloudXR.js client page.

3. Click Connect to begin teleoperation.

Note

For advanced usage and troubleshooting of CloudXR, see the CloudXR documentation for more details.

Offline / air-gapped use

On first run, the launcher fetches index.html and bundle.js from GitHub Pages and caches them in ~/.cloudxr/static-client/ (override with TELEOP_WEB_CLIENT_STATIC_DIR). Subsequent runs are fully offline.

For a true air-gapped machine, pre-stage the two files before the first run — copy them from https://nvidia.github.io/IsaacTeleop/client/ on a networked host, then transfer the ~/.cloudxr/static-client/ directory to the air-gapped machine.

The source code for the web client is in the deps/cloudxr/webxr_client/ directory. To build the web client from source, see Build WebXR Client.

Apple Vision Pro

For Apple Vision Pro, you will need to build and install the Isaac XR Teleop Sample Client. Follow the instructions in the Isaac XR Teleop Sample Client for Apple Vision Pro repository to build and install the sample client on your Apple Vision Pro.

Note

You will need v3.0.0 or newer of the Isaac XR Teleop Sample Client for Apple Vision Pro to connect to Isaac Teleop.

6. Load CloudXR environment variables

Open a new terminal and source the CloudXR environment variables posted from the CloudXR runtime in 3. Run CloudXR Server:

Source the setup script so that the OpenXR runtime points to CloudXR:

source ~/.cloudxr/run/cloudxr.env

Important

Make sure to run the rest of the commands in the same terminal. Or if have to open a new terminal, source the CloudXR environment variables again.

7. Run a teleop example

Run the simplified gripper retargeting example. This demonstrates the full pipeline: reading XR controller input via CloudXR, retargeting it through the GripperRetargeter, and printing the resulting gripper command values:

python examples/teleop/python/gripper_retargeting_example_simple.py

Once running, squeeze the controller triggers on your XR headset to control the gripper. You should see periodic status output:

============================================================
Gripper Retargeting - Squeeze triggers to control grippers
============================================================

[  0.5s] Right: 0.00
[  1.0s] Right: 0.73
[  1.5s] Right: 1.00
...

The example runs for 20 seconds and then exits. To try other examples, see examples/teleop/python/ — for instance:

• se3_retargeting_example.py — maps hand or controller poses to end-effector poses (absolute or relative)

• dex_bimanual_example.py — bimanual dexterous hand retargeting

• gripper_retargeting_example.py — full gripper example with more configuration options

Next steps

Teleoperation in Isaac Lab

Follow instructions in Teleoperation and Imitation Learning with Isaac Lab Mimic to know more about how to collect demonstrations with Isaac Lab and how to augment them with Isaac Lab Mimic and train imitation learning policies.

If you are new to Isaac Lab, follow instructions in Isaac Lab Quick Start to get started.

Teleoperation with Isaac ROS

Check out the examples/teleop_ros2/ directory for an example on how to make a ROS 2 message publisher using Isaac Teleop.

We are also working on a Unitree G1-based end-to-end teleoperation, data collection, and imitation learning solution for ROS2 in an upcoming Isaac ROS release. Stay tuned!

ROS is a trademark of Open Robotics.

More Information

• Teleop Session — learn how TeleopSession works and how to build custom retargeting pipelines

• Televiz — visualize robot camera and sensor feeds in an XR headset with the Televiz compositor (and share a single XR session with your teleop pipeline)

• Build from Source — build the C++ core, Python bindings, and plugins from source