mirror of
https://github.com/huggingface/lerobot.git
synced 2026-05-15 08:39:49 +00:00
Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 92f08933ec | |||
| 09c93c4aa1 | |||
| 19c6adef85 | |||
| 96b7f3dae0 | |||
| 885ef91892 | |||
| b0efa73520 |
@@ -165,7 +165,7 @@ hf auth login --token ${HUGGINGFACE_TOKEN} --add-to-git-credential
|
|||||||
Then store your Hugging Face repository name in a variable:
|
Then store your Hugging Face repository name in a variable:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
HF_USER=$(hf auth whoami | awk -F': *' 'NR==1 {print $2}')
|
HF_USER=$(NO_COLOR=1 hf auth whoami | awk -F': *' 'NR==1 {print $2}')
|
||||||
echo $HF_USER
|
echo $HF_USER
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
+84
-43
@@ -12,36 +12,59 @@ The Unitree G1 humanoid is now supported in LeRobot! You can teleoperate, train
|
|||||||
|
|
||||||
## Part 1: Getting Started
|
## Part 1: Getting Started
|
||||||
|
|
||||||
### Install LeRobot on Your Machine
|
### Install the Unitree SDK
|
||||||
|
|
||||||
|
Follow the [unitree_sdk2_python installation guide](https://github.com/unitreerobotics/unitree_sdk2_python#installation). Tested with `unitree_sdk2py==1.0.1` and `cyclonedds==0.10.2`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
conda create -y -n lerobot python=3.12
|
conda create -y -n lerobot python=3.12
|
||||||
conda activate lerobot
|
conda activate lerobot
|
||||||
git clone https://github.com/unitreerobotics/unitree_sdk2_python.git
|
git clone https://github.com/unitreerobotics/unitree_sdk2_python.git
|
||||||
cd unitree_sdk2_python && pip install -e .
|
cd unitree_sdk2_python
|
||||||
|
pip install -e .
|
||||||
|
cd ..
|
||||||
|
```
|
||||||
|
|
||||||
|
### Install LeRobot
|
||||||
|
|
||||||
|
```bash
|
||||||
|
conda install ffmpeg -c conda-forge
|
||||||
|
conda install -c conda-forge "pinocchio>=3.0.0,<4.0.0"
|
||||||
git clone https://github.com/huggingface/lerobot.git
|
git clone https://github.com/huggingface/lerobot.git
|
||||||
cd lerobot
|
cd lerobot
|
||||||
pip install -e '.[unitree_g1]'
|
pip install -e '.[unitree_g1]'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
<Tip>
|
||||||
|
For now, pinocchio must be installed from conda-forge (not pip) to include the
|
||||||
|
CasADi bindings needed for arm IK.
|
||||||
|
</Tip>
|
||||||
|
|
||||||
### Test the Installation (Simulation)
|
### Test the Installation (Simulation)
|
||||||
|
|
||||||
|
The simulation environment has its own dependencies. Check the Simulation environment dependencies: [Unitree G1 Mujoco EnvHub](https://huggingface.co/lerobot/unitree-g1-mujoco/tree/main).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install mujoco loguru msgpack msgpack-numpy
|
||||||
|
```
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
lerobot-teleoperate \
|
lerobot-teleoperate \
|
||||||
--robot.type=unitree_g1 \
|
--robot.type=unitree_g1 \
|
||||||
--robot.is_simulation=true \
|
--robot.is_simulation=true \
|
||||||
--teleop.type=unitree_g1 \
|
--teleop.type=unitree_g1 \
|
||||||
--teleop.id=wbc_unitree \
|
--teleop.id=wbc_unitree \
|
||||||
--robot.cameras='{"global_view": {"type": "zmq", "server_address": "localhost", "port": 5555, "camera_name": "head_camera", "width": 640, "height": 480, "fps": 30}}' \
|
--robot.cameras='{"global_view": {"type": "zmq", "server_address": "localhost", "port": 5555, "camera_name": "head_camera", "width": 640, "height": 480, "fps": 30, "warmup_s": 5}}' \
|
||||||
--display_data=true
|
--display_data=true \
|
||||||
|
--robot.controller=GrootLocomotionController
|
||||||
```
|
```
|
||||||
|
|
||||||
This will launch a [MuJoCo sim instance](https://huggingface.co/lerobot/unitree-g1-mujoco/tree/main) for the G1.
|
This will launch a [MuJoCo sim instance](https://huggingface.co/lerobot/unitree-g1-mujoco/tree/main) for the G1. You can connect a gamepad to your machine before launching in order to control the robot's locomotion in sim. We support both [HolosomaLocomotionController](https://github.com/amazon-far/holosoma) and [GrootLocomotionController](https://github.com/NVlabs/GR00T-WholeBodyControl) via `--robot.controller`.
|
||||||
|
|
||||||
- Press `9` to release the robot
|
- Press `9` to release the robot
|
||||||
- Press `7` / `8` to increase / decrease waist height
|
- Press `7` / `8` to increase / decrease waist height
|
||||||
|
|
||||||
### Connect to the Robot
|
### Connect to the Physical Robot
|
||||||
|
|
||||||
The G1's Ethernet IP is fixed at `192.168.123.164`. Your machine must have a static IP on the same subnet: `192.168.123.x` where `x ≠ 164`.
|
The G1's Ethernet IP is fixed at `192.168.123.164`. Your machine must have a static IP on the same subnet: `192.168.123.x` where `x ≠ 164`.
|
||||||
|
|
||||||
@@ -59,37 +82,11 @@ ssh unitree@192.168.123.164
|
|||||||
# Password: 123
|
# Password: 123
|
||||||
```
|
```
|
||||||
|
|
||||||
### Install LeRobot on the G1
|
### Share Internet via Ethernet
|
||||||
|
|
||||||
From the robot:
|
The G1 needs internet access to clone repos and install packages. Share your laptop's connection over Ethernet:
|
||||||
|
|
||||||
```bash
|
**On your laptop:**
|
||||||
conda create -y -n lerobot python=3.12
|
|
||||||
conda activate lerobot
|
|
||||||
git clone https://github.com/unitreerobotics/unitree_sdk2_python.git
|
|
||||||
cd unitree_sdk2_python && pip install -e .
|
|
||||||
git clone https://github.com/huggingface/lerobot.git
|
|
||||||
cd lerobot
|
|
||||||
pip install -e '.[unitree_g1]'
|
|
||||||
```
|
|
||||||
|
|
||||||
> **Note:** The Unitree SDK requires CycloneDDS v0.10.2. See the [Unitree SDK docs](https://github.com/unitreerobotics/unitree_sdk2_python) for details.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Part 2: Enable WiFi on the Robot
|
|
||||||
|
|
||||||
Wi-Fi connectivity is blocked by default on the G1. To activate:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo rfkill unblock all
|
|
||||||
sudo ip link set wlan0 up
|
|
||||||
sudo nmcli radio wifi on
|
|
||||||
sudo nmcli device set wlan0 managed yes
|
|
||||||
sudo systemctl restart NetworkManager
|
|
||||||
```
|
|
||||||
|
|
||||||
**On your laptop** (share internet via Ethernet):
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo sysctl -w net.ipv4.ip_forward=1
|
sudo sysctl -w net.ipv4.ip_forward=1
|
||||||
@@ -100,7 +97,7 @@ sudo iptables -A FORWARD -i wlp132s0f0 -o enp131s0 -m state --state RELATED,ESTA
|
|||||||
sudo iptables -A FORWARD -i enp131s0 -o wlp132s0f0 -j ACCEPT
|
sudo iptables -A FORWARD -i enp131s0 -o wlp132s0f0 -j ACCEPT
|
||||||
```
|
```
|
||||||
|
|
||||||
**On the G1** (set default route through your laptop):
|
**On the G1:**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo ip route del default 2>/dev/null || true
|
sudo ip route del default 2>/dev/null || true
|
||||||
@@ -111,6 +108,45 @@ echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf
|
|||||||
ping -c 3 8.8.8.8
|
ping -c 3 8.8.8.8
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Install the Unitree SDK on the G1
|
||||||
|
|
||||||
|
Follow the [unitree_sdk2_python installation guide](https://github.com/unitreerobotics/unitree_sdk2_python#installation):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
conda create -y -n lerobot python=3.12
|
||||||
|
conda activate lerobot
|
||||||
|
git clone https://github.com/unitreerobotics/unitree_sdk2_python.git
|
||||||
|
cd unitree_sdk2_python
|
||||||
|
python -m pip install -e .
|
||||||
|
cd ..
|
||||||
|
```
|
||||||
|
|
||||||
|
### Install LeRobot on the G1
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/huggingface/lerobot.git
|
||||||
|
cd lerobot
|
||||||
|
conda install -c conda-forge "pinocchio>=3.0.0,<4.0.0"
|
||||||
|
python -m pip install -e '.[unitree_g1]'
|
||||||
|
```
|
||||||
|
|
||||||
|
<Tip>
|
||||||
|
For now, pinocchio must be installed from conda-forge (not pip) to include the
|
||||||
|
CasADi bindings needed for arm IK.
|
||||||
|
</Tip>
|
||||||
|
|
||||||
|
### (Optional) Enable WiFi on the Robot
|
||||||
|
|
||||||
|
For wireless SSH access, you can enable WiFi on the G1 (it's blocked by default):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo rfkill unblock all
|
||||||
|
sudo ip link set wlan0 up
|
||||||
|
sudo nmcli radio wifi on
|
||||||
|
sudo nmcli device set wlan0 managed yes
|
||||||
|
sudo systemctl restart NetworkManager
|
||||||
|
```
|
||||||
|
|
||||||
**Connect to a WiFi network:**
|
**Connect to a WiFi network:**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -125,7 +161,7 @@ sudo nmcli connection up "YourNetwork"
|
|||||||
ip a show wlan0
|
ip a show wlan0
|
||||||
```
|
```
|
||||||
|
|
||||||
You can now SSH over WiFi:
|
You can then SSH over WiFi instead of Ethernet:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ssh unitree@<ROBOT_WIFI_IP>
|
ssh unitree@<ROBOT_WIFI_IP>
|
||||||
@@ -134,18 +170,23 @@ ssh unitree@<ROBOT_WIFI_IP>
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Part 3: Teleoperation & Locomotion
|
## Part 2: Teleoperation & Locomotion
|
||||||
|
|
||||||
### Run the Robot Server
|
### Run the Robot Server
|
||||||
|
|
||||||
On the robot:
|
On the robot (from `~/lerobot`):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
cd ~/lerobot
|
||||||
python src/lerobot/robots/unitree_g1/run_g1_server.py --camera
|
python src/lerobot/robots/unitree_g1/run_g1_server.py --camera
|
||||||
```
|
```
|
||||||
|
|
||||||
### Run the Locomotion Policy
|
### Run the Locomotion Policy
|
||||||
|
|
||||||
|
You can run the teleoperation client from your laptop over Ethernet, over WiFi (experimental), or directly on the robot itself. Mind potential latency introduced by your network.
|
||||||
|
|
||||||
|
**From your laptop:**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
lerobot-teleoperate \
|
lerobot-teleoperate \
|
||||||
--robot.type=unitree_g1 \
|
--robot.type=unitree_g1 \
|
||||||
@@ -158,13 +199,13 @@ lerobot-teleoperate \
|
|||||||
--robot.controller=HolosomaLocomotionController
|
--robot.controller=HolosomaLocomotionController
|
||||||
```
|
```
|
||||||
|
|
||||||
We support both [HolosomaLocomotionController](https://github.com/amazon-far/holosoma) and [GrootLocomotionController](https://github.com/NVlabs/GR00T-WholeBodyControl).
|
We support both [GrootLocomotionController](https://github.com/NVlabs/GR00T-WholeBodyControl) and [HolosomaLocomotionController](https://github.com/amazon-far/holosoma) via `--robot.controller`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Part 4: Loco-Manipulation with the Homunculus Exoskeleton
|
## Part 3: Loco-Manipulation with the Homunculus Exoskeleton
|
||||||
|
|
||||||
We provide a loco-manipulation solution via the Homunculus Exoskeleton — an open-source 7 DoF exoskeleton for whole-body control. Assembly instructions [here](https://github.com/nepyope/hmc_exo).
|
We provide a loco-manipulation solution via the Homunculus Exoskeleton — an open-source 7 DoF exoskeleton for whole-body control. Check it out [here](https://github.com/nepyope/hmc_exo).
|
||||||
|
|
||||||
### Calibrate
|
### Calibrate
|
||||||
|
|
||||||
@@ -205,7 +246,7 @@ Example dataset: [nepyope/unitree_box_move_blue_full](https://huggingface.co/dat
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Part 5: Training & Inference
|
## Part 4: Training & Inference
|
||||||
|
|
||||||
### Train
|
### Train
|
||||||
|
|
||||||
|
|||||||
+4
-5
@@ -25,7 +25,7 @@ discord = "https://discord.gg/s3KuuzsPFb"
|
|||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "lerobot"
|
name = "lerobot"
|
||||||
version = "0.5.0"
|
version = "0.5.1"
|
||||||
description = "🤗 LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch"
|
description = "🤗 LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch"
|
||||||
dynamic = ["readme"]
|
dynamic = ["readme"]
|
||||||
license = { text = "Apache-2.0" }
|
license = { text = "Apache-2.0" }
|
||||||
@@ -76,7 +76,7 @@ dependencies = [
|
|||||||
"torchvision>=0.21.0,<0.26.0",
|
"torchvision>=0.21.0,<0.26.0",
|
||||||
|
|
||||||
"einops>=0.8.0,<0.9.0",
|
"einops>=0.8.0,<0.9.0",
|
||||||
"opencv-python-headless>=4.9.0,<4.13.0",
|
"opencv-python-headless>=4.9.0,<4.14.0",
|
||||||
"av>=15.0.0,<16.0.0",
|
"av>=15.0.0,<16.0.0",
|
||||||
"jsonlines>=4.0.0,<5.0.0",
|
"jsonlines>=4.0.0,<5.0.0",
|
||||||
"pynput>=1.7.8,<1.9.0",
|
"pynput>=1.7.8,<1.9.0",
|
||||||
@@ -119,14 +119,13 @@ gamepad = ["lerobot[pygame-dep]", "hidapi>=0.14.0,<0.15.0"]
|
|||||||
hopejr = ["lerobot[feetech]", "lerobot[pygame-dep]"]
|
hopejr = ["lerobot[feetech]", "lerobot[pygame-dep]"]
|
||||||
lekiwi = ["lerobot[feetech]", "pyzmq>=26.2.1,<28.0.0"]
|
lekiwi = ["lerobot[feetech]", "pyzmq>=26.2.1,<28.0.0"]
|
||||||
unitree_g1 = [
|
unitree_g1 = [
|
||||||
"unitree-sdk2==1.0.1",
|
# "unitree-sdk2==1.0.1",
|
||||||
"pyzmq>=26.2.1,<28.0.0",
|
"pyzmq>=26.2.1,<28.0.0",
|
||||||
"onnxruntime>=1.16.0,<2.0.0",
|
"onnxruntime>=1.16.0,<2.0.0",
|
||||||
"pin>=3.0.0,<4.0.0",
|
"onnx>=1.16.0,<2.0.0",
|
||||||
"meshcat>=0.3.0,<0.4.0",
|
"meshcat>=0.3.0,<0.4.0",
|
||||||
"lerobot[matplotlib-dep]",
|
"lerobot[matplotlib-dep]",
|
||||||
"lerobot[pygame-dep]",
|
"lerobot[pygame-dep]",
|
||||||
"casadi>=3.6.0,<4.0.0",
|
|
||||||
]
|
]
|
||||||
reachy2 = ["reachy2_sdk>=1.0.15,<1.1.0"]
|
reachy2 = ["reachy2_sdk>=1.0.15,<1.1.0"]
|
||||||
kinematics = ["lerobot[placo-dep]"]
|
kinematics = ["lerobot[placo-dep]"]
|
||||||
|
|||||||
@@ -578,7 +578,7 @@ def get_hf_features_from_features(features: dict) -> datasets.Features:
|
|||||||
continue
|
continue
|
||||||
elif ft["dtype"] == "image":
|
elif ft["dtype"] == "image":
|
||||||
hf_features[key] = datasets.Image()
|
hf_features[key] = datasets.Image()
|
||||||
elif ft["shape"] == (1,):
|
elif ft["shape"] == (1,) and ft["names"] is None:
|
||||||
hf_features[key] = datasets.Value(dtype=ft["dtype"])
|
hf_features[key] = datasets.Value(dtype=ft["dtype"])
|
||||||
elif len(ft["shape"]) == 1:
|
elif len(ft["shape"]) == 1:
|
||||||
hf_features[key] = datasets.Sequence(
|
hf_features[key] = datasets.Sequence(
|
||||||
|
|||||||
@@ -57,6 +57,7 @@ import pyarrow as pa
|
|||||||
import tqdm
|
import tqdm
|
||||||
from datasets import Dataset, Features, Image
|
from datasets import Dataset, Features, Image
|
||||||
from huggingface_hub import HfApi, snapshot_download
|
from huggingface_hub import HfApi, snapshot_download
|
||||||
|
from huggingface_hub.errors import RevisionNotFoundError
|
||||||
from requests import HTTPError
|
from requests import HTTPError
|
||||||
|
|
||||||
from lerobot.datasets.compute_stats import aggregate_stats
|
from lerobot.datasets.compute_stats import aggregate_stats
|
||||||
@@ -511,7 +512,7 @@ def convert_dataset(
|
|||||||
hub_api = HfApi()
|
hub_api = HfApi()
|
||||||
try:
|
try:
|
||||||
hub_api.delete_tag(repo_id, tag=CODEBASE_VERSION, repo_type="dataset")
|
hub_api.delete_tag(repo_id, tag=CODEBASE_VERSION, repo_type="dataset")
|
||||||
except HTTPError as e:
|
except(HTTPError,RevisionNotFoundError) as e:
|
||||||
print(f"tag={CODEBASE_VERSION} probably doesn't exist. Skipping exception ({e})")
|
print(f"tag={CODEBASE_VERSION} probably doesn't exist. Skipping exception ({e})")
|
||||||
pass
|
pass
|
||||||
hub_api.delete_files(
|
hub_api.delete_files(
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ _peft_available = is_package_available("peft")
|
|||||||
_scipy_available = is_package_available("scipy")
|
_scipy_available = is_package_available("scipy")
|
||||||
_reachy2_sdk_available = is_package_available("reachy2_sdk")
|
_reachy2_sdk_available = is_package_available("reachy2_sdk")
|
||||||
_can_available = is_package_available("python-can", "can")
|
_can_available = is_package_available("python-can", "can")
|
||||||
_unitree_sdk_available = is_package_available("unitree-sdk2", "unitree_sdk2py")
|
_unitree_sdk_available = is_package_available("unitree-sdk2py", "unitree_sdk2py")
|
||||||
_pygame_available = is_package_available("pygame")
|
_pygame_available = is_package_available("pygame")
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user