admin/lerobot
1
0
Fork 0
mirror of https://github.com/huggingface/lerobot.git synced 2026-05-26 14:09:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: integrate PR #3315 review feedback

Browse Source 
- ci: add Docker Hub login step, add HF_USER_TOKEN guard on eval step
- docker: tie patches to pinned versions with removal guidance, remove
  unnecessary HF_TOKEN for public dataset, fix hadolint warnings
- docs: fix paper link to arxiv, add teaser image, fix camera names
  (4→3 cameras), fix observation dims (480x640→240x320)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Pepijn
2026-04-16 18:33:53 +02:00
parent ad9662b4a8
commit a4102ee86d
4 changed files with 51 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/benchmark_tests.yml
+7
View File
@@ -334,6 +334,12 @@ jobs:
with: with:
cache-binary: false cache-binary: false
- name: Login to Docker Hub
uses: docker/login-action@v3 # zizmor: ignore[unpinned-uses]
with:
username: ${{ secrets.DOCKERHUB_LEROBOT_USERNAME }}
password: ${{ secrets.DOCKERHUB_LEROBOT_PASSWORD }}
# Build the full-install image: SAPIEN, mplib, CuRobo, pytorch3d + # Build the full-install image: SAPIEN, mplib, CuRobo, pytorch3d +
# simulation assets (~4 GB). Layer cache lives in the runner's local # simulation assets (~4 GB). Layer cache lives in the runner's local
# Docker daemon — reused across re-runs on the same machine. # Docker daemon — reused across re-runs on the same machine.
@@ -349,6 +355,7 @@ jobs:
cache-to: type=local,dest=/tmp/.buildx-cache-robotwin,mode=max cache-to: type=local,dest=/tmp/.buildx-cache-robotwin,mode=max
- name: Run RoboTwin 2.0 smoke eval (1 episode) - name: Run RoboTwin 2.0 smoke eval (1 episode)
if: env.HF_USER_TOKEN != ''
run: | run: |
# Named container (no --rm) so we can docker cp artifacts out. # Named container (no --rm) so we can docker cp artifacts out.
docker run --name robotwin-eval --gpus all \ docker run --name robotwin-eval --gpus all \
docker/Dockerfile.benchmark.robotwin
+11 -5
View File
@@ -60,8 +60,11 @@ RUN cd ${ROBOTWIN_ROOT}/envs \
uv pip install -e . --no-build-isolation --no-cache uv pip install -e . --no-build-isolation --no-cache
# Upstream patches (mirror RoboTwin's script/_install.sh). # Upstream patches (mirror RoboTwin's script/_install.sh).
# mplib: drop a broken `or collide` clause in planner.py. # These patches target the exact versions pinned above; re-check when upgrading.
# sapien: fix URDF loader encoding + .srdf extension check. # mplib==0.2.1: drop a broken `or collide` clause in planner.py.
# Safe to remove once mplib > 0.2.1 ships with the fix upstream.
# sapien==3.0.0b1: fix URDF loader encoding + .srdf extension check.
# Safe to remove once sapien > 3.0.0b1 ships with the fix upstream.
RUN python - <<'EOF' RUN python - <<'EOF'
import pathlib, re, site import pathlib, re, site
for d in site.getsitepackages(): for d in site.getsitepackages():
@@ -80,7 +83,7 @@ EOF
# Simulation assets from TianxingChen/RoboTwin2.0: embodiments (~220 MB) + # Simulation assets from TianxingChen/RoboTwin2.0: embodiments (~220 MB) +
# objects (~3.74 GB). background_texture (~11 GB) is intentionally skipped. # objects (~3.74 GB). background_texture (~11 GB) is intentionally skipped.
ARG HF_TOKEN="" # The dataset is public — no auth token needed.
RUN python - <<'EOF' RUN python - <<'EOF'
import os, pathlib, zipfile import os, pathlib, zipfile
from huggingface_hub import hf_hub_download from huggingface_hub import hf_hub_download
@@ -92,7 +95,6 @@ for fname in ("embodiments.zip", "objects.zip"):
repo_id="TianxingChen/RoboTwin2.0", repo_id="TianxingChen/RoboTwin2.0",
repo_type="dataset", repo_type="dataset",
filename=fname, filename=fname,
token=os.environ.get("HF_TOKEN") or None,
local_dir=str(assets_dir), local_dir=str(assets_dir),
) )
with zipfile.ZipFile(local, "r") as z: with zipfile.ZipFile(local, "r") as z:
@@ -100,10 +102,14 @@ for fname in ("embodiments.zip", "objects.zip"):
pathlib.Path(local).unlink() pathlib.Path(local).unlink()
EOF EOF
RUN cd ${ROBOTWIN_ROOT} && python script/update_embodiment_config_path.py WORKDIR ${ROBOTWIN_ROOT}
RUN python script/update_embodiment_config_path.py
ENV PYTHONPATH="${ROBOTWIN_ROOT}:${PYTHONPATH}" ENV PYTHONPATH="${ROBOTWIN_ROOT}:${PYTHONPATH}"
# Return to the lerobot source directory (set by base image) before overlaying.
WORKDIR /lerobot
# Overlay the PR's source code on top of the nightly image. # Overlay the PR's source code on top of the nightly image.
COPY --chown=user_lerobot:user_lerobot . . COPY --chown=user_lerobot:user_lerobot . .
docs/source/robotwin.mdx
+20 -13
View File
@@ -2,19 +2,27 @@
RoboTwin 2.0 is a **large-scale dual-arm manipulation benchmark** built on the SAPIEN physics engine. It provides a standardized evaluation protocol for bimanual robotic policies across 60 tasks with strong domain randomization (clutter, lighting, background, tabletop height, and language instructions). RoboTwin 2.0 is a **large-scale dual-arm manipulation benchmark** built on the SAPIEN physics engine. It provides a standardized evaluation protocol for bimanual robotic policies across 60 tasks with strong domain randomization (clutter, lighting, background, tabletop height, and language instructions).
- Paper: [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://robotwin-platform.github.io) - Paper: [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://arxiv.org/abs/2504.13495)
- GitHub: [RoboTwin-Platform/RoboTwin](https://github.com/RoboTwin-Platform/RoboTwin) - GitHub: [RoboTwin-Platform/RoboTwin](https://github.com/RoboTwin-Platform/RoboTwin)
- Leaderboard: [robotwin-platform.github.io/leaderboard](https://robotwin-platform.github.io/leaderboard) - Leaderboard: [robotwin-platform.github.io/leaderboard](https://robotwin-platform.github.io/leaderboard)
- Dataset: [lerobot/robotwin_unified](https://huggingface.co/datasets/lerobot/robotwin_unified) - Dataset: [lerobot/robotwin_unified](https://huggingface.co/datasets/lerobot/robotwin_unified)
<div style={{ textAlign: "center" }}>
<img
src="https://robotwin-platform.github.io/static/images/teaser.png"
alt="RoboTwin 2.0 benchmark overview showing dual-arm manipulation tasks"
style={{ maxWidth: "100%" }}
/>
</div>
## Overview ## Overview
| Property | Value | | Property | Value |
| ------------- | ---------------------------------------------------------- | | ------------- | -------------------------------------------------------- |
| Tasks | 60 dual-arm manipulation tasks | | Tasks | 60 dual-arm manipulation tasks |
| Robot | Aloha-AgileX bimanual (14 DOF, 7 per arm) | | Robot | Aloha-AgileX bimanual (14 DOF, 7 per arm) |
| Action space | 14-dim joint-space, continuous in `[-1, 1]` | | Action space | 14-dim joint-space, continuous in `[-1, 1]` |
| Cameras | `head_camera`, `front_camera`, `left_wrist`, `right_wrist` | | Cameras | `head_camera`, `left_camera`, `right_camera` |
| Simulator | SAPIEN (not MuJoCo) | | Simulator | SAPIEN (not MuJoCo) |
| Eval protocol | 100 episodes/task, 50 demo_clean demonstrations | | Eval protocol | 100 episodes/task, 50 demo_clean demonstrations |
| Eval settings | **Easy** (`demo_clean`) and **Hard** (`demo_randomized`) | | Eval settings | **Easy** (`demo_clean`) and **Hard** (`demo_randomized`) |
@@ -160,14 +168,13 @@ lerobot-eval \
## Camera configuration ## Camera configuration
By default, all four cameras are included: By default, all three cameras are included:
| Camera key | Description | | Camera key | Description |
| -------------- | ------------------------------ | | -------------- | ------------------------------ |
| `head_camera` | Overhead / third-person view | | `head_camera` | Torso-mounted overhead view |
| `front_camera` | Front-facing static camera | | `left_camera` | Left arm wrist-mounted camera |
| `left_wrist` | Left arm wrist-mounted camera | | `right_camera` | Right arm wrist-mounted camera |
| `right_wrist` | Right arm wrist-mounted camera |
To use a subset of cameras, override `--env.camera_names`: To use a subset of cameras, override `--env.camera_names`:
@@ -176,7 +183,7 @@ lerobot-eval \
--policy.path="your-hf-policy-id" \ --policy.path="your-hf-policy-id" \
--env.type=robotwin \ --env.type=robotwin \
--env.task=beat_block_hammer \ --env.task=beat_block_hammer \
--env.camera_names="head_camera,left_wrist,right_wrist" \ --env.camera_names="head_camera,left_camera" \
--eval.batch_size=1 \ --eval.batch_size=1 \
--eval.n_episodes=10 --eval.n_episodes=10
``` ```
@@ -186,14 +193,14 @@ lerobot-eval \
Key parameters for `RoboTwinEnvConfig`: Key parameters for `RoboTwinEnvConfig`:
| Parameter | Default | Description | | Parameter | Default | Description |
| -------------------- | --------------------------------------------------- | ---------------------------------- | | -------------------- | ---------------------------------------- | ---------------------------------- |
| `task` | `"beat_block_hammer"` | Comma-separated task name(s) | | `task` | `"beat_block_hammer"` | Comma-separated task name(s) |
| `fps` | `25` | Simulation FPS | | `fps` | `25` | Simulation FPS |
| `episode_length` | `300` | Max steps per episode | | `episode_length` | `300` | Max steps per episode |
| `obs_type` | `"pixels_agent_pos"` | `"pixels"` or `"pixels_agent_pos"` | | `obs_type` | `"pixels_agent_pos"` | `"pixels"` or `"pixels_agent_pos"` |
| `camera_names` | `"head_camera,front_camera,left_wrist,right_wrist"` | Comma-separated active cameras | | `camera_names` | `"head_camera,left_camera,right_camera"` | Comma-separated active cameras |
| `observation_height` | `480` | Camera pixel height | | `observation_height` | `240` | Camera pixel height |
| `observation_width` | `640` | Camera pixel width | | `observation_width` | `320` | Camera pixel width |
## Leaderboard submission ## Leaderboard submission
src/lerobot/envs/configs.py
+1 -1
View File
@@ -583,7 +583,7 @@ class RoboTwinEnvConfig(EnvConfig):
RoboTwin 2.0 is a dual-arm manipulation benchmark with 60 tasks built on the RoboTwin 2.0 is a dual-arm manipulation benchmark with 60 tasks built on the
SAPIEN simulator. The robot is an Aloha-AgileX bimanual platform with 14 DOF SAPIEN simulator. The robot is an Aloha-AgileX bimanual platform with 14 DOF
(7 per arm). All four cameras are enabled by default. (7 per arm). All three cameras are enabled by default.
See: https://robotwin-platform.github.io See: https://robotwin-platform.github.io
Dataset: https://huggingface.co/datasets/lerobot/robotwin_unified Dataset: https://huggingface.co/datasets/lerobot/robotwin_unified