mirror of
https://github.com/huggingface/lerobot.git
synced 2026-05-21 11:39:50 +00:00
fix(robotwin): sync ROBOTWIN_TASKS + doc with upstream (50 tasks)
The local ROBOTWIN_TASKS tuple drifted from upstream RoboTwin-Platform/RoboTwin. Users passing names like `close_laptop`, `close_microwave`, `dump_bin`, `place_block`, `pour_water`, `fold_cloth`, etc. got past our validator (the names were in the tuple) but then crashed inside robosuite with a confusing error, because those tasks don't exist in upstream `envs/`. - Replace ROBOTWIN_TASKS with a verbatim mirror of upstream's `envs/` directory: 50 tasks as of main (was 60 with many stale entries). Added a `gh api`-based one-liner comment so future bumps are mechanical. - Update the `60 tasks` claims in robotwin.mdx and RoboTwinEnvConfig's docstring to `50`. - Replace the stale example-task table in robotwin.mdx with ten upstream-confirmed examples, and flag `open_laptop` as temporarily broken (its `check_success()` uses `self.arm_tag` which is only set inside `play_once()`; eval-mode callers hit AttributeError). - Rebuild the "Full benchmark" command with the actual 50-task list, omitting `open_laptop`. Made-with: Cursor
This commit is contained in:
+32
-16
@@ -1,6 +1,6 @@
|
|||||||
# RoboTwin 2.0
|
# RoboTwin 2.0
|
||||||
|
|
||||||
RoboTwin 2.0 is a **large-scale dual-arm manipulation benchmark** built on the SAPIEN physics engine. It provides a standardized evaluation protocol for bimanual robotic policies across 60 tasks with strong domain randomization (clutter, lighting, background, tabletop height, and language instructions).
|
RoboTwin 2.0 is a **large-scale dual-arm manipulation benchmark** built on the SAPIEN physics engine. It provides a standardized evaluation protocol for bimanual robotic policies across 50 tasks (as of upstream `main`) with strong domain randomization (clutter, lighting, background, tabletop height, and language instructions).
|
||||||
|
|
||||||
- Paper: [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://arxiv.org/abs/2506.18088)
|
- Paper: [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://arxiv.org/abs/2506.18088)
|
||||||
- GitHub: [RoboTwin-Platform/RoboTwin](https://github.com/RoboTwin-Platform/RoboTwin)
|
- GitHub: [RoboTwin-Platform/RoboTwin](https://github.com/RoboTwin-Platform/RoboTwin)
|
||||||
@@ -13,7 +13,7 @@ RoboTwin 2.0 is a **large-scale dual-arm manipulation benchmark** built on the S
|
|||||||
|
|
||||||
| Property | Value |
|
| Property | Value |
|
||||||
| ------------- | -------------------------------------------------------- |
|
| ------------- | -------------------------------------------------------- |
|
||||||
| Tasks | 60 dual-arm manipulation tasks |
|
| Tasks | 50 dual-arm manipulation tasks |
|
||||||
| Robot | Aloha-AgileX bimanual (14 DOF, 7 per arm) |
|
| Robot | Aloha-AgileX bimanual (14 DOF, 7 per arm) |
|
||||||
| Action space | 14-dim joint-space, continuous in `[-1, 1]` |
|
| Action space | 14-dim joint-space, continuous in `[-1, 1]` |
|
||||||
| Cameras | `head_camera`, `left_camera`, `right_camera` |
|
| Cameras | `head_camera`, `left_camera`, `right_camera` |
|
||||||
@@ -23,21 +23,31 @@ RoboTwin 2.0 is a **large-scale dual-arm manipulation benchmark** built on the S
|
|||||||
|
|
||||||
## Available tasks
|
## Available tasks
|
||||||
|
|
||||||
RoboTwin 2.0 ships with 60 dual-arm manipulation tasks. The full list appears on the [leaderboard](https://robotwin-platform.github.io/leaderboard). Example tasks:
|
RoboTwin 2.0 ships 50 dual-arm manipulation tasks in its upstream `envs/` directory. The canonical list is the `ROBOTWIN_TASKS` tuple in `src/lerobot/envs/robotwin.py`, mirrored verbatim from the upstream repo. Example tasks:
|
||||||
|
|
||||||
| Task | CLI name | Category |
|
| Task | CLI name | Category |
|
||||||
| ---------------------- | ------------------------ | ---------------- |
|
| ------------------------ | ------------------------ | ----------------- |
|
||||||
| Beat block with hammer | `beat_block_hammer` | Tool use |
|
| Beat block with hammer | `beat_block_hammer` | Tool use |
|
||||||
| Open / close laptop | `open_laptop` | Articulated obj |
|
| Click bell / alarm clock | `click_bell` | Precision press |
|
||||||
| Stack blocks (2 / 3) | `stack_blocks_two/three` | Stacking |
|
| Stack blocks (2 / 3) | `stack_blocks_two/three` | Stacking |
|
||||||
| Pour water | `pour_water` | Deformable/fluid |
|
| Stack bowls (2 / 3) | `stack_bowls_two/three` | Stacking |
|
||||||
| Fold cloth | `fold_cloth` | Deformable |
|
| Handover block / mic | `handover_block` | Bimanual coord. |
|
||||||
| Handover block | `handover_block` | Bimanual coord. |
|
| Lift pot | `lift_pot` | Bimanual lift |
|
||||||
| Place shoes | `place_shoes_left/right` | Precision place |
|
| Shake bottle | `shake_bottle` | Continuous motion |
|
||||||
| Scan object | `scan_object` | Mobile manip. |
|
| Turn switch | `turn_switch` | Articulated obj |
|
||||||
|
| Stamp seal | `stamp_seal` | Precision place |
|
||||||
|
| Scan object | `scan_object` | Mobile manip. |
|
||||||
|
|
||||||
Pass a comma-separated list to `--env.task` to run multiple tasks in a single eval sweep.
|
Pass a comma-separated list to `--env.task` to run multiple tasks in a single eval sweep.
|
||||||
|
|
||||||
|
<Tip warning={true}>
|
||||||
|
`open_laptop` is currently broken upstream (its `check_success()` uses
|
||||||
|
`self.arm_tag`, which is only set inside the scripted-expert `play_once()`
|
||||||
|
path and therefore unavailable during normal policy eval). Avoid it until the
|
||||||
|
upstream bug is fixed, or patch the task to default `self.arm_tag = "left"` in
|
||||||
|
`load_actors()`.
|
||||||
|
</Tip>
|
||||||
|
|
||||||
## Dataset
|
## Dataset
|
||||||
|
|
||||||
The RoboTwin 2.0 dataset is available in **LeRobot v3.0 format** on the Hugging Face Hub:
|
The RoboTwin 2.0 dataset is available in **LeRobot v3.0 format** on the Hugging Face Hub:
|
||||||
@@ -46,7 +56,7 @@ The RoboTwin 2.0 dataset is available in **LeRobot v3.0 format** on the Hugging
|
|||||||
lerobot/robotwin_unified
|
lerobot/robotwin_unified
|
||||||
```
|
```
|
||||||
|
|
||||||
It contains over 100,000 pre-collected trajectories across all 60 tasks (79.6 GB, Apache 2.0 license). No format conversion is needed — it is already in the correct LeRobot v3.0 schema with video observations and action labels.
|
It contains over 100,000 pre-collected trajectories across all 50 tasks (79.6 GB, Apache 2.0 license). No format conversion is needed — it is already in the correct LeRobot v3.0 schema with video observations and action labels.
|
||||||
|
|
||||||
You can load it directly with the HF Datasets library:
|
You can load it directly with the HF Datasets library:
|
||||||
|
|
||||||
@@ -149,17 +159,23 @@ lerobot-eval \
|
|||||||
--eval.n_episodes=100
|
--eval.n_episodes=100
|
||||||
```
|
```
|
||||||
|
|
||||||
### Full benchmark (all 60 tasks)
|
### Full benchmark (all 50 tasks)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
lerobot-eval \
|
lerobot-eval \
|
||||||
--policy.path="your-hf-policy-id" \
|
--policy.path="your-hf-policy-id" \
|
||||||
--env.type=robotwin \
|
--env.type=robotwin \
|
||||||
--env.task=adjust_bottle,beat_block_hammer,blocks_ranking_rgb,blocks_ranking_size,click_alarmclock,click_bell,close_laptop,close_microwave,dump_bin,grab_roller,handover_block,handover_cup,handover_diverse_bottles,handover_mic,hanging_mug,insert_pin,lift_pot,make_tea,open_laptop,open_microwave,pick_diverse_bottles,pick_dual_bottles,place_basket,place_block,place_cable,place_can,place_chopsticks,place_cloth,place_container,place_cup,place_diverse_bottles,place_dual_bottles,place_fork,place_knife,place_object_basket,place_ring,place_ruler,place_shoes_left,place_shoes_right,place_spoon,place_toy,pour_water,press_stapler,put_bottles_dustbin,put_object_cabinet,put_shoes_box,rotate_qrcode,scan_object,shake_bottle,shake_bottle_horizontally,stack_blocks_three,stack_blocks_two,stack_bowls_three,stack_bowls_two,stamp_seal,turn_switch,wipe_board,arrange_tools,build_tower,fold_cloth \
|
--env.task=adjust_bottle,beat_block_hammer,blocks_ranking_rgb,blocks_ranking_size,click_alarmclock,click_bell,dump_bin_bigbin,grab_roller,handover_block,handover_mic,hanging_mug,lift_pot,move_can_pot,move_pillbottle_pad,move_playingcard_away,move_stapler_pad,open_microwave,pick_diverse_bottles,pick_dual_bottles,place_a2b_left,place_a2b_right,place_bread_basket,place_bread_skillet,place_burger_fries,place_can_basket,place_cans_plasticbox,place_container_plate,place_dual_shoes,place_empty_cup,place_fan,place_mouse_pad,place_object_basket,place_object_scale,place_object_stand,place_phone_stand,place_shoe,press_stapler,put_bottles_dustbin,put_object_cabinet,rotate_qrcode,scan_object,shake_bottle,shake_bottle_horizontally,stack_blocks_three,stack_blocks_two,stack_bowls_three,stack_bowls_two,stamp_seal,turn_switch \
|
||||||
--eval.batch_size=1 \
|
--eval.batch_size=1 \
|
||||||
--eval.n_episodes=100
|
--eval.n_episodes=100
|
||||||
```
|
```
|
||||||
|
|
||||||
|
<Tip>
|
||||||
|
`open_laptop` is intentionally omitted above because of the upstream
|
||||||
|
`self.arm_tag` bug (see the **Available tasks** section). Re-add it once the
|
||||||
|
upstream fix lands.
|
||||||
|
</Tip>
|
||||||
|
|
||||||
## Camera configuration
|
## Camera configuration
|
||||||
|
|
||||||
By default, all three cameras are included:
|
By default, all three cameras are included:
|
||||||
|
|||||||
@@ -581,7 +581,7 @@ class IsaaclabArenaEnv(HubEnvConfig):
|
|||||||
class RoboTwinEnvConfig(EnvConfig):
|
class RoboTwinEnvConfig(EnvConfig):
|
||||||
"""Configuration for RoboTwin 2.0 benchmark environments.
|
"""Configuration for RoboTwin 2.0 benchmark environments.
|
||||||
|
|
||||||
RoboTwin 2.0 is a dual-arm manipulation benchmark with 60 tasks built on the
|
RoboTwin 2.0 is a dual-arm manipulation benchmark with 50 tasks built on the
|
||||||
SAPIEN simulator. The robot is an Aloha-AgileX bimanual platform with 14 DOF
|
SAPIEN simulator. The robot is an Aloha-AgileX bimanual platform with 14 DOF
|
||||||
(7 per arm). All three cameras are enabled by default.
|
(7 per arm). All three cameras are enabled by default.
|
||||||
|
|
||||||
|
|||||||
@@ -44,7 +44,11 @@ DEFAULT_EPISODE_LENGTH = 300
|
|||||||
DEFAULT_CAMERA_H = 240
|
DEFAULT_CAMERA_H = 240
|
||||||
DEFAULT_CAMERA_W = 320
|
DEFAULT_CAMERA_W = 320
|
||||||
|
|
||||||
# Complete task list from RoboTwin 2.0 (60 tasks, as listed on the leaderboard).
|
# Task list from RoboTwin 2.0's `envs/` directory — mirrors upstream exactly
|
||||||
|
# (50 tasks as of main; earlier revisions had 60 with a different split).
|
||||||
|
# Keep this in sync with:
|
||||||
|
# gh api /repos/RoboTwin-Platform/RoboTwin/contents/envs --paginate \
|
||||||
|
# | jq -r '.[].name' | grep -E '\.py$' | grep -v '^_' | sed 's/\.py$//'
|
||||||
ROBOTWIN_TASKS: tuple[str, ...] = (
|
ROBOTWIN_TASKS: tuple[str, ...] = (
|
||||||
"adjust_bottle",
|
"adjust_bottle",
|
||||||
"beat_block_hammer",
|
"beat_block_hammer",
|
||||||
@@ -52,46 +56,40 @@ ROBOTWIN_TASKS: tuple[str, ...] = (
|
|||||||
"blocks_ranking_size",
|
"blocks_ranking_size",
|
||||||
"click_alarmclock",
|
"click_alarmclock",
|
||||||
"click_bell",
|
"click_bell",
|
||||||
"close_laptop",
|
"dump_bin_bigbin",
|
||||||
"close_microwave",
|
|
||||||
"dump_bin",
|
|
||||||
"grab_roller",
|
"grab_roller",
|
||||||
"handover_block",
|
"handover_block",
|
||||||
"handover_cup",
|
|
||||||
"handover_diverse_bottles",
|
|
||||||
"handover_mic",
|
"handover_mic",
|
||||||
"hanging_mug",
|
"hanging_mug",
|
||||||
"insert_pin",
|
|
||||||
"lift_pot",
|
"lift_pot",
|
||||||
"make_tea",
|
"move_can_pot",
|
||||||
|
"move_pillbottle_pad",
|
||||||
|
"move_playingcard_away",
|
||||||
|
"move_stapler_pad",
|
||||||
"open_laptop",
|
"open_laptop",
|
||||||
"open_microwave",
|
"open_microwave",
|
||||||
"pick_diverse_bottles",
|
"pick_diverse_bottles",
|
||||||
"pick_dual_bottles",
|
"pick_dual_bottles",
|
||||||
"place_basket",
|
"place_a2b_left",
|
||||||
"place_block",
|
"place_a2b_right",
|
||||||
"place_cable",
|
"place_bread_basket",
|
||||||
"place_can",
|
"place_bread_skillet",
|
||||||
"place_chopsticks",
|
"place_burger_fries",
|
||||||
"place_cloth",
|
"place_can_basket",
|
||||||
"place_container",
|
"place_cans_plasticbox",
|
||||||
"place_cup",
|
"place_container_plate",
|
||||||
"place_diverse_bottles",
|
"place_dual_shoes",
|
||||||
"place_dual_bottles",
|
"place_empty_cup",
|
||||||
"place_fork",
|
"place_fan",
|
||||||
"place_knife",
|
"place_mouse_pad",
|
||||||
"place_object_basket",
|
"place_object_basket",
|
||||||
"place_ring",
|
"place_object_scale",
|
||||||
"place_ruler",
|
"place_object_stand",
|
||||||
"place_shoes_left",
|
"place_phone_stand",
|
||||||
"place_shoes_right",
|
"place_shoe",
|
||||||
"place_spoon",
|
|
||||||
"place_toy",
|
|
||||||
"pour_water",
|
|
||||||
"press_stapler",
|
"press_stapler",
|
||||||
"put_bottles_dustbin",
|
"put_bottles_dustbin",
|
||||||
"put_object_cabinet",
|
"put_object_cabinet",
|
||||||
"put_shoes_box",
|
|
||||||
"rotate_qrcode",
|
"rotate_qrcode",
|
||||||
"scan_object",
|
"scan_object",
|
||||||
"shake_bottle",
|
"shake_bottle",
|
||||||
@@ -102,10 +100,6 @@ ROBOTWIN_TASKS: tuple[str, ...] = (
|
|||||||
"stack_bowls_two",
|
"stack_bowls_two",
|
||||||
"stamp_seal",
|
"stamp_seal",
|
||||||
"turn_switch",
|
"turn_switch",
|
||||||
"wipe_board",
|
|
||||||
"arrange_tools",
|
|
||||||
"build_tower",
|
|
||||||
"fold_cloth",
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user