From 9ead70f01698b6652a3afe49d80c313d8daf9f7e Mon Sep 17 00:00:00 2001 From: Pepijn Date: Fri, 17 Apr 2026 15:18:20 +0100 Subject: [PATCH] fix(ci): swap 4 broken RoboTwin tasks in smoke eval MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The smoke eval hit two upstream issues: - `open_laptop`: bug in OpenMOSS/RoboTwin main — `check_success()` uses `self.arm_tag`, but that attribute is only set inside `play_once()` (the scripted-expert path). During eval `take_action()` calls `check_success()` directly, hitting `AttributeError: 'open_laptop' object has no attribute 'arm_tag'`. - `close_laptop`, `close_microwave`, `place_block`: not present in upstream RoboTwin `envs/` at all — our ROBOTWIN_TASKS tuple drifted from upstream and these names leaked into CI. Replace the four broken tasks with upstream-confirmed equivalents that exist both in ROBOTWIN_TASKS and in RoboTwin's `envs/`: `adjust_bottle`, `lift_pot`, `stamp_seal`, `turn_switch`. New 10-task smoke set: beat_block_hammer, click_bell, handover_block, stack_blocks_two, click_alarmclock, open_microwave, adjust_bottle, lift_pot, stamp_seal, turn_switch. Made-with: Cursor --- .github/workflows/benchmark_tests.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/benchmark_tests.yml b/.github/workflows/benchmark_tests.yml index 523a31175..0c64c159c 100644 --- a/.github/workflows/benchmark_tests.yml +++ b/.github/workflows/benchmark_tests.yml @@ -368,7 +368,7 @@ jobs: cd /opt/robotwin && lerobot-eval \ --policy.path=lerobot/smolvla_robotwin \ --env.type=robotwin \ - --env.task=beat_block_hammer,click_bell,handover_block,open_laptop,stack_blocks_two,click_alarmclock,close_laptop,close_microwave,open_microwave,place_block \ + --env.task=beat_block_hammer,click_bell,handover_block,stack_blocks_two,click_alarmclock,open_microwave,adjust_bottle,lift_pot,stamp_seal,turn_switch \ --eval.batch_size=1 \ --eval.n_episodes=1 \ --eval.use_async_envs=false \ @@ -377,7 +377,7 @@ jobs: --output_dir=/tmp/eval-artifacts python /lerobot/scripts/ci/extract_task_descriptions.py \ --env robotwin \ - --task beat_block_hammer,click_bell,handover_block,open_laptop,stack_blocks_two,click_alarmclock,close_laptop,close_microwave,open_microwave,place_block \ + --task beat_block_hammer,click_bell,handover_block,stack_blocks_two,click_alarmclock,open_microwave,adjust_bottle,lift_pot,stamp_seal,turn_switch \ --output /tmp/eval-artifacts/task_descriptions.json " @@ -394,7 +394,7 @@ jobs: python3 scripts/ci/parse_eval_metrics.py \ --artifacts-dir /tmp/robotwin-artifacts \ --env robotwin \ - --task beat_block_hammer,click_bell,handover_block,open_laptop,stack_blocks_two,click_alarmclock,close_laptop,close_microwave,open_microwave,place_block \ + --task beat_block_hammer,click_bell,handover_block,stack_blocks_two,click_alarmclock,open_microwave,adjust_bottle,lift_pot,stamp_seal,turn_switch \ --policy lerobot/smolvla_robotwin - name: Upload RoboTwin rollout video