admin/lerobot
1
0
Fork 0
mirror of https://github.com/huggingface/lerobot.git synced 2026-07-26 19:26:16 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(ci): align RoboCerebra eval with training pipeline

Browse Source 
Fixes 5 mismatches that caused 0% success rate:
- env.type: robocerebra (unregistered) → libero
- resolution: 360x360 (default) → 256x256 (matches dataset)
- camera_name_mapping: map eye_in_hand → wrist_image (not image2)
- empty_cameras: 3 → 1 (matches training)
- add HF_USER_TOKEN guard on eval step

Made-with: Cursor
This commit is contained in:
Pepijn
2026-04-16 16:48:06 +02:00
parent 05833e7139
commit bab08a6a49
1 changed files with 6 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/benchmark_tests.yml
+6 -2
View File
@@ -353,6 +353,7 @@ jobs:
bash -c "hf auth login --token '$HF_USER_TOKEN' --add-to-git-credential && hf auth whoami" bash -c "hf auth login --token '$HF_USER_TOKEN' --add-to-git-credential && hf auth whoami"
- name: Run RoboCerebra smoke eval (1 episode) - name: Run RoboCerebra smoke eval (1 episode)
if: env.HF_USER_TOKEN != ''
run: | run: |
docker run --name robocerebra-eval --gpus all \ docker run --name robocerebra-eval --gpus all \
--shm-size=4g \ --shm-size=4g \
@@ -365,14 +366,17 @@ jobs:
hf auth login --token \"\$HF_USER_TOKEN\" --add-to-git-credential 2>/dev/null || true hf auth login --token \"\$HF_USER_TOKEN\" --add-to-git-credential 2>/dev/null || true
lerobot-eval \ lerobot-eval \
--policy.path=lerobot/smolvla_robocerebra \ --policy.path=lerobot/smolvla_robocerebra \
--env.type=robocerebra \ --env.type=libero \
--env.task=libero_10 \ --env.task=libero_10 \
--env.observation_height=256 \
--env.observation_width=256 \
'--env.camera_name_mapping={\"agentview_image\": \"image\", \"robot0_eye_in_hand_image\": \"wrist_image\"}' \
--eval.batch_size=1 \ --eval.batch_size=1 \
--eval.n_episodes=1 \ --eval.n_episodes=1 \
--eval.use_async_envs=false \ --eval.use_async_envs=false \
--policy.device=cuda \ --policy.device=cuda \
'--rename_map={\"observation.images.image\": \"observation.images.camera1\", \"observation.images.wrist_image\": \"observation.images.camera2\"}' \ '--rename_map={\"observation.images.image\": \"observation.images.camera1\", \"observation.images.wrist_image\": \"observation.images.camera2\"}' \
--policy.empty_cameras=3 \ --policy.empty_cameras=1 \
--output_dir=/tmp/eval-artifacts --output_dir=/tmp/eval-artifacts
python scripts/ci/extract_task_descriptions.py \ python scripts/ci/extract_task_descriptions.py \
--env libero --task libero_10 \ --env libero --task libero_10 \