mirror of
https://github.com/huggingface/lerobot.git
synced 2026-07-26 11:16:00 +00:00
fix: integrate PR #3314 review feedback
- ci(robocerebra): drop redundant hf auth login step (auth is already performed inside the eval step's container). - ci(robocerebra): add Docker Hub login before the image build to pick up the authenticated rate limit. - docs(robocerebra): align eval snippet with the CI command (observation size, camera_name_mapping, use_async_envs, device, empty_cameras=1). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -333,6 +333,12 @@ jobs:
|
|||||||
with:
|
with:
|
||||||
cache-binary: false
|
cache-binary: false
|
||||||
|
|
||||||
|
- name: Login to Docker Hub
|
||||||
|
uses: docker/login-action@v3 # zizmor: ignore[unpinned-uses]
|
||||||
|
with:
|
||||||
|
username: ${{ secrets.DOCKERHUB_LEROBOT_USERNAME }}
|
||||||
|
password: ${{ secrets.DOCKERHUB_LEROBOT_PASSWORD }}
|
||||||
|
|
||||||
- name: Build RoboCerebra benchmark image
|
- name: Build RoboCerebra benchmark image
|
||||||
uses: docker/build-push-action@v6 # zizmor: ignore[unpinned-uses]
|
uses: docker/build-push-action@v6 # zizmor: ignore[unpinned-uses]
|
||||||
with:
|
with:
|
||||||
@@ -344,14 +350,6 @@ jobs:
|
|||||||
cache-from: type=local,src=/tmp/.buildx-cache-robocerebra
|
cache-from: type=local,src=/tmp/.buildx-cache-robocerebra
|
||||||
cache-to: type=local,dest=/tmp/.buildx-cache-robocerebra,mode=max
|
cache-to: type=local,dest=/tmp/.buildx-cache-robocerebra,mode=max
|
||||||
|
|
||||||
- name: Login to Hugging Face
|
|
||||||
if: env.HF_USER_TOKEN != ''
|
|
||||||
run: |
|
|
||||||
docker run --rm \
|
|
||||||
-e HF_HOME=/tmp/hf \
|
|
||||||
lerobot-benchmark-robocerebra:ci \
|
|
||||||
bash -c "hf auth login --token '$HF_USER_TOKEN' --add-to-git-credential && hf auth whoami"
|
|
||||||
|
|
||||||
- name: Run RoboCerebra smoke eval (1 episode)
|
- name: Run RoboCerebra smoke eval (1 episode)
|
||||||
if: env.HF_USER_TOKEN != ''
|
if: env.HF_USER_TOKEN != ''
|
||||||
run: |
|
run: |
|
||||||
|
|||||||
@@ -36,7 +36,7 @@ export MUJOCO_GL=egl # for headless servers (HPC, cloud)
|
|||||||
|
|
||||||
## Evaluation
|
## Evaluation
|
||||||
|
|
||||||
RoboCerebra eval runs against LIBERO's `libero_10` suite with RoboCerebra's camera naming (`image` + `wrist_image`) and an extra empty-camera slot for policies trained with three views:
|
RoboCerebra eval runs against LIBERO's `libero_10` suite with RoboCerebra's camera naming (`image` + `wrist_image`) and an extra empty-camera slot so a three-view-trained policy receives the expected input layout:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
lerobot-eval \
|
lerobot-eval \
|
||||||
@@ -45,10 +45,15 @@ lerobot-eval \
|
|||||||
--env.task=libero_10 \
|
--env.task=libero_10 \
|
||||||
--env.fps=20 \
|
--env.fps=20 \
|
||||||
--env.obs_type=pixels_agent_pos \
|
--env.obs_type=pixels_agent_pos \
|
||||||
|
--env.observation_height=256 \
|
||||||
|
--env.observation_width=256 \
|
||||||
|
'--env.camera_name_mapping={"agentview_image": "image", "robot0_eye_in_hand_image": "wrist_image"}' \
|
||||||
--eval.batch_size=1 \
|
--eval.batch_size=1 \
|
||||||
--eval.n_episodes=10 \
|
--eval.n_episodes=10 \
|
||||||
|
--eval.use_async_envs=false \
|
||||||
|
--policy.device=cuda \
|
||||||
'--rename_map={"observation.images.image": "observation.images.camera1", "observation.images.wrist_image": "observation.images.camera2"}' \
|
'--rename_map={"observation.images.image": "observation.images.camera1", "observation.images.wrist_image": "observation.images.camera2"}' \
|
||||||
--policy.empty_cameras=3
|
--policy.empty_cameras=1
|
||||||
```
|
```
|
||||||
|
|
||||||
### Recommended evaluation episodes
|
### Recommended evaluation episodes
|
||||||
|
|||||||
Reference in New Issue
Block a user