- Remove broken Triton issue link from Dockerfile.benchmark.libero
- Add module-level _safe_int helper to guard n_episodes against NaN
- Move _safe_float to module level alongside _safe_int
- Add # zizmor: ignore[unpinned-uses] to all upload-artifact@v4 steps
- Add if: env.HF_USER_TOKEN != '' to Libero smoke eval for fork PRs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Security:
- Remove "Login to Hugging Face" step — it was a no-op (ephemeral
--rm container) that exposed the HF token via CLI argument in
docker inspect / /proc/*/cmdline. The eval step already
re-authenticates via env var.
Functional:
- Remove feat/benchmark-ci from push trigger branches (won't exist
post-merge).
Dockerfiles:
- Pin uv to 0.8.0 (was unpinned, fetching whatever latest ships).
- Add comment explaining the chmod +x ptxas workaround (Triton
packaging bug — ships ptxas without execute bit).
Scripts:
- parse_eval_metrics.py: add note that it runs on bare host and must
stay stdlib-only.
- parse_eval_metrics.py: add NaN guard for avg_sum_reward and eval_s
(was only guarding pc_success).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add scripts/ci/extract_task_descriptions.py: runs inside the benchmark
Docker container (LIBERO/MetaWorld installed) after lerobot-eval and
writes task_descriptions.json mapping task keys to NL instructions.
LIBERO: uses libero.libero.benchmark to get suite.get_task(i).language.
MetaWorld: formats task name as human-readable label.
- Call extraction at the end of each eval bash-c (|| true so never fatal).
- parse_eval_metrics.py reads task_descriptions.json and includes it in
metrics.json so the health dashboard Space can label videos by task.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds scripts/ci/parse_eval_metrics.py and wires it into both Libero and
MetaWorld jobs so the dashboard can read pc_success, avg_sum_reward and
eval_s from the metrics artifact instead of relying on GitHub step timing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pepijn