fix: integrate PR #3396 review feedback (round 2)

- envs(vlabench): drop the unused `episode_index` / `n_envs`
  constructor args — they were stored but never referenced.
- envs(vlabench): replace module-level `print()` calls with the
  standard `logging.getLogger(__name__)` logger (matches the rest
  of the repo's script convention). Covers the two PhysicsError
  retry/step paths and the `create_vlabench_envs` progress lines.
- envs(vlabench): drop `_make_env_fns` and build factories
  MetaWorld-style via a lambda list comprehension — simpler, no
  extra helper.
- docker(vlabench): pin upstream clones to commit SHAs
  (`OpenMOSS/VLABench@cf588fe6`, `motion-planning/rrt-algorithms@e51d95ee`)
  so benchmark images are reproducible.
- ci(vlabench): run `scripts/ci/extract_task_descriptions.py` after
  the eval so `metrics.json` carries per-task natural-language
  labels, matching LIBERO / MetaWorld jobs. Added a VLABench
  extractor that produces cleaned-name labels keyed by `<task>_0`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Pepijn
2026-04-20 14:33:23 +02:00
parent 0a0ebec97a
commit eff1d5a528
4 changed files with 57 additions and 46 deletions
+9 -2
View File
@@ -28,8 +28,15 @@ FROM huggingface/lerobot-gpu:latest
# Patch: constant.py calls os.listdir on ~100 asset/obj/meshes/* dirs at import
# time. Guard the call so missing dirs return [] instead of crashing (in case
# the asset download is partial).
RUN git clone --depth 1 https://github.com/OpenMOSS/VLABench.git ~/VLABench && \
git clone --depth 1 https://github.com/motion-planning/rrt-algorithms.git ~/rrt-algorithms && \
#
# Pinned upstream SHAs for reproducible benchmark runs. Bump when you need
# an upstream fix; don't rely on `main`/`develop` drift.
ARG VLABENCH_SHA=cf588fe60c0c7282174fe979f5913170cfe69017
ARG RRT_ALGORITHMS_SHA=e51d95ee489a225220d6ae2a764c4111f6ba7d85
RUN git clone https://github.com/OpenMOSS/VLABench.git ~/VLABench && \
git -C ~/VLABench checkout ${VLABENCH_SHA} && \
git clone https://github.com/motion-planning/rrt-algorithms.git ~/rrt-algorithms && \
git -C ~/rrt-algorithms checkout ${RRT_ALGORITHMS_SHA} && \
python3 -c "\
import pathlib; \
p = pathlib.Path.home() / 'VLABench/VLABench/configs/constant.py'; \