mirror of
https://github.com/huggingface/lerobot.git
synced 2026-07-26 11:16:00 +00:00
fix(envs): preserve lazy-env metadata + stabilize VLABench reset
Two independent fixes: 1. Port of #3416: add `metadata` / `unwrapped` to `_LazyAsyncVectorEnv` and cache the metadata alongside obs/action spaces in the LIBERO, MetaWorld, and VLABench factories so async eval keeps `env.unwrapped.metadata['render_fps']` working. 2. VLABench `select_mahjong` kept failing the smoke eval with `mjWARN_BADQACC` after the 5-retry ceiling. Bump `_ENSURE_ENV_MAX_ATTEMPTS` to 20, reseed NumPy's global RNG between attempts (VLABench's upstream layout sampler is deterministic given current RNG state — without a reseed, retries replay the same diverging configuration), and raise a clearer `RuntimeError` on exhaustion. Also swap `select_mahjong` out of the CI task list in favor of `select_billiards`, which has a stable layout sampler. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -372,7 +372,7 @@ jobs:
|
||||
lerobot-eval \
|
||||
--policy.path=lerobot/smolvla_vlabench \
|
||||
--env.type=vlabench \
|
||||
--env.task=select_fruit,select_toy,select_book,select_painting,select_drink,select_ingredient,select_mahjong,select_poker,add_condiment,insert_flower \
|
||||
--env.task=select_fruit,select_toy,select_book,select_painting,select_drink,select_ingredient,select_billiards,select_poker,add_condiment,insert_flower \
|
||||
--eval.batch_size=1 \
|
||||
--eval.n_episodes=1 \
|
||||
--eval.use_async_envs=false \
|
||||
@@ -394,7 +394,7 @@ jobs:
|
||||
python3 scripts/ci/parse_eval_metrics.py \
|
||||
--artifacts-dir /tmp/vlabench-artifacts \
|
||||
--env vlabench \
|
||||
--task select_fruit,select_toy,select_book,select_painting,select_drink,select_ingredient,select_mahjong,select_poker,add_condiment,insert_flower \
|
||||
--task select_fruit,select_toy,select_book,select_painting,select_drink,select_ingredient,select_billiards,select_poker,add_condiment,insert_flower \
|
||||
--policy lerobot/smolvla_vlabench
|
||||
|
||||
- name: Upload VLABench rollout video
|
||||
|
||||
Reference in New Issue
Block a user