mirror of
https://github.com/huggingface/lerobot.git
synced 2026-05-11 14:49:43 +00:00
Pepijn
a07f22e22c
* feat(envs): add LIBERO-plus robustness benchmark integration
- LiberoPlusEnv config (subclass of LiberoEnv, same gym interface)
- Docker image installing LIBERO-plus fork via PYTHONPATH
- CI workflow: 1-episode smoke eval with pepijn223/smolvla_libero_plus
- pyproject.toml: libero_plus extra
* fix(libero): use suite's perturbation-aware init_states loader
LIBERO-plus's Benchmark class exposes a `get_task_init_states(i)` method that
strips perturbation suffixes (`_table_N`, `_tb_N`, `_view_`, `_language_`,
`_light_`, `_add_`, `_level`) and loads the underlying base `.pruned_init`
file — the on-disk name for a perturbation variant doesn't exist as a file,
only the base does. lerobot's loader was bypassing that logic and trying to
read the suffix-bearing filename directly, which failed for every non-zero
task id and killed the eval before any rollout video could be written.
Delegate to the suite's method when it exists; fall back to the path-based
loader for vanilla LIBERO (which does not provide the method).
Also drop the hf-libero install + init_files copy from the LIBERO-plus
Dockerfile — the LIBERO-plus clone already ships both `bddl_files/` and
`init_files/` for all five suites, so the copy was unnecessary and the
`cp -r` into an existing dir produced a confusing nested layout.
* fix(libero): resolve LIBERO-plus perturbation init_states path ourselves
Delegating to `task_suite.get_task_init_states(i)` works for path resolution
but LIBERO-plus's method calls `torch.load(path)` without `weights_only=False`,
which fails on PyTorch 2.6+ because the pickled init_states contains numpy
objects not in the default allowlist:
_pickle.UnpicklingError: Weights only load failed.
WeightsUnpickler error: Unsupported global:
GLOBAL numpy.core.multiarray._reconstruct was not an allowed global.
Mirror LIBERO-plus's suffix-stripping logic (`_table_N`, `_tb_N`, `_view_`,
`_language_`, `_light_`, `_add_`, `_level`) in our own helper so we can pass
`weights_only=False` ourselves. Vanilla LIBERO task names don't contain any
of these patterns except for `_table_` when followed by the word `center`
(e.g. `pick_up_the_black_bowl_from_table_center_...`), and the regex
requires `_table_\\d+` so semantic uses are preserved.
* fix(libero-plus): download perturbation assets from Sylvest/LIBERO-plus
LIBERO-plus's bddl_base_domain.py resolves scene XMLs with
`os.path.join(DIR_PATH, "../assets")`, so the `assets` key in config.yaml
has no effect on scene lookup — MuJoCo always opens
`<clone>/libero/libero/assets/scenes/...`. With no such directory present,
every perturbation task fails on:
FileNotFoundError: No such file or directory:
.../libero-plus/libero/libero/assets/scenes/tabletop_table_Cobblestone01_GLOSS_6K.xml
These textures, views, and extra objects ship only in the 6.4 GB `assets.zip`
published at `Sylvest/LIBERO-plus` (the LIBERO-plus README explicitly says
to download and unzip it into the package dir). Fetch it via `hf_hub_download`,
unzip into `${LIBERO_PLUS_ROOT}/`, install `unzip`, and point config.yaml at
the extracted dir so everything stays consistent. The download lives in its
own Docker layer so subsequent rebuilds reuse the cached assets.
Drops the lerobot/libero-assets snapshot_download — that mirror only has
vanilla LIBERO textures and is ignored for scene loading anyway.
* fix(libero-plus): flatten deep path prefix from Sylvest/LIBERO-plus assets.zip
The 6.4 GB zip ships with every entry prefixed by
`inspire/hdd/project/embodied-multimodality/public/syfei/libero_new/release/dataset/LIBERO-plus-0/assets/...`
(the author's internal filesystem layout, not the layout the LIBERO-plus
README promises), so the previous `unzip -d ${LIBERO_PLUS_ROOT}/` created
`${LIBERO_PLUS_ROOT}/inspire/.../assets/` — robosuite still opened
`${LIBERO_PLUS_ROOT}/assets/scenes/tabletop_table_Cobblestone01_GLOSS_6K.xml`
and hit the same FileNotFoundError.
Extract to a scratch dir, then `mv` the nested `assets/` subtree to the
expected location. Verified the target file exists in the zip central
directory under that exact prefix.
* refactor(libero): inline init_states resolver behind single regex
Collapse the three-style suffix stripper (split/re.sub/in) into one
compiled regex, drop the (Path, bool) tuple return, and move the
`_add_`/`_level` reshape branch into the caller so each branch loads
its own file and returns directly. Net: -11 lines, one fewer helper.
* refactor(libero-plus): rebase docker image on huggingface/lerobot-gpu
Mirror the libero/metaworld/robomme pattern: start from the nightly GPU
image (apt deps, python, uv, venv, lerobot[all] already there) and only
layer on what LIBERO-plus uniquely needs — its wand/ImageMagick build
deps, the non-extra runtime pips (robosuite==1.4.1, bddl, …), the
PYTHONPATH-shadowed fork, and the 6.4 GB assets.zip.
Drops ~50 lines of duplicated base setup (CUDA FROM, apt python, uv
install, user creation, venv init) the nightly already provides.
123 → 73 lines.
Also:
- Add libero_plus to docs/source/_toctree.yml under Benchmarks so
doc-builder's TOC integrity check stops failing.
- Repoint the docs dataset link from pepijn223/libero_plus_lerobot to
the canonical lerobot/libero_plus.
- Revert the stray uv.lock churn (revision/marker diff that crept in
from an unrelated resolve — unrelated to LIBERO-plus).
* fix(libero-plus): stop touching pyproject + uv.lock
The fast-tests job was rejecting the branch because pyproject.toml had a
[libero_plus] extra whose git dep wasn't represented in uv.lock.
The Docker image no longer needs the extra — it clones LIBERO-plus
directly and PYTHONPATH-shadows hf-libero. Drop [libero_plus] from
pyproject and restore pyproject.toml + uv.lock to exactly what's on
origin/main, so `uv sync --locked --extra test` is a no-op for this PR.
Also repoint the doc/CI/env comments that still mentioned the extra at
the Docker install path.
* fix(libero-plus): strip perturbation metadata from task descriptions
LIBERO-plus builds task.language by space-joining the perturbation-variant
filename, so every non-_language_ variant inherits a trailing blob like
"view 0 0 100 0 0 initstate 0 noise 45" or "add 16". That shows up in the
dashboard video labels and no longer matches the base instruction stored
in the training dataset.
Strip those tokens in extract_task_descriptions.py with an end-anchored
regex over the {view,initstate,noise,add,tb,table,light,level}(+digits)
vocabulary. The anchor preserves mid-sentence literal uses of those words
(e.g. "from table center and place it on the plate") — only the trailing
metadata chain is removed. _language_ variants carry real BDDL-sourced
text and are left untouched.
* ci: point benchmark eval checkpoints at the lerobot/ org mirrors
pepijn223/smolvla_* → lerobot/smolvla_* across every benchmark job in
this branch (libero, metaworld, and the per-branch benchmark). The
checkpoints were mirrored into the lerobot/ org and that's the canonical
location going forward.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: integrate PR #3313 review feedback
- docs: fix paper link to arxiv, add benchmark image, add suite descriptions,
add LIBERO-plus replacement warning, restructure eval section to match
LIBERO doc style, fix policy I/O section, remove false try/except claim
- docker: fix shell grouping for hf-libero uninstall, replace hardcoded
asset path with dynamic find
- ci: add Docker Hub login step, add HF_USER_TOKEN guard on eval step
- envs: add is_libero_plus param to get_task_init_states so vanilla LIBERO
always takes the simple path
* fix(docs): use correct LIBERO-plus teaser image URL
* ci(libero-plus): drop redundant hf auth login step
The standalone login step ran `hf auth login` in a throwaway
`docker run --rm` container, so no credentials persisted. Auth is
already performed inside the eval step's container. Removing the
redundant step per PR #3313 review feedback.
* fix(envs): preserve AsyncVectorEnv metadata/unwrapped in lazy eval envs
Port of #3416 onto this branch. Without these attributes eval crashes
when calling `env.unwrapped.metadata["render_fps"]` with async vector
envs. Adds `metadata` / `unwrapped` to `_LazyAsyncVectorEnv` and
caches the metadata alongside obs/action spaces in the LIBERO and
MetaWorld factories.
* ci: gate Docker Hub login on secret availability
Fork PRs cannot access `secrets.DOCKERHUB_LEROBOT_{USERNAME,PASSWORD}`,
which made every benchmark job fail at the login step before any of
the actual build/eval work could run. Gate the login on the env-var
expansion of the username so the step is skipped (not failed) when
secrets are absent. Mirrors the existing pattern in the VLABench job.
* Update .github/workflows/benchmark_tests.yml
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* Update scripts/ci/extract_task_descriptions.py
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* Update .github/workflows/benchmark_tests.yml
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* Update docker/Dockerfile.benchmark.libero_plus
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* Update .github/workflows/benchmark_tests.yml
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* fix(libero-plus): address review feedback
* ci(libero-plus): fix YAML indentation in upload-artifact steps
The `uses:` key on two upload-artifact steps was at column 0 instead
of nested under the step, causing `pre-commit run check-yaml` to fail
with "expected <block end>, but found '<block mapping start>'".
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
157 lines
3.7 KiB
YAML
157 lines
3.7 KiB
YAML
- sections:
|
|
- local: index
|
|
title: LeRobot
|
|
- local: installation
|
|
title: Installation
|
|
title: Get started
|
|
- sections:
|
|
- local: il_robots
|
|
title: Imitation Learning for Robots
|
|
- local: bring_your_own_policies
|
|
title: Bring Your Own Policies
|
|
- local: integrate_hardware
|
|
title: Bring Your Own Hardware
|
|
- local: hilserl
|
|
title: Train a Robot with RL
|
|
- local: hilserl_sim
|
|
title: Train RL in Simulation
|
|
- local: multi_gpu_training
|
|
title: Multi GPU training
|
|
- local: hil_data_collection
|
|
title: Human In the Loop Data Collection
|
|
- local: peft_training
|
|
title: Training with PEFT (e.g., LoRA)
|
|
- local: rename_map
|
|
title: Using Rename Map and Empty Cameras
|
|
title: "Tutorials"
|
|
- sections:
|
|
- local: lerobot-dataset-v3
|
|
title: Using LeRobotDataset
|
|
- local: porting_datasets_v3
|
|
title: Porting Large Datasets
|
|
- local: using_dataset_tools
|
|
title: Using the Dataset Tools
|
|
- local: dataset_subtask
|
|
title: Using Subtasks in the Dataset
|
|
- local: streaming_video_encoding
|
|
title: Streaming Video Encoding
|
|
title: "Datasets"
|
|
- sections:
|
|
- local: act
|
|
title: ACT
|
|
- local: smolvla
|
|
title: SmolVLA
|
|
- local: pi0
|
|
title: π₀ (Pi0)
|
|
- local: pi0fast
|
|
title: π₀-FAST (Pi0Fast)
|
|
- local: pi05
|
|
title: π₀.₅ (Pi05)
|
|
- local: groot
|
|
title: NVIDIA GR00T N1.5
|
|
- local: xvla
|
|
title: X-VLA
|
|
- local: multi_task_dit
|
|
title: Multitask DiT Policy
|
|
- local: walloss
|
|
title: WALL-OSS
|
|
title: "Policies"
|
|
- sections:
|
|
- local: sarm
|
|
title: SARM
|
|
title: "Reward Models"
|
|
- sections:
|
|
- local: async
|
|
title: Use Async Inference
|
|
- local: rtc
|
|
title: Real-Time Chunking (RTC)
|
|
title: "Inference"
|
|
- sections:
|
|
- local: envhub
|
|
title: Environments from the Hub
|
|
- local: envhub_leisaac
|
|
title: Control & Train Robots in Sim (LeIsaac)
|
|
title: "Simulation"
|
|
- sections:
|
|
- local: adding_benchmarks
|
|
title: Adding a New Benchmark
|
|
- local: libero
|
|
title: LIBERO
|
|
- local: libero_plus
|
|
title: LIBERO-plus
|
|
- local: metaworld
|
|
title: Meta-World
|
|
- local: robotwin
|
|
title: RoboTwin 2.0
|
|
- local: robocasa
|
|
title: RoboCasa365
|
|
- local: robocerebra
|
|
title: RoboCerebra
|
|
- local: robomme
|
|
title: RoboMME
|
|
- local: envhub_isaaclab_arena
|
|
title: NVIDIA IsaacLab Arena Environments
|
|
title: "Benchmarks"
|
|
- sections:
|
|
- local: introduction_processors
|
|
title: Introduction to Robot Processors
|
|
- local: debug_processor_pipeline
|
|
title: Debug your processor pipeline
|
|
- local: implement_your_own_processor
|
|
title: Implement your own processor
|
|
- local: processors_robots_teleop
|
|
title: Processors for Robots and Teleoperators
|
|
- local: env_processor
|
|
title: Environment Processors
|
|
- local: action_representations
|
|
title: Action Representations
|
|
title: "Robot Processors"
|
|
- sections:
|
|
- local: so101
|
|
title: SO-101
|
|
- local: so100
|
|
title: SO-100
|
|
- local: koch
|
|
title: Koch v1.1
|
|
- local: lekiwi
|
|
title: LeKiwi
|
|
- local: hope_jr
|
|
title: Hope Jr
|
|
- local: reachy2
|
|
title: Reachy 2
|
|
- local: unitree_g1
|
|
title: Unitree G1
|
|
- local: earthrover_mini_plus
|
|
title: Earth Rover Mini
|
|
- local: omx
|
|
title: OMX
|
|
- local: openarm
|
|
title: OpenArm
|
|
title: "Robots"
|
|
- sections:
|
|
- local: phone_teleop
|
|
title: Phone
|
|
title: "Teleoperators"
|
|
- sections:
|
|
- local: cameras
|
|
title: Cameras
|
|
title: "Sensors"
|
|
- sections:
|
|
- local: torch_accelerators
|
|
title: PyTorch accelerators
|
|
title: "Supported Hardware"
|
|
- sections:
|
|
- local: notebooks
|
|
title: Notebooks
|
|
- local: feetech
|
|
title: Updating Feetech Firmware
|
|
- local: damiao
|
|
title: Damiao Motors and CAN Bus
|
|
title: "Resources"
|
|
- sections:
|
|
- local: contributing
|
|
title: Contribute to LeRobot
|
|
- local: backwardcomp
|
|
title: Backward compatibility
|
|
title: "About"
|