mirror of
https://github.com/huggingface/lerobot.git
synced 2026-07-29 04:36:04 +00:00
Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 45243dcf7c | |||
| 034693f724 | |||
| bb3ef3537f |
-113
@@ -1,113 +0,0 @@
|
|||||||
# LeRobot Docs Audit
|
|
||||||
|
|
||||||
**Status:** Fresh baseline for a new docs redesign effort · **Date:** 2026-07-27
|
|
||||||
**Supersedes:** the June 2026 `DOCS_REDESIGN.md` proposal (lived on the unmerged, now-stale `docs/complete-docs-redesign` branch — abandoned; that branch is ~67k lines behind `main` and should not be resurrected). This document re-audits `docs/source/` from scratch against current `main`, since the docs tree grew substantially in the ~6 weeks between the two audits (many new policies, robots, and benchmarks landed).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Snapshot (current state, in numbers)
|
|
||||||
|
|
||||||
- **IA**: 15 top-level `_toctree.yml` sections, 81 `local:` entries, 0 broken toctree links.
|
|
||||||
- **Files**: 97 total files in `docs/source/` = 80 real `.mdx` pages + 16 orphaned `policy_*_README.md` stubs (up from 14 in June) + 1 `contributing.md` (a genuine symlink to root `CONTRIBUTING.md`).
|
|
||||||
- **Growth**: `.mdx` pages grew 49 → 80 in 6 months (+63%, ~5.2 pages/month), arriving in **bursts against a shared, hand-edited `_toctree.yml`** — e.g. 6 benchmark pages landed via 6 separate PRs within 48 hours (Apr 2026); 3 policy pages within 3 days (Jul 2026).
|
|
||||||
- **CLI docs coverage improved sharply since June**: 17 of 18 `lerobot-*` commands now documented (only `lerobot-info`, a diagnostics helper, is not) — the June baseline was 6/17 undocumented.
|
|
||||||
- **Decision content**: zero comparison/decision pages exist in `docs/source/` for Policies (15 options), Robots (11), Benchmarks (10), or Teleoperators — this equivalent content already exists, fully written, in root `AGENT_GUIDE.md` §6, but was never ported or linked in.
|
|
||||||
- **Teleoperators**: dedicated toctree section has only 2 pages vs. 16-17 registered teleoperator types in `src/`, but all types do get at least one doc mention somewhere (usually buried inside a robot's hardware page) — a discoverability gap, not an absence of content.
|
|
||||||
- **Orphan policy READMEs**: of the 16, 3 (Diffusion, TD-MPC, VQ-BeT) have **zero real published documentation anywhere** — the orphan stub is their only artifact. 13 duplicate an actively-maintained `.mdx` page under a different name. One (MolmoAct2) is proven to have silently rotted: a 39-line pre-refactor stub sits next to a 495-line maintained page that diverged 481 lines ago.
|
|
||||||
- **Root `README.md` itself has drifted**: its "Supported Hardware" line omits SO-101 (the flagship robot) entirely; its "SoTA Models" table links several policies to thin orphan stubs instead of the richer tutorial pages that exist for them.
|
|
||||||
- **4 concrete broken/stale copy-paste commands** verified in published pages (`act.mdx`, `il_robots.mdx` ×2 spots, `lerobot-dataset-v3.mdx`, `openarm.mdx` — the last added as recently as Jan 2026, so a fresh miss, not just aging drift).
|
|
||||||
- **Process/CI**: no CODEOWNERS, no `_redirects.yml`, no doc-consistency or toctree-completeness check anywhere; the docs CI workflow triggers only on `paths: docs/**`, so code-only PRs get zero automated docs signal (empirically responsible for the one real lag found — OMX's docs page, 41 days after its code). Despite this, 18 of 19 spot-checked new integrations (EVO1, FastWAM, LingBot-VA, MolmoAct2, RTC, VLA-JEPA, X-VLA, WallOSS, Damiao, Hope Jr, OpenArm, reBot B601, Reachy 2, Unitree G1, RoboCasa365, VLABench, IsaacLab Arena, etc.) shipped docs in the *same commit* as the feature — organic discipline is currently strong.
|
|
||||||
- **No content template** exists for policy or robot doc pages: heading sets/length vary 80-528 lines with almost no shared vocabulary; by contrast, benchmarks *do* have a written template (`adding_benchmarks.mdx`) and the resulting 8 pages are visibly consistent — direct proof a template is what produces consistency here.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Strengths
|
|
||||||
|
|
||||||
1. **Organic contributor discipline is currently much better than the June audit's numbers implied.** 18/19 spot-checked new policies/robots/benchmarks shipped docs in the same PR as the code; 17/18 CLI commands are documented; all teleoperator types get at least one mention. The underlying practice is healthier than the structural numbers (orphans, thin nav sections) suggest — the real problem is increasingly discoverability and consistency, not absence of effort.
|
|
||||||
2. **The doc-builder CI pipeline (PR previews, main build, versioned release builds) is solid, standard infrastructure already in place** — nothing bespoke to build or maintain, and it's the same tooling Transformers uses.
|
|
||||||
3. **A working, provably-effective template pattern already exists for one catalog (benchmarks)** — `adding_benchmarks.mdx`'s checklist + "at a glance" table produces 8 structurally consistent pages. This is the single best evidence in the repo for what fixes the policy/robot template gap, and it needs no new invention, just extension.
|
|
||||||
4. **Genuine single-source-of-truth patterns already exist and work**: `docs/source/contributing.md` is a real symlink to root `CONTRIBUTING.md` (verified via `ls -la`/`find -type l`). Most policy `src/README.md` files are also symlinked into `docs/source/policy_*_README.md`, eliminating drift at that layer (though the docs-side stub itself remains an unlinked orphan — see weaknesses).
|
|
||||||
5. **Zero broken toctree links**, and specific pages are genuinely high quality: `cheat-sheet.mdx` (spot-checked line-by-line against source, fully accurate), `bring_your_own_policies.mdx` (clear scope, working PR checklist), `installation.mdx`'s tables/OS tabs, and the SO-100/SO-101 hardware pages.
|
|
||||||
6. **Root `README.md` already contains a usable taxonomy for policies** (Imitation Learning / RL / VLAs / World Models / Reward Models) that the flat 15-item Policies sidebar never adopted — the raw material for a better IA already exists in-repo.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Weaknesses
|
|
||||||
|
|
||||||
### High
|
|
||||||
- **No comparison/decision-guidance page for any multi-option category** (Policies=15, Robots=11, Benchmarks=10, Teleoperators) — the exact content (decision rules, profiling snapshot) already exists in `AGENT_GUIDE.md` §6 but was never ported into `docs/source`.
|
|
||||||
- **The landing page (`index.mdx`) provides zero navigation** — 23 lines of marketing copy and a Discord link, no path to installation, cheat-sheet, or audience-specific guidance.
|
|
||||||
- **Teleoperators' dedicated nav section (2 pages) badly undersells real coverage** (16-17 implementations, all mentioned somewhere) — a new user browsing that sidebar section would wrongly conclude only phone/Isaac teleop exist. Content exists; it's a pure discoverability failure.
|
|
||||||
- **Multiple concrete broken/stale CLI examples in published, frequently copy-pasted pages**: `act.mdx` tells readers to eval with `lerobot-record` while the shown code uses `lerobot-rollout`; an identical shell-breaking snippet (`\`-terminated comment swallowing a continuation) is duplicated in `il_robots.mdx` and `lerobot-dataset-v3.mdx`; `il_robots.mdx:421` uses the dead `--control.push_to_hub` flag namespace; `openarm.mdx:208-210` (added Jan 2026) gives a `lerobot-record` example with flat pre-draccus flags that don't exist on the current config classes.
|
|
||||||
- **3 shipping policies (Diffusion, TD-MPC, VQ-BeT) have zero real published documentation** — only a thin, toctree-unreferenced stub exists for each.
|
|
||||||
- **Root `README.md` has drifted as the project scaled**: its Supported Hardware line omits SO-101 entirely; its SoTA Models table links ACT/Diffusion/VQ-BeT/Multitask-DiT/TDMPC/GR00T/SmolVLA to thin orphan stubs instead of the richer tutorial pages that exist for several of them.
|
|
||||||
- **No content template for policy/robot doc pages, unlike benchmarks** — heading sets and length vary 80-528 lines with almost no shared vocabulary (`molmoact2.mdx` has no Overview section at all; `smolvla.mdx` has no Overview/Architecture/Citation/License and reads as pure tutorial).
|
|
||||||
- **Docs CI triggers only on `paths: docs/**`** — code-only PRs get zero automated docs signal; this is the mechanism directly responsible for the one measured lag (OMX's docs page shipped 41 days after its code, in a separate PR).
|
|
||||||
- **Proof that orphan-stub drift isn't hypothetical**: `policy_molmoact2_README.md` is a stale 39-line pre-refactor snapshot sitting 481 lines behind the maintained 495-line page it once mirrored — nothing caught this silently rotting.
|
|
||||||
|
|
||||||
### Medium
|
|
||||||
- 16 orphaned `policy_*_README.md` files unreferenced by the toctree; 13 are pure duplicate cruft shadowing a maintained same-topic `.mdx` page, creating a false "two files to keep in sync" impression for contributors.
|
|
||||||
- No `_redirects.yml` anywhere — future cleanup of the orphans/renames has no 404 safety net.
|
|
||||||
- `installation.mdx` ends on a dangling forward-reference ("follow the link below to use LeRobot with your robot") — no link follows.
|
|
||||||
- `cheat-sheet.mdx`'s "Policy Types" line is stale (`act, diffusion, smolvla, pi05`) against the current 15-entry Policies section, and lists `diffusion`, which has no working toctree page at all.
|
|
||||||
- "Tutorials" (10 pages) mixes beginner, contributor, and RL-researcher content in one flat, unlabeled list.
|
|
||||||
- The EnvHub feature family is split inconsistently across two unrelated sections (`envhub.mdx`/`envhub_leisaac.mdx` under Simulation, `envhub_isaaclab_arena.mdx` under Benchmarks) despite identical framing/opening text.
|
|
||||||
- Admonition syntax is split GFM `[!NOTE]` vs. doc-builder `<Tip>` across sampled pages; since the site actually builds with `hf-doc-builder`, the majority pattern may not render as a styled callout at all (plausible, not independently render-verified).
|
|
||||||
- No CODEOWNERS file anywhere — zero designated reviewer routing for `docs/source`.
|
|
||||||
- `CONTRIBUTING.md` never links the good in-repo extension guides (`adding_benchmarks.mdx`, `bring_your_own_policies.mdx`, `integrate_hardware.mdx`) and states no docs-required policy.
|
|
||||||
- The "docs required" PR checklist convention is applied inconsistently: Policies and Benchmarks have an explicit required-docs checklist; `integrate_hardware.mdx` (Robots/Teleoperators) has none — a plausible root cause of the Teleoperators gap above.
|
|
||||||
- Growth repeatedly stacks simultaneous PRs against one shared, hand-edited `_toctree.yml` (6 benchmark PRs in 48h; 3 policy PRs in 3 days) — a merge/oversight risk even though no damage from it was found yet.
|
|
||||||
- `policy_sarm_README.md` has no source of truth left to sync against at all (SARM moved `policies/`→`rewards/`, no README carried over) — pure abandoned cruft next to the real 593-line `sarm.mdx`.
|
|
||||||
- No automated check anywhere for toctree completeness or README/mdx sync — the only CI gate is whether the doc-builder build succeeds, which doesn't catch missing coverage.
|
|
||||||
|
|
||||||
### Low
|
|
||||||
- No dedicated "Motors" section despite `motors/` being named as a distinct hardware layer in `CLAUDE.md`; `feetech.mdx`/`damiao.mdx` sit in a catch-all "Resources" section instead.
|
|
||||||
- "Sensors" is a single-page top-level nav section (`cameras.mdx` only).
|
|
||||||
- Policies (15), Benchmarks (10), and Robots (11) are flat, ungrouped lists with no internal sub-headings, despite `README.md` already having a usable taxonomy that could be reused.
|
|
||||||
- "SO-101" vs. "SO101" naming is inconsistent across ~15 files, and even within a single file (`so100.mdx` uses both).
|
|
||||||
- `hilserl.mdx` (950 lines) and `il_robots.mdx` (638 lines) each mix tutorial, reference, and troubleshooting content in one long page.
|
|
||||||
- `reachy2_camera` has zero doc mentions anywhere; the `zmq` camera backend is documented only incidentally inside `unitree_g1.mdx` rather than centrally in `cameras.mdx`.
|
|
||||||
- The PR template's "Documentation updated" line is a self-reported, unenforced checkbox.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Recommendations
|
|
||||||
|
|
||||||
### 1. Quick wins (small effort, ship this week, no maintainer proposal needed)
|
|
||||||
- Fix the 4 verified broken commands: `act.mdx` (`lerobot-record`→`lerobot-rollout`), the duplicated shell-breaking snippet in `il_robots.mdx` + `lerobot-dataset-v3.mdx`, `il_robots.mdx`'s `--control.push_to_hub`→`--dataset.push_to_hub`, and `openarm.mdx`'s invalid flat-flag record example.
|
|
||||||
- Delete the 3 fully-orphaned policy stubs (Diffusion, TD-MPC, VQ-BeT) and give each a real toctree page by promoting the existing `src/` README content.
|
|
||||||
- Delete the remaining 13 duplicate orphan stubs and the dead `policy_sarm_README.md`.
|
|
||||||
- Fix `installation.mdx`'s dangling ending and `cheat-sheet.mdx`'s stale policy-type list.
|
|
||||||
- Fix root `README.md`: add SO-101 to the Supported Hardware line; repoint the SoTA Models table's links from orphan stubs to the richer existing tutorial pages.
|
|
||||||
- Move `envhub_isaaclab_arena.mdx` into Simulation (one-line YAML change).
|
|
||||||
- Add 2-4 orientation sentences + links to the top of `index.mdx` (installation, cheat-sheet, "I have hardware" / "I don't" / "I want to contribute") without touching its marketing framing.
|
|
||||||
- Publish a single "Choosing a policy" page that ports the already-written decision rules from `AGENT_GUIDE.md` §6 — highest-leverage fix available.
|
|
||||||
- Start a `docs/source/_redirects.yml` now, before the orphan cleanup above creates the first real dead links.
|
|
||||||
- No action needed on the `contributing.md` "duplication" claim — verified it's a working symlink, not an anti-pattern.
|
|
||||||
|
|
||||||
### 2. Structural / UX changes (need a proposal + maintainer buy-in, phase as separate PRs)
|
|
||||||
- Split "Tutorials" into an explicit beginner-facing section vs. an "Advanced & Research/Contributor" section (mechanical YAML reorg, no content rewrites).
|
|
||||||
- Add a "Teleoperators" index page listing all types with one-line descriptions and links to wherever each is actually documented today.
|
|
||||||
- Re-group the flat Policies/Benchmarks/Robots lists into sub-headings reusing the taxonomy `README.md` already has.
|
|
||||||
- Treat a dedicated "Motors" section, deeper sub-grouping, and closing the remaining teleoperator/robot coverage gaps as a phased backlog of independently reviewable PRs rather than one restructure.
|
|
||||||
- Standardize SO-101/SO101 naming and (after confirming actual doc-builder rendering behavior) the admonition syntax, each as one mechanical, low-risk PR.
|
|
||||||
- Longer-term and biggest-ticket: port more of `AGENT_GUIDE.md`'s procedural content (training duration heuristics, eval targets, data-collection tips) into the published site.
|
|
||||||
|
|
||||||
### 3. Maintainability / process changes (prevent debt from reaccumulating at the current growth rate)
|
|
||||||
- Port `adding_benchmarks.mdx`'s explicit "writing a doc page" checklist/template into `bring_your_own_policies.mdx` and `integrate_hardware.mdx`.
|
|
||||||
- Add a CODEOWNERS entry for `docs/source/` (and ideally `_toctree.yml` specifically) so docs PRs get routed to a real reviewer.
|
|
||||||
- Link the extension guides from `CONTRIBUTING.md` and state a docs-required policy there.
|
|
||||||
- Add a lightweight CI/pre-commit script (not a full doc-builder run) that fails when (a) a `docs/source/*.mdx` file isn't reachable from `_toctree.yml`, or (b) a new `register_subclass` policy/robot/teleoperator/env lands with no corresponding doc file in the same diff.
|
|
||||||
- Decide the fate of the `policy_*_README.md` symlink convention going forward: it is currently self-perpetuating because `bring_your_own_policies.mdx`'s own checklist instructs new contributors to create the file that ends up orphaned. Either fold citation/paper content into the main `.mdx` tutorial, or wire the stub into the toctree as a linked citation anchor.
|
|
||||||
- Make the PR template's "Documentation updated" checkbox actionable (e.g., "(N/A if this PR only touches tests/CI/refactors)").
|
|
||||||
- Defer heavier generated-registry/support-matrix tooling (Transformers/Ultralytics-style single source of truth) until closer to 1.0.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Notes on cross-checking
|
|
||||||
|
|
||||||
Five independent audit passes fed this report; two disagreements were resolved during synthesis:
|
|
||||||
- **Toctree section count** — 15 is correct (independently parsed twice from `_toctree.yml`).
|
|
||||||
- **`contributing.md`** — it's a working symlink, not a hand-duplicated anti-pattern; one pass's `diff`-based claim didn't survive checking `find -type l`.
|
|
||||||
- **Orphan-README "sync mechanism"** — mostly fixed at the `src/`↔`docs` layer via symlinks, but that just moved the unresolved drift to the docs-side stub's absence from the toctree, and left old pre-symlink stubs (MolmoAct2) as dead leftovers.
|
|
||||||
- **Teleoperators coverage** — the nav-section framing ("~2/16") and the "mentioned somewhere" framing are both true; the real gap is discoverability, not content.
|
|
||||||
+1
-1
@@ -76,7 +76,7 @@ If your local computer doesn't have a powerful GPU, you can utilize Google Colab
|
|||||||
|
|
||||||
## Evaluating ACT
|
## Evaluating ACT
|
||||||
|
|
||||||
Once training is complete, you can evaluate your ACT policy using the `lerobot-rollout` command with your trained policy. This will run inference and record evaluation episodes:
|
Once training is complete, you can evaluate your ACT policy using the `lerobot-record` command with your trained policy. This will run inference and record evaluation episodes:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
lerobot-rollout \
|
lerobot-rollout \
|
||||||
|
|||||||
@@ -194,8 +194,8 @@ lerobot-record \
|
|||||||
--dataset.single_task="Navigate around obstacles" \
|
--dataset.single_task="Navigate around obstacles" \
|
||||||
--dataset.streaming_encoding=true \
|
--dataset.streaming_encoding=true \
|
||||||
--dataset.encoder_threads=2 \
|
--dataset.encoder_threads=2 \
|
||||||
|
# --dataset.rgb_encoder.vcodec=auto \
|
||||||
--display_data=true
|
--display_data=true
|
||||||
# Optionally, set --dataset.rgb_encoder.vcodec=auto to pick a specific video codec
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Replace `your_username/dataset_name` with your Hugging Face username and a name for your dataset.
|
Replace `your_username/dataset_name` with your Hugging Face username and a name for your dataset.
|
||||||
|
|||||||
@@ -232,8 +232,8 @@ lerobot-record \
|
|||||||
--dataset.private=true \
|
--dataset.private=true \
|
||||||
--dataset.streaming_encoding=true \
|
--dataset.streaming_encoding=true \
|
||||||
--dataset.encoder_threads=2 \
|
--dataset.encoder_threads=2 \
|
||||||
|
# --dataset.rgb_encoder.vcodec=auto \
|
||||||
--display_data=true
|
--display_data=true
|
||||||
# Optionally, set --dataset.rgb_encoder.vcodec=auto to pick a specific video codec
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Replay
|
### Replay
|
||||||
@@ -278,6 +278,6 @@ lerobot-record \
|
|||||||
--dataset.num_episodes=10 \
|
--dataset.num_episodes=10 \
|
||||||
--dataset.streaming_encoding=true \
|
--dataset.streaming_encoding=true \
|
||||||
--dataset.encoder_threads=2 \
|
--dataset.encoder_threads=2 \
|
||||||
|
# --dataset.rgb_encoder.vcodec=auto \
|
||||||
--policy.path=outputs/train/hopejr_hand/checkpoints/last/pretrained_model
|
--policy.path=outputs/train/hopejr_hand/checkpoints/last/pretrained_model
|
||||||
# Optionally, set --dataset.rgb_encoder.vcodec=auto to pick a specific video codec
|
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -207,8 +207,8 @@ lerobot-record \
|
|||||||
--dataset.num_episodes=5 \
|
--dataset.num_episodes=5 \
|
||||||
--dataset.single_task="Grab the black cube" \
|
--dataset.single_task="Grab the black cube" \
|
||||||
--dataset.streaming_encoding=true \
|
--dataset.streaming_encoding=true \
|
||||||
|
# --dataset.rgb_encoder.vcodec=auto \
|
||||||
--dataset.encoder_threads=2
|
--dataset.encoder_threads=2
|
||||||
# Optionally, set --dataset.rgb_encoder.vcodec=auto to pick a specific video codec
|
|
||||||
```
|
```
|
||||||
</hfoption>
|
</hfoption>
|
||||||
<hfoption id="API example">
|
<hfoption id="API example">
|
||||||
@@ -418,7 +418,7 @@ If you want to dive deeper into this important topic, you can check out the [blo
|
|||||||
|
|
||||||
## Visualize a dataset
|
## Visualize a dataset
|
||||||
|
|
||||||
If you uploaded your dataset to the hub with `--dataset.push_to_hub=true`, you can [visualize your dataset online](https://huggingface.co/spaces/lerobot/visualize_dataset) by copy pasting your repo id given by:
|
If you uploaded your dataset to the hub with `--control.push_to_hub=true`, you can [visualize your dataset online](https://huggingface.co/spaces/lerobot/visualize_dataset) by copy pasting your repo id given by:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
echo ${HF_USER}/so101_test
|
echo ${HF_USER}/so101_test
|
||||||
|
|||||||
@@ -44,8 +44,8 @@ lerobot-record \
|
|||||||
--dataset.num_episodes=5 \
|
--dataset.num_episodes=5 \
|
||||||
--dataset.single_task="Grab the black cube" \
|
--dataset.single_task="Grab the black cube" \
|
||||||
--dataset.streaming_encoding=true \
|
--dataset.streaming_encoding=true \
|
||||||
|
# --dataset.rgb_encoder.vcodec=auto \
|
||||||
--dataset.encoder_threads=2
|
--dataset.encoder_threads=2
|
||||||
# Optionally, set --dataset.rgb_encoder.vcodec=auto to pick a specific video codec
|
|
||||||
```
|
```
|
||||||
|
|
||||||
See the [recording guide](./il_robots#record-a-dataset) for more details.
|
See the [recording guide](./il_robots#record-a-dataset) for more details.
|
||||||
|
|||||||
@@ -205,10 +205,9 @@ lerobot-record \
|
|||||||
--teleop.type=openarm_leader \
|
--teleop.type=openarm_leader \
|
||||||
--teleop.port=can1 \
|
--teleop.port=can1 \
|
||||||
--teleop.id=my_leader \
|
--teleop.id=my_leader \
|
||||||
--dataset.repo_id=my_hf_username/my_openarm_dataset \
|
--repo-id=my_hf_username/my_openarm_dataset \
|
||||||
--dataset.single_task="Grab the black cube" \
|
--fps=30 \
|
||||||
--dataset.fps=30 \
|
--num-episodes=10
|
||||||
--dataset.num_episodes=10
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Configuration Options
|
## Configuration Options
|
||||||
|
|||||||
@@ -161,8 +161,8 @@ lerobot-record \
|
|||||||
--dataset.private=true \
|
--dataset.private=true \
|
||||||
--dataset.streaming_encoding=true \
|
--dataset.streaming_encoding=true \
|
||||||
--dataset.encoder_threads=2 \
|
--dataset.encoder_threads=2 \
|
||||||
|
# --dataset.rgb_encoder.vcodec=auto \
|
||||||
--display_data=true
|
--display_data=true
|
||||||
# Optionally, set --dataset.rgb_encoder.vcodec=auto to pick a specific video codec
|
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Specific Options
|
#### Specific Options
|
||||||
@@ -203,8 +203,8 @@ lerobot-record \
|
|||||||
--dataset.private=true \
|
--dataset.private=true \
|
||||||
--dataset.streaming_encoding=true \
|
--dataset.streaming_encoding=true \
|
||||||
--dataset.encoder_threads=2 \
|
--dataset.encoder_threads=2 \
|
||||||
|
# --dataset.rgb_encoder.vcodec=auto \
|
||||||
--display_data=true
|
--display_data=true
|
||||||
# Optionally, set --dataset.rgb_encoder.vcodec=auto to pick a specific video codec
|
|
||||||
```
|
```
|
||||||
|
|
||||||
##### `--robot.use_external_commands`
|
##### `--robot.use_external_commands`
|
||||||
|
|||||||
+14
-13
@@ -100,19 +100,20 @@ Once you are logged in, you can run inference in your setup by doing:
|
|||||||
lerobot-rollout \
|
lerobot-rollout \
|
||||||
--strategy.type=base \
|
--strategy.type=base \
|
||||||
--robot.type=so101_follower \
|
--robot.type=so101_follower \
|
||||||
--robot.port=/dev/ttyACM0 \
|
--robot.port=/dev/ttyACM0 \ # <- Use your port
|
||||||
--robot.id=my_blue_follower_arm \
|
--robot.id=my_blue_follower_arm \ # <- Use your robot id
|
||||||
--robot.cameras="{ front: {type: opencv, index_or_path: 8, width: 640, height: 480, fps: 30}}" \
|
--robot.cameras="{ front: {type: opencv, index_or_path: 8, width: 640, height: 480, fps: 30}}" \ # <- Use your cameras
|
||||||
--task="Grasp a lego block and put it in the bin." \
|
--task="Grasp a lego block and put it in the bin." \ # <- Use the same task description you used in your dataset recording
|
||||||
--policy.path=HF_USER/FINETUNE_MODEL_NAME
|
# <- RTC optional, use when running on low power hardware \
|
||||||
|
# --inference.type=rtc \
|
||||||
|
# --inference.rtc.execution_horizon=10 \
|
||||||
|
# --inference.rtc.max_guidance_weight=10.0 \
|
||||||
|
# <- Teleop optional if you want to teleoperate in between episodes \
|
||||||
|
# --teleop.type=so100_leader \
|
||||||
|
# --teleop.port=/dev/ttyACM0 \
|
||||||
|
# --teleop.id=my_red_leader_arm \
|
||||||
|
# --display_data=true #optional use if you want to see the camera stream \
|
||||||
|
--policy.path=HF_USER/FINETUNE_MODEL_NAME # <- Use your fine-tuned model
|
||||||
```
|
```
|
||||||
|
|
||||||
Replace `--robot.port`, `--robot.id`, `--robot.cameras`, `--task`, and `--policy.path` with your own port, robot ID, camera setup, task description (matching what you used when recording your dataset), and fine-tuned model repo ID.
|
|
||||||
|
|
||||||
A few optional flags you can add to the command above:
|
|
||||||
|
|
||||||
- **RTC** (useful on low-power hardware): `--inference.type=rtc --inference.rtc.execution_horizon=10 --inference.rtc.max_guidance_weight=10.0`
|
|
||||||
- **Teleoperate in between episodes**: `--teleop.type=so100_leader --teleop.port=/dev/ttyACM0 --teleop.id=my_red_leader_arm`
|
|
||||||
- **See the camera stream**: `--display_data=true`
|
|
||||||
|
|
||||||
Depending on your evaluation setup, you can configure the duration and the number of episodes to record for your evaluation suite.
|
Depending on your evaluation setup, you can configure the duration and the number of episodes to record for your evaluation suite.
|
||||||
|
|||||||
@@ -14,7 +14,6 @@
|
|||||||
import builtins
|
import builtins
|
||||||
import datetime as dt
|
import datetime as dt
|
||||||
import json
|
import json
|
||||||
import multiprocessing
|
|
||||||
import os
|
import os
|
||||||
import tempfile
|
import tempfile
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
@@ -102,12 +101,6 @@ class TrainPipelineConfig(HubMixin):
|
|||||||
batch_size: int = 8
|
batch_size: int = 8
|
||||||
prefetch_factor: int = 4
|
prefetch_factor: int = 4
|
||||||
persistent_workers: bool = True
|
persistent_workers: bool = True
|
||||||
# DataLoader worker start method. "spawn" is safer than "fork" with
|
|
||||||
# non-fork-safe libs (PyAV / torchcodec / ffmpeg), but adds some
|
|
||||||
# worker-startup time per run since workers re-import modules instead
|
|
||||||
# of inheriting parent state. Override with `--dataloader_multiprocessing_context=fork`
|
|
||||||
# when appropriate, or set it to `null` to use Python's platform default.
|
|
||||||
dataloader_multiprocessing_context: str | None = "spawn"
|
|
||||||
steps: int = 100_000
|
steps: int = 100_000
|
||||||
# Run policy in the simulation environment every N steps to measure reward/success (0 = disabled).
|
# Run policy in the simulation environment every N steps to measure reward/success (0 = disabled).
|
||||||
env_eval_freq: int = 20_000
|
env_eval_freq: int = 20_000
|
||||||
@@ -219,17 +212,6 @@ class TrainPipelineConfig(HubMixin):
|
|||||||
self.reward_model.pretrained_path = str(policy_dir)
|
self.reward_model.pretrained_path = str(policy_dir)
|
||||||
|
|
||||||
def validate(self) -> None:
|
def validate(self) -> None:
|
||||||
available_contexts = multiprocessing.get_all_start_methods()
|
|
||||||
if (
|
|
||||||
self.dataloader_multiprocessing_context is not None
|
|
||||||
and self.dataloader_multiprocessing_context not in available_contexts
|
|
||||||
):
|
|
||||||
raise ValueError(
|
|
||||||
"`dataloader_multiprocessing_context` must be None or one of "
|
|
||||||
f"{available_contexts} on this platform, got "
|
|
||||||
f"{self.dataloader_multiprocessing_context!r}."
|
|
||||||
)
|
|
||||||
|
|
||||||
self._resolve_pretrained_from_cli()
|
self._resolve_pretrained_from_cli()
|
||||||
|
|
||||||
if self.policy is None and self.reward_model is None:
|
if self.policy is None and self.reward_model is None:
|
||||||
|
|||||||
@@ -519,6 +519,13 @@ def compute_episode_stats(
|
|||||||
if features[key]["dtype"] in {"string", "language"}:
|
if features[key]["dtype"] in {"string", "language"}:
|
||||||
continue
|
continue
|
||||||
|
|
||||||
|
# Features with a zero-width dimension contain no statistics-bearing
|
||||||
|
# values. Skip them like strings instead of letting
|
||||||
|
# get_feature_stats -> RunningQuantileStats.update reshape a size-0 array,
|
||||||
|
# which raises "ValueError: cannot reshape array of size 0".
|
||||||
|
if any(dim == 0 for dim in features[key].get("shape", ())):
|
||||||
|
continue
|
||||||
|
|
||||||
if features[key]["dtype"] in ["image", "video"]:
|
if features[key]["dtype"] in ["image", "video"]:
|
||||||
ep_ft_array = sample_images(data)
|
ep_ft_array = sample_images(data)
|
||||||
axes_to_reduce = (0, 2, 3)
|
axes_to_reduce = (0, 2, 3)
|
||||||
|
|||||||
@@ -64,12 +64,20 @@ def get_hf_features_from_features(features: dict) -> datasets.Features:
|
|||||||
continue
|
continue
|
||||||
elif ft["dtype"] == "image":
|
elif ft["dtype"] == "image":
|
||||||
hf_features[key] = datasets.Image()
|
hf_features[key] = datasets.Image()
|
||||||
|
elif len(ft["shape"]) > 1 and any(dim == 0 for dim in ft["shape"]):
|
||||||
|
raise ValueError(
|
||||||
|
f"Multidimensional features with a zero-width dimension are not supported: "
|
||||||
|
f"'{key}' has shape {ft['shape']}. Only the one-dimensional shape (0,) is supported."
|
||||||
|
)
|
||||||
elif ft["shape"] == (1,):
|
elif ft["shape"] == (1,):
|
||||||
hf_features[key] = datasets.Value(dtype=ft["dtype"])
|
hf_features[key] = datasets.Value(dtype=ft["dtype"])
|
||||||
elif len(ft["shape"]) == 1:
|
elif len(ft["shape"]) == 1:
|
||||||
hf_features[key] = datasets.Sequence(
|
# A zero-width feature (shape=(0,)) has no fixed-size Arrow representation:
|
||||||
length=ft["shape"][0], feature=datasets.Value(dtype=ft["dtype"])
|
# pyarrow rejects a fixed-size list of length 0 ("list_size needs to be a
|
||||||
)
|
# strict positive integer"). Store it as a variable-length sequence
|
||||||
|
# (length=-1) so each per-frame value is simply an empty list.
|
||||||
|
seq_length = ft["shape"][0] if ft["shape"][0] > 0 else -1
|
||||||
|
hf_features[key] = datasets.Sequence(length=seq_length, feature=datasets.Value(dtype=ft["dtype"]))
|
||||||
elif len(ft["shape"]) == 2:
|
elif len(ft["shape"]) == 2:
|
||||||
hf_features[key] = datasets.Array2D(shape=ft["shape"], dtype=ft["dtype"])
|
hf_features[key] = datasets.Array2D(shape=ft["shape"], dtype=ft["dtype"])
|
||||||
elif len(ft["shape"]) == 3:
|
elif len(ft["shape"]) == 3:
|
||||||
|
|||||||
@@ -155,7 +155,6 @@ class MetaworldEnv(gym.Env):
|
|||||||
env.model.cam_pos[2] = [0.75, 0.075, 0.7]
|
env.model.cam_pos[2] = [0.75, 0.075, 0.7]
|
||||||
env.reset()
|
env.reset()
|
||||||
env._freeze_rand_vec = False # otherwise no randomization
|
env._freeze_rand_vec = False # otherwise no randomization
|
||||||
env.seeded_rand_vec = True # use seeded RNG so reset(seed=X) controls object positions
|
|
||||||
self._env = env
|
self._env = env
|
||||||
|
|
||||||
def render(self) -> np.ndarray:
|
def render(self) -> np.ndarray:
|
||||||
@@ -221,8 +220,6 @@ class MetaworldEnv(gym.Env):
|
|||||||
self._ensure_env()
|
self._ensure_env()
|
||||||
super().reset(seed=seed)
|
super().reset(seed=seed)
|
||||||
|
|
||||||
if seed is not None:
|
|
||||||
self._env.seed(seed)
|
|
||||||
raw_obs, info = self._env.reset(seed=seed)
|
raw_obs, info = self._env.reset(seed=seed)
|
||||||
|
|
||||||
observation = self._format_raw_obs(raw_obs)
|
observation = self._format_raw_obs(raw_obs)
|
||||||
|
|||||||
@@ -177,7 +177,6 @@ def make_pre_post_processors(
|
|||||||
return make_groot_pre_post_processors_from_pretrained(
|
return make_groot_pre_post_processors_from_pretrained(
|
||||||
config=policy_cfg,
|
config=policy_cfg,
|
||||||
pretrained_path=pretrained_path,
|
pretrained_path=pretrained_path,
|
||||||
revision=pretrained_revision,
|
|
||||||
dataset_stats=kwargs.get("dataset_stats"),
|
dataset_stats=kwargs.get("dataset_stats"),
|
||||||
dataset_meta=kwargs.get("dataset_meta"),
|
dataset_meta=kwargs.get("dataset_meta"),
|
||||||
preprocessor_overrides=kwargs.get("preprocessor_overrides"),
|
preprocessor_overrides=kwargs.get("preprocessor_overrides"),
|
||||||
|
|||||||
@@ -475,7 +475,6 @@ def make_groot_pre_post_processors_from_pretrained(
|
|||||||
config: GrootConfig,
|
config: GrootConfig,
|
||||||
pretrained_path: str,
|
pretrained_path: str,
|
||||||
*,
|
*,
|
||||||
revision: str | None = None,
|
|
||||||
dataset_stats: dict[str, dict[str, torch.Tensor]] | None = None,
|
dataset_stats: dict[str, dict[str, torch.Tensor]] | None = None,
|
||||||
dataset_meta: Any | None = None,
|
dataset_meta: Any | None = None,
|
||||||
preprocessor_overrides: dict[str, Any] | None = None,
|
preprocessor_overrides: dict[str, Any] | None = None,
|
||||||
@@ -512,7 +511,6 @@ def make_groot_pre_post_processors_from_pretrained(
|
|||||||
|
|
||||||
preprocessor, postprocessor = _load_groot_processor_pipelines(
|
preprocessor, postprocessor = _load_groot_processor_pipelines(
|
||||||
pretrained_path,
|
pretrained_path,
|
||||||
revision=revision,
|
|
||||||
preprocessor_overrides=preprocessor_overrides,
|
preprocessor_overrides=preprocessor_overrides,
|
||||||
postprocessor_overrides=postprocessor_overrides,
|
postprocessor_overrides=postprocessor_overrides,
|
||||||
preprocessor_config_filename=preprocessor_config_filename,
|
preprocessor_config_filename=preprocessor_config_filename,
|
||||||
@@ -528,7 +526,6 @@ def make_groot_pre_post_processors_from_pretrained(
|
|||||||
def _load_groot_processor_pipelines(
|
def _load_groot_processor_pipelines(
|
||||||
pretrained_path: str,
|
pretrained_path: str,
|
||||||
*,
|
*,
|
||||||
revision: str | None,
|
|
||||||
preprocessor_overrides: dict[str, Any],
|
preprocessor_overrides: dict[str, Any],
|
||||||
postprocessor_overrides: dict[str, Any],
|
postprocessor_overrides: dict[str, Any],
|
||||||
preprocessor_config_filename: str,
|
preprocessor_config_filename: str,
|
||||||
@@ -543,7 +540,6 @@ def _load_groot_processor_pipelines(
|
|||||||
preprocessor = PolicyProcessorPipeline.from_pretrained(
|
preprocessor = PolicyProcessorPipeline.from_pretrained(
|
||||||
pretrained_model_name_or_path=pretrained_path,
|
pretrained_model_name_or_path=pretrained_path,
|
||||||
config_filename=preprocessor_config_filename,
|
config_filename=preprocessor_config_filename,
|
||||||
revision=revision,
|
|
||||||
overrides=preprocessor_overrides,
|
overrides=preprocessor_overrides,
|
||||||
to_transition=batch_to_transition,
|
to_transition=batch_to_transition,
|
||||||
to_output=transition_to_batch,
|
to_output=transition_to_batch,
|
||||||
@@ -551,7 +547,6 @@ def _load_groot_processor_pipelines(
|
|||||||
postprocessor = PolicyProcessorPipeline.from_pretrained(
|
postprocessor = PolicyProcessorPipeline.from_pretrained(
|
||||||
pretrained_model_name_or_path=pretrained_path,
|
pretrained_model_name_or_path=pretrained_path,
|
||||||
config_filename=postprocessor_config_filename,
|
config_filename=postprocessor_config_filename,
|
||||||
revision=revision,
|
|
||||||
overrides=postprocessor_overrides,
|
overrides=postprocessor_overrides,
|
||||||
to_transition=policy_action_to_transition,
|
to_transition=policy_action_to_transition,
|
||||||
to_output=transition_to_policy_action,
|
to_output=transition_to_policy_action,
|
||||||
|
|||||||
@@ -713,8 +713,6 @@ class DataProcessorPipeline[TInput, TOutput](HubMixin):
|
|||||||
ProcessorMigrationError: If the model requires migration to processor format.
|
ProcessorMigrationError: If the model requires migration to processor format.
|
||||||
"""
|
"""
|
||||||
model_id = str(pretrained_model_name_or_path)
|
model_id = str(pretrained_model_name_or_path)
|
||||||
model_path = Path(model_id)
|
|
||||||
is_local_source = model_path.is_dir() or model_path.is_file()
|
|
||||||
hub_download_kwargs = {
|
hub_download_kwargs = {
|
||||||
"force_download": force_download,
|
"force_download": force_download,
|
||||||
"resume_download": resume_download,
|
"resume_download": resume_download,
|
||||||
@@ -733,7 +731,7 @@ class DataProcessorPipeline[TInput, TOutput](HubMixin):
|
|||||||
|
|
||||||
# 3. Build steps with overrides
|
# 3. Build steps with overrides
|
||||||
steps, validated_overrides = cls._build_steps_with_overrides(
|
steps, validated_overrides = cls._build_steps_with_overrides(
|
||||||
loaded_config, overrides or {}, model_id, base_path, hub_download_kwargs, is_local_source
|
loaded_config, overrides or {}, model_id, base_path, hub_download_kwargs
|
||||||
)
|
)
|
||||||
|
|
||||||
# 4. Validate that all overrides were used
|
# 4. Validate that all overrides were used
|
||||||
@@ -923,7 +921,6 @@ class DataProcessorPipeline[TInput, TOutput](HubMixin):
|
|||||||
model_id: str,
|
model_id: str,
|
||||||
base_path: Path | None,
|
base_path: Path | None,
|
||||||
hub_download_kwargs: dict[str, Any],
|
hub_download_kwargs: dict[str, Any],
|
||||||
is_local_source: bool = False,
|
|
||||||
) -> tuple[list[ProcessorStep], set[str]]:
|
) -> tuple[list[ProcessorStep], set[str]]:
|
||||||
"""Build all processor steps with overrides and state loading.
|
"""Build all processor steps with overrides and state loading.
|
||||||
|
|
||||||
@@ -947,7 +944,7 @@ class DataProcessorPipeline[TInput, TOutput](HubMixin):
|
|||||||
3. **State Loading** (via _load_step_state):
|
3. **State Loading** (via _load_step_state):
|
||||||
- **If step has "state_file"**: Load tensor state from .safetensors
|
- **If step has "state_file"**: Load tensor state from .safetensors
|
||||||
- **Local first**: Check base_path/state_file.safetensors
|
- **Local first**: Check base_path/state_file.safetensors
|
||||||
- **Hub fallback**: Download state file if the pipeline was loaded from the Hub
|
- **Hub fallback**: Download state file if not found locally
|
||||||
- **Optional**: Only load if step has load_state_dict method
|
- **Optional**: Only load if step has load_state_dict method
|
||||||
|
|
||||||
4. **Override Tracking**:
|
4. **Override Tracking**:
|
||||||
@@ -965,7 +962,6 @@ class DataProcessorPipeline[TInput, TOutput](HubMixin):
|
|||||||
model_id: The model identifier (needed for Hub state file downloads)
|
model_id: The model identifier (needed for Hub state file downloads)
|
||||||
base_path: Local directory path for finding state files
|
base_path: Local directory path for finding state files
|
||||||
hub_download_kwargs: Parameters for hf_hub_download (tokens, cache, etc.)
|
hub_download_kwargs: Parameters for hf_hub_download (tokens, cache, etc.)
|
||||||
is_local_source: Whether model_id resolved to a local directory or config file.
|
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Tuple of (instantiated_steps_list, unused_override_keys)
|
Tuple of (instantiated_steps_list, unused_override_keys)
|
||||||
@@ -979,9 +975,7 @@ class DataProcessorPipeline[TInput, TOutput](HubMixin):
|
|||||||
steps, remaining_override_keys = cls._build_steps_from_config(loaded_config, overrides)
|
steps, remaining_override_keys = cls._build_steps_from_config(loaded_config, overrides)
|
||||||
|
|
||||||
for step_instance, step_entry in zip(steps, loaded_config["steps"], strict=True):
|
for step_instance, step_entry in zip(steps, loaded_config["steps"], strict=True):
|
||||||
cls._load_step_state(
|
cls._load_step_state(step_instance, step_entry, model_id, base_path, hub_download_kwargs)
|
||||||
step_instance, step_entry, model_id, base_path, hub_download_kwargs, is_local_source
|
|
||||||
)
|
|
||||||
|
|
||||||
return steps, remaining_override_keys
|
return steps, remaining_override_keys
|
||||||
|
|
||||||
@@ -1145,7 +1139,6 @@ class DataProcessorPipeline[TInput, TOutput](HubMixin):
|
|||||||
model_id: str,
|
model_id: str,
|
||||||
base_path: Path | None,
|
base_path: Path | None,
|
||||||
hub_download_kwargs: dict[str, Any],
|
hub_download_kwargs: dict[str, Any],
|
||||||
is_local_source: bool = False,
|
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Load state dictionary for a processor step if available.
|
"""Load state dictionary for a processor step if available.
|
||||||
|
|
||||||
@@ -1164,7 +1157,7 @@ class DataProcessorPipeline[TInput, TOutput](HubMixin):
|
|||||||
- **Use case**: Loading from local saved model directory
|
- **Use case**: Loading from local saved model directory
|
||||||
|
|
||||||
2. **Hub download fallback**: Download state file from repository
|
2. **Hub download fallback**: Download state file from repository
|
||||||
- **When triggered**: Local file not found and the pipeline source is a Hub repo
|
- **When triggered**: Local file not found or base_path is None
|
||||||
- **Process**: Use hf_hub_download with same parameters as config
|
- **Process**: Use hf_hub_download with same parameters as config
|
||||||
- **Example**: Download "normalize_step_0.safetensors" from "user/repo"
|
- **Example**: Download "normalize_step_0.safetensors" from "user/repo"
|
||||||
- **Result**: Downloaded to local cache, path returned
|
- **Result**: Downloaded to local cache, path returned
|
||||||
@@ -1185,7 +1178,6 @@ class DataProcessorPipeline[TInput, TOutput](HubMixin):
|
|||||||
model_id: The model identifier (used for Hub downloads if needed)
|
model_id: The model identifier (used for Hub downloads if needed)
|
||||||
base_path: Local directory path for finding state files (None for Hub-only)
|
base_path: Local directory path for finding state files (None for Hub-only)
|
||||||
hub_download_kwargs: Parameters for hf_hub_download (tokens, cache, etc.)
|
hub_download_kwargs: Parameters for hf_hub_download (tokens, cache, etc.)
|
||||||
is_local_source: Whether model_id resolved to a local directory or config file.
|
|
||||||
|
|
||||||
Note:
|
Note:
|
||||||
This method modifies step_instance in-place and returns None.
|
This method modifies step_instance in-place and returns None.
|
||||||
@@ -1199,12 +1191,6 @@ class DataProcessorPipeline[TInput, TOutput](HubMixin):
|
|||||||
# Try local file first
|
# Try local file first
|
||||||
if base_path and (base_path / state_filename).exists():
|
if base_path and (base_path / state_filename).exists():
|
||||||
state_path = str(base_path / state_filename)
|
state_path = str(base_path / state_filename)
|
||||||
elif is_local_source:
|
|
||||||
state_path = base_path / state_filename if base_path else Path(state_filename)
|
|
||||||
raise FileNotFoundError(
|
|
||||||
f"State file '{state_filename}' was not found for local processor pipeline "
|
|
||||||
f"'{model_id}' at '{state_path}'."
|
|
||||||
)
|
|
||||||
else:
|
else:
|
||||||
# Download from Hub
|
# Download from Hub
|
||||||
state_path = hf_hub_download(
|
state_path = hf_hub_download(
|
||||||
|
|||||||
@@ -323,10 +323,6 @@ class LeKiwiClient(Robot):
|
|||||||
np.ndarray: the action sent to the motors, potentially clipped.
|
np.ndarray: the action sent to the motors, potentially clipped.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# Action values may be torch tensors (e.g. replayed from a dataset) or numpy
|
|
||||||
# scalars; json.dumps only serializes Python primitives, so coerce each value to a
|
|
||||||
# plain float before sending.
|
|
||||||
action = {key: float(value) for key, value in action.items()}
|
|
||||||
self.zmq_cmd_socket.send_string(json.dumps(action)) # action is in motor space
|
self.zmq_cmd_socket.send_string(json.dumps(action)) # action is in motor space
|
||||||
|
|
||||||
# TODO(Steven): Remove the np conversion when it is possible to record a non-numpy array value
|
# TODO(Steven): Remove the np conversion when it is possible to record a non-numpy array value
|
||||||
|
|||||||
@@ -326,17 +326,8 @@ class RolloutConfig:
|
|||||||
|
|
||||||
policy_path = parser.get_path_arg("policy")
|
policy_path = parser.get_path_arg("policy")
|
||||||
if policy_path:
|
if policy_path:
|
||||||
yaml_overrides = parser.get_yaml_overrides("policy")
|
cli_overrides = parser.get_cli_overrides("policy")
|
||||||
cli_overrides = parser.get_cli_overrides("policy") or []
|
self.policy = PreTrainedConfig.from_pretrained(policy_path, cli_overrides=cli_overrides)
|
||||||
policy_overrides = yaml_overrides + cli_overrides
|
|
||||||
pretrained_revision = parser.parse_arg("pretrained_revision", cli_overrides)
|
|
||||||
if pretrained_revision is None:
|
|
||||||
pretrained_revision = parser.parse_arg("pretrained_revision", yaml_overrides)
|
|
||||||
self.policy = PreTrainedConfig.from_pretrained(
|
|
||||||
policy_path,
|
|
||||||
revision=pretrained_revision,
|
|
||||||
cli_overrides=policy_overrides,
|
|
||||||
)
|
|
||||||
self.policy.pretrained_path = policy_path
|
self.policy.pretrained_path = policy_path
|
||||||
if self.policy is None:
|
if self.policy is None:
|
||||||
raise ValueError("--policy.path is required for rollout")
|
raise ValueError("--policy.path is required for rollout")
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ from threading import Event
|
|||||||
|
|
||||||
import torch
|
import torch
|
||||||
|
|
||||||
from lerobot.configs import FeatureType, PreTrainedConfig
|
from lerobot.configs import FeatureType
|
||||||
from lerobot.datasets import (
|
from lerobot.datasets import (
|
||||||
LeRobotDataset,
|
LeRobotDataset,
|
||||||
aggregate_pipeline_dataset_features,
|
aggregate_pipeline_dataset_features,
|
||||||
@@ -159,35 +159,6 @@ class RolloutContext:
|
|||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def _load_pretrained_policy(policy_config: PreTrainedConfig) -> PreTrainedPolicy:
|
|
||||||
"""Load policy weights, keeping adapter and base-model revisions independent."""
|
|
||||||
pretrained_revision = policy_config.pretrained_revision
|
|
||||||
policy_class = get_policy_class(policy_config.type)
|
|
||||||
|
|
||||||
if not policy_config.use_peft:
|
|
||||||
return policy_class.from_pretrained(
|
|
||||||
policy_config.pretrained_path,
|
|
||||||
config=policy_config,
|
|
||||||
revision=pretrained_revision,
|
|
||||||
)
|
|
||||||
|
|
||||||
from peft import PeftConfig, PeftModel
|
|
||||||
|
|
||||||
peft_path = policy_config.pretrained_path
|
|
||||||
peft_config = PeftConfig.from_pretrained(peft_path, revision=pretrained_revision)
|
|
||||||
policy = policy_class.from_pretrained(
|
|
||||||
pretrained_name_or_path=peft_config.base_model_name_or_path,
|
|
||||||
config=policy_config,
|
|
||||||
revision=peft_config.revision,
|
|
||||||
)
|
|
||||||
return PeftModel.from_pretrained(
|
|
||||||
policy,
|
|
||||||
peft_path,
|
|
||||||
config=peft_config,
|
|
||||||
revision=pretrained_revision,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def build_rollout_context(
|
def build_rollout_context(
|
||||||
cfg: RolloutConfig,
|
cfg: RolloutConfig,
|
||||||
shutdown_event: Event,
|
shutdown_event: Event,
|
||||||
@@ -205,6 +176,7 @@ def build_rollout_context(
|
|||||||
# --- 1. Policy (heavy I/O, but no hardware yet) -------------------
|
# --- 1. Policy (heavy I/O, but no hardware yet) -------------------
|
||||||
logger.info("Loading policy from '%s'...", cfg.policy.pretrained_path)
|
logger.info("Loading policy from '%s'...", cfg.policy.pretrained_path)
|
||||||
policy_config = cfg.policy
|
policy_config = cfg.policy
|
||||||
|
policy_class = get_policy_class(policy_config.type)
|
||||||
|
|
||||||
if hasattr(policy_config, "compile_model"):
|
if hasattr(policy_config, "compile_model"):
|
||||||
policy_config.compile_model = cfg.use_torch_compile
|
policy_config.compile_model = cfg.use_torch_compile
|
||||||
@@ -215,7 +187,17 @@ def build_rollout_context(
|
|||||||
"Please use `cpu` or `cuda` backend."
|
"Please use `cpu` or `cuda` backend."
|
||||||
)
|
)
|
||||||
|
|
||||||
policy = _load_pretrained_policy(policy_config)
|
if policy_config.use_peft:
|
||||||
|
from peft import PeftConfig, PeftModel
|
||||||
|
|
||||||
|
peft_path = policy_config.pretrained_path
|
||||||
|
peft_config = PeftConfig.from_pretrained(peft_path)
|
||||||
|
policy = policy_class.from_pretrained(
|
||||||
|
pretrained_name_or_path=peft_config.base_model_name_or_path, config=policy_config
|
||||||
|
)
|
||||||
|
policy = PeftModel.from_pretrained(policy, peft_path, config=peft_config)
|
||||||
|
else:
|
||||||
|
policy = policy_class.from_pretrained(policy_config.pretrained_path, config=policy_config)
|
||||||
|
|
||||||
if is_rtc:
|
if is_rtc:
|
||||||
policy.config.rtc_config = cfg.inference.rtc
|
policy.config.rtc_config = cfg.inference.rtc
|
||||||
@@ -410,7 +392,6 @@ def build_rollout_context(
|
|||||||
preprocessor, postprocessor = make_pre_post_processors(
|
preprocessor, postprocessor = make_pre_post_processors(
|
||||||
policy_cfg=policy_config,
|
policy_cfg=policy_config,
|
||||||
pretrained_path=cfg.policy.pretrained_path,
|
pretrained_path=cfg.policy.pretrained_path,
|
||||||
pretrained_revision=policy_config.pretrained_revision,
|
|
||||||
dataset_stats=dataset_stats,
|
dataset_stats=dataset_stats,
|
||||||
preprocessor_overrides={
|
preprocessor_overrides={
|
||||||
"device_processor": {"device": cfg.device},
|
"device_processor": {"device": cfg.device},
|
||||||
|
|||||||
@@ -61,7 +61,6 @@ from lerobot.robots import ( # noqa: F401
|
|||||||
earthrover_mini_plus,
|
earthrover_mini_plus,
|
||||||
hope_jr,
|
hope_jr,
|
||||||
koch_follower,
|
koch_follower,
|
||||||
lekiwi,
|
|
||||||
make_robot_from_config,
|
make_robot_from_config,
|
||||||
omx_follower,
|
omx_follower,
|
||||||
openarm_follower,
|
openarm_follower,
|
||||||
|
|||||||
@@ -71,16 +71,6 @@ from lerobot.utils.utils import (
|
|||||||
from .lerobot_eval import eval_policy_all
|
from .lerobot_eval import eval_policy_all
|
||||||
|
|
||||||
|
|
||||||
def _dataloader_worker_kwargs(cfg: TrainPipelineConfig) -> dict[str, Any]:
|
|
||||||
"""Return worker-only DataLoader options, disabling them for single-process loading."""
|
|
||||||
workers_enabled = cfg.num_workers > 0
|
|
||||||
return {
|
|
||||||
"prefetch_factor": cfg.prefetch_factor if workers_enabled else None,
|
|
||||||
"persistent_workers": cfg.persistent_workers and workers_enabled,
|
|
||||||
"multiprocessing_context": cfg.dataloader_multiprocessing_context if workers_enabled else None,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def update_policy(
|
def update_policy(
|
||||||
train_metrics: MetricsTracker,
|
train_metrics: MetricsTracker,
|
||||||
policy: PreTrainedPolicy,
|
policy: PreTrainedPolicy,
|
||||||
@@ -483,7 +473,8 @@ def train(cfg: TrainPipelineConfig, accelerator: "Accelerator | None" = None):
|
|||||||
pin_memory=device.type == "cuda",
|
pin_memory=device.type == "cuda",
|
||||||
drop_last=False,
|
drop_last=False,
|
||||||
collate_fn=collate_fn,
|
collate_fn=collate_fn,
|
||||||
**_dataloader_worker_kwargs(cfg),
|
prefetch_factor=cfg.prefetch_factor if cfg.num_workers > 0 else None,
|
||||||
|
persistent_workers=cfg.persistent_workers and cfg.num_workers > 0,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Build eval dataloader if a held-out split exists
|
# Build eval dataloader if a held-out split exists
|
||||||
@@ -509,7 +500,8 @@ def train(cfg: TrainPipelineConfig, accelerator: "Accelerator | None" = None):
|
|||||||
pin_memory=device.type == "cuda",
|
pin_memory=device.type == "cuda",
|
||||||
drop_last=False,
|
drop_last=False,
|
||||||
collate_fn=eval_collate_fn,
|
collate_fn=eval_collate_fn,
|
||||||
**_dataloader_worker_kwargs(cfg),
|
prefetch_factor=cfg.prefetch_factor if cfg.num_workers > 0 else None,
|
||||||
|
persistent_workers=cfg.persistent_workers and cfg.num_workers > 0,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Prepare everything with accelerator
|
# Prepare everything with accelerator
|
||||||
|
|||||||
@@ -687,6 +687,26 @@ def test_compute_episode_stats_string_features_skipped():
|
|||||||
assert "q01" in stats["action"]
|
assert "q01" in stats["action"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("shape", [(0,), (0, 2), (2, 0), (1, 0, 2)])
|
||||||
|
def test_compute_episode_stats_zero_width_feature_skipped(shape):
|
||||||
|
"""Features with any zero-width dimension carry no values and are skipped."""
|
||||||
|
episode_data = {
|
||||||
|
"action": np.random.normal(0, 1, (100, 5)).astype(np.float32),
|
||||||
|
"target": np.zeros((100, *shape), dtype=np.float32),
|
||||||
|
}
|
||||||
|
features = {
|
||||||
|
"action": {"dtype": "float32", "shape": (5,)},
|
||||||
|
"target": {"dtype": "float32", "shape": shape},
|
||||||
|
}
|
||||||
|
|
||||||
|
stats = compute_episode_stats(episode_data, features)
|
||||||
|
|
||||||
|
# Zero-width features are skipped, just like strings; non-empty features are unaffected.
|
||||||
|
assert "target" not in stats
|
||||||
|
assert "action" in stats
|
||||||
|
assert "q01" in stats["action"]
|
||||||
|
|
||||||
|
|
||||||
def test_aggregate_feature_stats_with_quantiles():
|
def test_aggregate_feature_stats_with_quantiles():
|
||||||
"""Test aggregating feature stats that include quantiles."""
|
"""Test aggregating feature stats that include quantiles."""
|
||||||
stats_ft_list = [
|
stats_ft_list = [
|
||||||
|
|||||||
@@ -27,6 +27,7 @@ pytest.importorskip("datasets", reason="datasets is required (install lerobot[da
|
|||||||
|
|
||||||
from lerobot.configs import VideoEncoderConfig
|
from lerobot.configs import VideoEncoderConfig
|
||||||
from lerobot.datasets.dataset_writer import _encode_video_worker
|
from lerobot.datasets.dataset_writer import _encode_video_worker
|
||||||
|
from lerobot.datasets.feature_utils import get_hf_features_from_features
|
||||||
from lerobot.datasets.lerobot_dataset import LeRobotDataset
|
from lerobot.datasets.lerobot_dataset import LeRobotDataset
|
||||||
from lerobot.datasets.utils import DEFAULT_IMAGE_PATH
|
from lerobot.datasets.utils import DEFAULT_IMAGE_PATH
|
||||||
from tests.fixtures.constants import DEFAULT_FPS, DUMMY_REPO_ID
|
from tests.fixtures.constants import DEFAULT_FPS, DUMMY_REPO_ID
|
||||||
@@ -189,6 +190,36 @@ def test_save_multiple_episodes(tmp_path):
|
|||||||
assert dataset.meta.total_frames == total_frames
|
assert dataset.meta.total_frames == total_frames
|
||||||
|
|
||||||
|
|
||||||
|
def test_save_episode_with_zero_width_feature(tmp_path):
|
||||||
|
"""A one-dimensional empty numeric feature round-trips and has no statistics."""
|
||||||
|
features = {
|
||||||
|
**SIMPLE_FEATURES,
|
||||||
|
"target": {"dtype": "float32", "shape": (0,), "names": None},
|
||||||
|
}
|
||||||
|
root = tmp_path / "ds"
|
||||||
|
dataset = LeRobotDataset.create(repo_id=DUMMY_REPO_ID, fps=DEFAULT_FPS, features=features, root=root)
|
||||||
|
for _ in range(4):
|
||||||
|
dataset.add_frame(_make_frame(features))
|
||||||
|
dataset.save_episode()
|
||||||
|
dataset.finalize()
|
||||||
|
|
||||||
|
assert dataset.meta.total_episodes == 1
|
||||||
|
assert dataset.meta.total_frames == 4
|
||||||
|
|
||||||
|
reloaded = LeRobotDataset(repo_id=DUMMY_REPO_ID, root=root)
|
||||||
|
target = np.asarray(reloaded[0]["target"])
|
||||||
|
assert target.shape == (0,)
|
||||||
|
assert "target" not in (reloaded.meta.stats or {})
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("shape", [(0, 2), (2, 0), (1, 0, 2)])
|
||||||
|
def test_multidimensional_zero_width_feature_rejected(shape):
|
||||||
|
features = {"target": {"dtype": "float32", "shape": shape, "names": None}}
|
||||||
|
|
||||||
|
with pytest.raises(ValueError, match="Multidimensional features with a zero-width dimension"):
|
||||||
|
get_hf_features_from_features(features)
|
||||||
|
|
||||||
|
|
||||||
# ── clear / lifecycle ────────────────────────────────────────────────
|
# ── clear / lifecycle ────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -26,17 +26,8 @@ import tempfile
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
import torch
|
|
||||||
from safetensors.torch import save_file
|
|
||||||
|
|
||||||
from lerobot.configs import PipelineFeatureType, PolicyFeature
|
from lerobot.processor.pipeline import DataProcessorPipeline, ProcessorMigrationError
|
||||||
from lerobot.processor.pipeline import (
|
|
||||||
DataProcessorPipeline,
|
|
||||||
ProcessorMigrationError,
|
|
||||||
ProcessorStep,
|
|
||||||
ProcessorStepRegistry,
|
|
||||||
)
|
|
||||||
from lerobot.types import EnvTransition
|
|
||||||
|
|
||||||
# Simplified Config Loading Tests
|
# Simplified Config Loading Tests
|
||||||
|
|
||||||
@@ -107,140 +98,6 @@ def test_load_config_nonexistent_path_tries_hub():
|
|||||||
DataProcessorPipeline._load_config("nonexistent/path", "processor.json", {})
|
DataProcessorPipeline._load_config("nonexistent/path", "processor.json", {})
|
||||||
|
|
||||||
|
|
||||||
def test_from_pretrained_local_directory_missing_state_does_not_call_hub(monkeypatch):
|
|
||||||
"""Local processor dirs must fail locally when a state file is missing."""
|
|
||||||
|
|
||||||
@ProcessorStepRegistry.register("local_missing_state_step")
|
|
||||||
class LocalMissingStateStep(ProcessorStep):
|
|
||||||
def __call__(self, transition: EnvTransition) -> EnvTransition:
|
|
||||||
return transition
|
|
||||||
|
|
||||||
def transform_features(
|
|
||||||
self, features: dict[PipelineFeatureType, dict[str, PolicyFeature]]
|
|
||||||
) -> dict[PipelineFeatureType, dict[str, PolicyFeature]]:
|
|
||||||
return features
|
|
||||||
|
|
||||||
def load_state_dict(self, state: dict[str, torch.Tensor]) -> None:
|
|
||||||
pass
|
|
||||||
|
|
||||||
try:
|
|
||||||
with tempfile.TemporaryDirectory() as tmp_dir:
|
|
||||||
tmp_path = Path(tmp_dir)
|
|
||||||
config = {
|
|
||||||
"name": "LocalMissingStatePipeline",
|
|
||||||
"steps": [{"registry_name": "local_missing_state_step", "state_file": "missing.safetensors"}],
|
|
||||||
}
|
|
||||||
(tmp_path / "processor.json").write_text(json.dumps(config))
|
|
||||||
|
|
||||||
def fail_hub_download(*args, **kwargs):
|
|
||||||
pytest.fail("local missing processor state should not call hf_hub_download")
|
|
||||||
|
|
||||||
monkeypatch.setattr("lerobot.processor.pipeline.hf_hub_download", fail_hub_download)
|
|
||||||
|
|
||||||
with pytest.raises(FileNotFoundError, match="missing.safetensors.*local processor pipeline"):
|
|
||||||
DataProcessorPipeline.from_pretrained(tmp_path, config_filename="processor.json")
|
|
||||||
finally:
|
|
||||||
ProcessorStepRegistry.unregister("local_missing_state_step")
|
|
||||||
|
|
||||||
|
|
||||||
def test_from_pretrained_local_config_file_missing_state_does_not_call_hub(monkeypatch):
|
|
||||||
"""Local single-file processor configs must also keep missing state resolution local."""
|
|
||||||
|
|
||||||
@ProcessorStepRegistry.register("local_file_missing_state_step")
|
|
||||||
class LocalFileMissingStateStep(ProcessorStep):
|
|
||||||
def __call__(self, transition: EnvTransition) -> EnvTransition:
|
|
||||||
return transition
|
|
||||||
|
|
||||||
def transform_features(
|
|
||||||
self, features: dict[PipelineFeatureType, dict[str, PolicyFeature]]
|
|
||||||
) -> dict[PipelineFeatureType, dict[str, PolicyFeature]]:
|
|
||||||
return features
|
|
||||||
|
|
||||||
def load_state_dict(self, state: dict[str, torch.Tensor]) -> None:
|
|
||||||
pass
|
|
||||||
|
|
||||||
try:
|
|
||||||
with tempfile.TemporaryDirectory() as tmp_dir:
|
|
||||||
tmp_path = Path(tmp_dir)
|
|
||||||
config_path = tmp_path / "processor.json"
|
|
||||||
config = {
|
|
||||||
"name": "LocalFileMissingStatePipeline",
|
|
||||||
"steps": [
|
|
||||||
{"registry_name": "local_file_missing_state_step", "state_file": "missing.safetensors"}
|
|
||||||
],
|
|
||||||
}
|
|
||||||
config_path.write_text(json.dumps(config))
|
|
||||||
|
|
||||||
def fail_hub_download(*args, **kwargs):
|
|
||||||
pytest.fail("local missing processor state should not call hf_hub_download")
|
|
||||||
|
|
||||||
monkeypatch.setattr("lerobot.processor.pipeline.hf_hub_download", fail_hub_download)
|
|
||||||
|
|
||||||
with pytest.raises(FileNotFoundError, match="missing.safetensors.*local processor pipeline"):
|
|
||||||
DataProcessorPipeline.from_pretrained(config_path, config_filename="ignored.json")
|
|
||||||
finally:
|
|
||||||
ProcessorStepRegistry.unregister("local_file_missing_state_step")
|
|
||||||
|
|
||||||
|
|
||||||
def test_from_pretrained_hub_source_missing_local_state_still_calls_hub(monkeypatch, tmp_path):
|
|
||||||
"""Hub sources still fall back to hf_hub_download for state files."""
|
|
||||||
|
|
||||||
@ProcessorStepRegistry.register("hub_state_step")
|
|
||||||
class HubStateStep(ProcessorStep):
|
|
||||||
def __init__(self):
|
|
||||||
self.value = torch.tensor(0)
|
|
||||||
|
|
||||||
def __call__(self, transition: EnvTransition) -> EnvTransition:
|
|
||||||
return transition
|
|
||||||
|
|
||||||
def transform_features(
|
|
||||||
self, features: dict[PipelineFeatureType, dict[str, PolicyFeature]]
|
|
||||||
) -> dict[PipelineFeatureType, dict[str, PolicyFeature]]:
|
|
||||||
return features
|
|
||||||
|
|
||||||
def load_state_dict(self, state: dict[str, torch.Tensor]) -> None:
|
|
||||||
self.value = state["value"]
|
|
||||||
|
|
||||||
try:
|
|
||||||
state_path = tmp_path / "downloaded.safetensors"
|
|
||||||
save_file({"value": torch.tensor(7)}, state_path)
|
|
||||||
loaded_config = {
|
|
||||||
"name": "HubStatePipeline",
|
|
||||||
"steps": [{"registry_name": "hub_state_step", "state_file": "hub_state.safetensors"}],
|
|
||||||
}
|
|
||||||
calls = []
|
|
||||||
|
|
||||||
def fake_load_config(cls, model_id, config_filename, hub_download_kwargs):
|
|
||||||
return loaded_config, tmp_path / "hub_cache"
|
|
||||||
|
|
||||||
def fake_hub_download(**kwargs):
|
|
||||||
calls.append(kwargs)
|
|
||||||
return str(state_path)
|
|
||||||
|
|
||||||
monkeypatch.setattr(DataProcessorPipeline, "_load_config", classmethod(fake_load_config))
|
|
||||||
monkeypatch.setattr("lerobot.processor.pipeline.hf_hub_download", fake_hub_download)
|
|
||||||
|
|
||||||
pipeline = DataProcessorPipeline.from_pretrained("user/repo", config_filename="processor.json")
|
|
||||||
|
|
||||||
assert calls == [
|
|
||||||
{
|
|
||||||
"repo_id": "user/repo",
|
|
||||||
"filename": "hub_state.safetensors",
|
|
||||||
"repo_type": "model",
|
|
||||||
"force_download": False,
|
|
||||||
"resume_download": None,
|
|
||||||
"proxies": None,
|
|
||||||
"token": None,
|
|
||||||
"cache_dir": None,
|
|
||||||
"local_files_only": False,
|
|
||||||
"revision": None,
|
|
||||||
}
|
|
||||||
]
|
|
||||||
assert pipeline.steps[0].value.item() == 7
|
|
||||||
finally:
|
|
||||||
ProcessorStepRegistry.unregister("hub_state_step")
|
|
||||||
|
|
||||||
|
|
||||||
# Config Validation Tests
|
# Config Validation Tests
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -17,8 +17,6 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import dataclasses
|
import dataclasses
|
||||||
import sys
|
|
||||||
from types import SimpleNamespace
|
|
||||||
from unittest.mock import MagicMock
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -108,109 +106,6 @@ def test_sentry_config_defaults():
|
|||||||
assert cfg.target_video_file_size_mb is None
|
assert cfg.target_video_file_size_mb is None
|
||||||
|
|
||||||
|
|
||||||
def test_rollout_config_passes_policy_pretrained_revision(monkeypatch):
|
|
||||||
from lerobot.configs import PreTrainedConfig, parser
|
|
||||||
from lerobot.rollout import RolloutConfig
|
|
||||||
from tests.mocks.mock_robot import MockRobotConfig
|
|
||||||
|
|
||||||
captured = {}
|
|
||||||
|
|
||||||
def fake_from_pretrained(cls, pretrained_name_or_path, **kwargs):
|
|
||||||
captured["pretrained_name_or_path"] = pretrained_name_or_path
|
|
||||||
captured.update(kwargs)
|
|
||||||
return SimpleNamespace(device="cpu", pretrained_revision=kwargs["revision"])
|
|
||||||
|
|
||||||
monkeypatch.setattr(parser, "get_yaml_overrides", lambda _: ["--pretrained_revision=yaml-sha"])
|
|
||||||
monkeypatch.setattr(
|
|
||||||
sys,
|
|
||||||
"argv",
|
|
||||||
["lerobot-rollout", "--policy.path=user/policy", "--policy.pretrained_revision=cli-sha"],
|
|
||||||
)
|
|
||||||
monkeypatch.setattr(PreTrainedConfig, "from_pretrained", classmethod(fake_from_pretrained))
|
|
||||||
|
|
||||||
cfg = RolloutConfig(robot=MockRobotConfig())
|
|
||||||
|
|
||||||
assert captured["pretrained_name_or_path"] == "user/policy"
|
|
||||||
assert captured["revision"] == "cli-sha"
|
|
||||||
assert captured["cli_overrides"] == [
|
|
||||||
"--pretrained_revision=yaml-sha",
|
|
||||||
"--pretrained_revision=cli-sha",
|
|
||||||
]
|
|
||||||
assert cfg.policy.pretrained_path == "user/policy"
|
|
||||||
assert cfg.policy.pretrained_revision == "cli-sha"
|
|
||||||
|
|
||||||
|
|
||||||
def test_load_pretrained_policy_passes_revision(monkeypatch):
|
|
||||||
import lerobot.rollout.context as rollout_context
|
|
||||||
|
|
||||||
policy_config = SimpleNamespace(
|
|
||||||
type="mock",
|
|
||||||
use_peft=False,
|
|
||||||
pretrained_path="user/policy",
|
|
||||||
pretrained_revision="policy-sha",
|
|
||||||
)
|
|
||||||
policy_class = MagicMock()
|
|
||||||
loaded_policy = MagicMock()
|
|
||||||
policy_class.from_pretrained.return_value = loaded_policy
|
|
||||||
monkeypatch.setattr(rollout_context, "get_policy_class", lambda _: policy_class)
|
|
||||||
|
|
||||||
policy = rollout_context._load_pretrained_policy(policy_config)
|
|
||||||
|
|
||||||
assert policy is loaded_policy
|
|
||||||
policy_class.from_pretrained.assert_called_once_with(
|
|
||||||
"user/policy",
|
|
||||||
config=policy_config,
|
|
||||||
revision="policy-sha",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_load_pretrained_peft_policy_keeps_adapter_and_base_revisions_separate(monkeypatch):
|
|
||||||
import lerobot.rollout.context as rollout_context
|
|
||||||
|
|
||||||
policy_config = SimpleNamespace(
|
|
||||||
type="mock",
|
|
||||||
use_peft=True,
|
|
||||||
pretrained_path="user/adapter",
|
|
||||||
pretrained_revision="adapter-sha",
|
|
||||||
)
|
|
||||||
policy_class = MagicMock()
|
|
||||||
base_policy = MagicMock()
|
|
||||||
policy_class.from_pretrained.return_value = base_policy
|
|
||||||
monkeypatch.setattr(rollout_context, "get_policy_class", lambda _: policy_class)
|
|
||||||
|
|
||||||
peft_config = SimpleNamespace(
|
|
||||||
base_model_name_or_path="user/base-policy",
|
|
||||||
revision="base-sha",
|
|
||||||
)
|
|
||||||
peft_config_from_pretrained = MagicMock(return_value=peft_config)
|
|
||||||
adapted_policy = MagicMock()
|
|
||||||
peft_model_from_pretrained = MagicMock(return_value=adapted_policy)
|
|
||||||
monkeypatch.setitem(
|
|
||||||
sys.modules,
|
|
||||||
"peft",
|
|
||||||
SimpleNamespace(
|
|
||||||
PeftConfig=SimpleNamespace(from_pretrained=peft_config_from_pretrained),
|
|
||||||
PeftModel=SimpleNamespace(from_pretrained=peft_model_from_pretrained),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
policy = rollout_context._load_pretrained_policy(policy_config)
|
|
||||||
|
|
||||||
assert policy is adapted_policy
|
|
||||||
peft_config_from_pretrained.assert_called_once_with("user/adapter", revision="adapter-sha")
|
|
||||||
policy_class.from_pretrained.assert_called_once_with(
|
|
||||||
pretrained_name_or_path="user/base-policy",
|
|
||||||
config=policy_config,
|
|
||||||
revision="base-sha",
|
|
||||||
)
|
|
||||||
peft_model_from_pretrained.assert_called_once_with(
|
|
||||||
base_policy,
|
|
||||||
"user/adapter",
|
|
||||||
config=peft_config,
|
|
||||||
revision="adapter-sha",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# RolloutRingBuffer
|
# RolloutRingBuffer
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
Reference in New Issue
Block a user