Compare commits

...

3 Commits

Author SHA1 Message Date
Nikodem Bartnik cc13858aff Create DOCS_REDESIGN.md 2026-06-22 09:20:57 +02:00
Nikodem Bartnik a24d10f5bb add quick AI draft for quickstart 2026-05-26 13:10:24 +02:00
Nikodem Bartnik 32279544ea add new docs chapters structure 2026-05-26 12:01:33 +02:00
4 changed files with 963 additions and 126 deletions
+404
View File
@@ -0,0 +1,404 @@
# LeRobot Documentation Redesign — Proposal
**Status:** Draft for maintainer review · **Author:** Dev Rel · **Date:** June 2026
**Scope:** Information architecture + UX redesign of https://huggingface.co/docs/lerobot (source: `docs/source/`)
---
## 1. Executive summary
**The problem, in five points:**
1. **No guided beginner path.** "Get started" is three pages (landing, install, CLI cheat-sheet). After that, a newcomer lands in a "Tutorials" section that mixes the core 652-line SO-101 walkthrough (`il_robots.mdx`) with RL (950-line `hilserl.mdx`), PEFT, multi-GPU, and rename-map content — four different audiences in one flat list.
2. **Zero decision guidance.** 11 robot pages, 12 policy pages, 10 benchmark pages — all flat alphabetical-ish lists. Nothing answers "which robot should I buy?", "which policy after ACT?", "which benchmark fits my claim?".
3. **The best beginner content isn't on the docs site.** `AGENT_GUIDE.md` (repo root) contains the most complete procedural guidance we have — data-collection golden rules, policy selection by VRAM tier, training-duration heuristics, eval success-rate targets — and none of it is published.
4. **Structural debt.** 14 orphan `policy_*_README.md` files sit in `docs/source/` unreferenced by the toctree (verbatim hand-copies of `src/lerobot/policies/*/README.md`); 6 of 17 CLI commands are completely undocumented; 1 of 15 teleoperators has a doc page; there is no troubleshooting page and no `_redirects.yml`.
5. **Pages do too many jobs.** Tutorial, how-to, and reference content is interleaved (e.g., `lerobot-dataset-v3.mdx` is simultaneously a format spec, a Python API guide, and a migration guide), which makes every page long and none of them skimmable.
**The proposal, in five points:**
1. **A numbered "Learn: your first robot" course** (FastAPI-style): 11 small CLI-only pages taking an SO-101 owner from install to a trained, evaluated ACT policy. Each page ends with a checkpoint and a "Next" link. A one-page no-hardware quickstart runs in parallel.
2. **A sidebar that reads top-to-bottom as increasing expertise**, organized into 17 sections: Get started → task-oriented how-tos (Collect data / Train / Deploy & evaluate) → hardware catalogs → policy catalogs → concepts → RL → extending → reference → community.
3. **Every catalog section opens with a comparison/decision page**: Choosing a robot, Teleoperator overview, Choosing a policy, Reward models overview, Choosing a benchmark.
4. **Publish the missing content**: AGENT_GUIDE.md's tips become real docs pages; a full CLI reference covers all 17 commands; a symptom-organized troubleshooting page aggregates fixes currently buried across five pages.
5. **Minimal URL breakage**: existing slugs are kept wherever possible. Only **2 redirects** are needed (`cheat-sheet → cli_reference`, `il_robots → learn/index`); everything else is a toctree reposition.
**Effort estimate:** ~45 person-weeks across 5 phases; phases 3 and 5 are parallelizable and community-friendly (see §9).
---
## 2. Current-state audit
### 2.1 By the numbers
| Metric | Value |
|---|---|
| Files in `docs/source/` (excl. `_toctree.yml`) | 90 |
| Pages in the toctree | 76 |
| Orphan files (on disk, not in toctree) | 14 (`policy_*_README.md`) |
| Top-level sidebar sections | 15, all flat (no nesting) |
| Longest pages | `hilserl.mdx` ~950 lines · `il_robots.mdx` ~650 · `so100.mdx` ~640 |
| CLI commands in `pyproject.toml [project.scripts]` | 17 — **6 with zero doc presence** (`lerobot-find-cameras`, `lerobot-setup-can`, `lerobot-find-joint-limits`, `lerobot-train-tokenizer`, `lerobot-imgtransform-viz`, `lerobot-info`) |
| Teleoperators in `src/lerobot/teleoperators/` | 15 — **1 documented** (`phone_teleop.mdx`); keyboard, gamepad, leader arms covered only implicitly inside robot/tutorial pages |
| Camera backends | 4 — `zmq` undocumented, `reachy2_camera` implicit |
| Policies in `src/lerobot/policies/` | 19 — `diffusion`, `tdmpc`, `vqbet` have orphan READMEs but **no doc page**; `sac`, `gaussian_actor` undocumented |
| Redirect infrastructure | None (`_redirects.yml` does not exist) |
### 2.2 Prioritized UX problems
**Critical (blocks the main funnel):**
| # | Problem | Evidence |
|---|---|---|
| C1 | No linear beginner path: install → ??? | "Get started" = `index.mdx` (brand copy) + `installation.mdx` + `cheat-sheet.mdx`. The actual path lives inside `il_robots.mdx`, filed under "Tutorials" with 9 unrelated pages. |
| C2 | No "which robot do I buy?" guidance | 11 robot pages, no comparison. SO-101 vs SO-100 relationship is never stated; SO-100's page is *longer* than SO-101's. |
| C3 | `il_robots.mdx` monolith | One 652-line page covers calibrate + teleoperate + cameras + record + train + evaluate, doubling every step in CLI *and* Python, all SO-101-specific despite the generic title. |
| C4 | Best procedural content unpublished | `AGENT_GUIDE.md` §5 (data tips), §6 (policy choice), §7 (training duration), §8 (eval targets) exist only in the repo root. |
| C5 | No hardware-free on-ramp | No "train on a Hub dataset in 15 minutes" page; `notebooks.mdx` is 30 lines of links. |
**High (breaks common tasks):**
| # | Problem | Evidence |
|---|---|---|
| H1 | Training guidance scattered | Spread across `act.mdx`, `hardware_guide.mdx`, `multi_gpu_training.mdx`, `il_robots.mdx` — no single "how to train" page. |
| H2 | No policy comparison/decision page | 12 policy pages; ACT says "recommended first" but nothing maps VRAM/task/data-size → policy. |
| H3 | Recording how-to fused with format spec | `il_robots.mdx` (procedure) vs `lerobot-dataset-v3.mdx` (spec + Python API + migration) — neither is a clean reference or a clean guide. |
| H4 | No benchmark selection guidance | 10 benchmark pages in a flat list; "Adding a New Benchmark" (contributor content) sits *first*. |
| H5 | No troubleshooting page | Fixes are buried in `installation.mdx`, `il_robots.mdx`, `lerobot-dataset-v3.mdx` "Common Issues", and AGENT_GUIDE §5.8. |
| H6 | Teleoperators near-invisible | 15 implementations, 1 page. Keyboard/gamepad (`--teleop.type=keyboard|gamepad`) have no docs at all. |
**Medium / low (debt and polish):**
| # | Problem | Evidence |
|---|---|---|
| M1 | 14 orphan `policy_*_README.md` in `docs/source/` | Verbatim copies of `src/lerobot/policies/*/README.md`; not in toctree, not built; no sync script exists (verified by repo-wide grep) — hand-copied and already drifting. |
| M2 | 6/17 CLI commands undocumented | See §2.1. `cheat-sheet.mdx` covers only the main workflow commands. |
| M3 | Mixed-audience sections | "Tutorials" holds beginner + RL-researcher + contributor content; "Benchmarks" holds user + contributor content. |
| M4 | Naming inconsistencies | "SO-101"/"SO101"/"so101" in prose; `<Tip>` vs `> [!NOTE]` admonitions mixed; hyphen vs underscore slugs (`lerobot-dataset-v3` vs `multi_gpu_training`). |
| M5 | Section opener pages missing | "Robot Processors" starts at "Introduction to Robot Processors" (good) but Robots/Policies/Benchmarks/Teleoperators start with an arbitrary instance page. |
---
## 3. What great docs do (and what we borrow)
**Diátaxis** (diataxis.fr — adopted by Canonical, Cloudflare, Gatsby): documentation serves four distinct needs that should never share a page — *tutorials* (learning by doing), *how-to guides* (task recipes), *reference* (lookup facts), *explanation* (concepts). LeRobot's biggest pages fail precisely because they fuse three of these. → **We borrow:** the four-way separation as the organizing principle of the new sidebar; splits like `lerobot-dataset-v3` (spec) vs `using_lerobot_dataset` (how-to).
**FastAPI**: famous for a numbered, linear tutorial where each chapter is small, runnable, and builds on the previous one; advanced topics live in a separate section so the learn-path is never interrupted. → **We borrow:** the numbered `learn/` course with one outcome per page and a "Next:" footer; advanced content (RL, multi-GPU) moved out of the beginner's way.
**Ultralytics YOLO**: handles a many-models × many-tasks × many-modes matrix with *orthogonal* navigation axes plus comparison pages, instead of forcing one hierarchy. → **We borrow:** Robots, Teleoperators, Policies, and Benchmarks as separate catalog sections, each opening with a comparison table ("Choosing a…" pages).
**HF Transformers** (same doc-builder tooling — everything it does, we can do): card-grid landing page, `<hfoptions>` tabs with **persistent selection** (pick PyTorch once, every code block follows), nested collapsible toctree sections, subdirectory slugs (`model_doc/bert`), `_redirects.yml`. → **We borrow:** card-based `index`, site-wide `<hfoptions id="robot">` / `id="os"` / `id="train_env"` tabs, a collapsible nested course section, and the redirects file.
---
## 4. Proposed information architecture
**17 sections, ~101 toctree pages** (today: 15 sections, 76 pages — growth is almost entirely the 11 deliberately-small course pages; the 14 orphans are absorbed into 2 pages). The sidebar reads top-to-bottom as increasing expertise. Slugs in parentheses; annotations: **[NEW]** to be written · **[KEPT]** same file/slug · **[MOVED]** same file, new section · **[SPLIT from X]** · **[MERGED from X+Y]**.
```
Get started
├─ LeRobot (index) ........................................ [KEPT — rewritten as card-grid landing]
├─ Installation (installation) ............................ [KEPT — full install matrix, reference]
├─ Learn: your first robot — SO-101 course (nested, collapsible)
│ ├─ Welcome: what you'll build (learn/index) ............ [NEW]
│ ├─ 1. Install LeRobot (learn/install) .................. [SPLIT from installation — happy path only]
│ ├─ 2. Assemble your SO-101 (learn/assemble) ............ [SPLIT from so101]
│ ├─ 3. Set up the motors (learn/motors) ................. [SPLIT from so101]
│ ├─ 4. Calibrate (learn/calibrate) ...................... [SPLIT from so101 + il_robots]
│ ├─ 5. Teleoperate your arms (learn/teleoperate) ........ [SPLIT from il_robots]
│ ├─ 6. Connect your cameras (learn/cameras) ............. [SPLIT from cameras + il_robots]
│ ├─ 7. Record a dataset (learn/record) .................. [SPLIT from il_robots + AGENT_GUIDE §5.7]
│ ├─ 8. Train your first policy (learn/train) ............ [SPLIT from il_robots + AGENT_GUIDE §7]
│ ├─ 9. Evaluate your policy (learn/evaluate) ............ [SPLIT from il_robots + AGENT_GUIDE §8]
│ └─ 10. Next steps (learn/next_steps) ................... [NEW]
├─ Quickstart without a robot (quickstart_no_robot) ....... [NEW — AGENT_GUIDE §3 Path B + notebooks]
└─ Troubleshooting (troubleshooting) ...................... [NEW — aggregated from 5 pages + AGENT_GUIDE §5.8]
Collect data
├─ Record a dataset (record_dataset) ...................... [SPLIT from il_robots — robot-agnostic, CLI+Python]
├─ Get high-quality data (data_collection_tips) ........... [NEW — AGENT_GUIDE §5]
├─ Visualize & replay episodes (visualize_replay) ......... [SPLIT from il_robots + using_dataset_tools]
├─ Edit datasets (using_dataset_tools) .................... [KEPT — retitled]
├─ Port large datasets to v3 (porting_datasets_v3) ........ [KEPT]
└─ Add language instructions (language_and_recipes) ....... [KEPT — retitled]
Train policies
├─ Train a policy (train_policy) .......................... [SPLIT from il_robots + AGENT_GUIDE §7; local/Colab/HF-Jobs tabs]
├─ Compute requirements (hardware_guide) .................. [KEPT — retitled]
├─ Multi-GPU training (multi_gpu_training) ................ [MOVED]
├─ Fine-tune with PEFT / LoRA (peft_training) ............. [MOVED]
├─ PyTorch accelerators (torch_accelerators) .............. [KEPT]
└─ Rename map & empty cameras (rename_map) ................ [MOVED]
Deploy & evaluate
├─ Evaluate a policy on your robot (evaluate_policy) ...... [SPLIT from il_robots + AGENT_GUIDE §8]
├─ Deploy a policy — lerobot-rollout (inference) .......... [KEPT]
├─ Async inference (async) ................................ [KEPT]
└─ Real-time chunking — RTC (rtc) ......................... [KEPT — absorbs policy_rtc_README.md]
Robots
├─ Choosing a robot (choose_a_robot) ...................... [NEW — comparison table + decision guidance]
├─ SO-101 (so101) ......................................... [SPLIT — assembly/motors/calibration → course; page becomes spec + sourcing + links]
├─ SO-100 — previous generation (so100) ................... [KEPT + legacy banner]
├─ Koch v1.1 (koch) · LeKiwi (lekiwi) · Hope Jr (hope_jr) · Reachy 2 (reachy2)
│ Unitree G1 (unitree_g1) · Earth Rover Mini (earthrover_mini_plus)
│ OMX (omx) · OpenArm (openarm) · reBot B601-DM (rebot_b601) ... [all KEPT, normalized to robot template]
Teleoperators
├─ Teleoperator overview (teleoperators_overview) ......... [NEW — pairing matrix for all 15; keyboard/gamepad usage]
└─ Phone teleoperation (phone_teleop) ..................... [KEPT]
Cameras & motors
├─ Cameras (cameras) ...................................... [KEPT — expanded: ZMQ + Reachy 2 backends]
├─ Feetech motors & firmware (feetech) .................... [MOVED]
└─ Damiao motors & CAN bus (damiao) ....................... [MOVED — documents lerobot-setup-can]
Policies
├─ Choosing a policy (choose_a_policy) .................... [NEW — AGENT_GUIDE §6: VRAM tiers, decision rules]
├─ ACT (act) .............................................. [KEPT — absorbs policy_act_README.md]
├─ Diffusion Policy (diffusion) ........................... [NEW — from policy_diffusion_README.md; no page exists today]
├─ SmolVLA (smolvla) · π₀ (pi0) · π₀-FAST (pi0fast) · π₀.₅ (pi05)
│ GR00T N1.5 (groot) · MolmoAct2 (molmoact2) · VLA-JEPA (vla_jepa) · EO-1 (eo1)
│ X-VLA (xvla) · Multitask DiT (multi_task_dit) · WALL-OSS (walloss) ... [all KEPT, absorb matching READMEs, normalized to policy template]
└─ Legacy policies — VQ-BeT, TDMPC (legacy_policies) ...... [NEW — MERGED from policy_vqbet_README + policy_tdmpc_README]
Reward models
├─ Reward models overview (reward_models_overview) ........ [NEW]
└─ SARM (sarm) · ROBOMETER (robometer) · TOPReward (topreward) ... [KEPT]
Datasets in depth
├─ LeRobotDataset format — v3 spec (lerobot-dataset-v3) ... [SPLIT — keeps slug; format/layout/migration only]
├─ Load & stream datasets in Python (using_lerobot_dataset) [SPLIT from lerobot-dataset-v3 — Python API, streaming, transforms]
├─ Tool-calling columns (tools) ........................... [KEPT — retitled]
├─ Video encoding parameters (video_encoding_parameters) .. [KEPT]
└─ Streaming video encoding (streaming_video_encoding) .... [KEPT]
Simulation
├─ Environments from the Hub (envhub) ..................... [KEPT — section opener]
├─ LeIsaac: control & train in Isaac Sim (envhub_leisaac) . [KEPT]
└─ NVIDIA IsaacLab Arena (envhub_isaaclab_arena) .......... [MOVED from Benchmarks]
Benchmarks
├─ Choosing a benchmark (choose_a_benchmark) .............. [NEW — comparison + match-benchmark-to-claim guidance]
└─ LIBERO (libero) · LIBERO-plus (libero_plus) · Meta-World (metaworld)
RoboTwin 2.0 (robotwin) · RoboCasa365 (robocasa) · RoboCerebra (robocerebra)
RoboMME (robomme) · VLABench (vlabench) ................ [all KEPT]
Processors
├─ What are processors? (introduction_processors) ......... [KEPT — entry-level rewrite + diagram]
├─ Processors for robots & teleops (processors_robots_teleop) [KEPT]
├─ Environment processors (env_processor) ................. [KEPT]
├─ Action representations (action_representations) ........ [KEPT]
└─ Debug a processor pipeline (debug_processor_pipeline) .. [KEPT]
Reinforcement learning
├─ Train with RL on a real robot — HIL-SERL (hilserl) ..... [MOVED — split candidate, see §10 Q7]
├─ Train RL in simulation (hilserl_sim) ................... [MOVED]
└─ Human-in-the-loop data collection (hil_data_collection) [MOVED]
Extending LeRobot
├─ Add a policy (bring_your_own_policies) ................. [MOVED]
├─ Add a robot (integrate_hardware) ....................... [MOVED — retitled]
├─ Write your own processor (implement_your_own_processor) [MOVED]
└─ Add a benchmark (adding_benchmarks) .................... [MOVED]
Reference & resources
├─ CLI reference (cli_reference) .......................... [MERGED from cheat-sheet + NEW content for 6 undocumented commands]
├─ LeLab: browser GUI (lelab) ............................. [MOVED — placement is open question Q5]
├─ Notebooks (notebooks) .................................. [MOVED]
└─ Backward compatibility (backwardcomp) .................. [MOVED]
Community
└─ Contribute to LeRobot (contributing) ................... [KEPT]
```
**`_redirects.yml`** (complete file — only two entries):
```yaml
cheat-sheet: cli_reference
il_robots: learn/index
```
Everything else keeps its slug; toctree moves and retitles don't change URLs in doc-builder.
---
## 5. The beginner course, page by page
**Template for every course page:** goal sentence → "You need" list → numbered steps, each ending with a verifiable checkpoint ("you should see…") → mini-troubleshooting (≤3 most common failures) → "Next:" footer link. **CLI-only — Python appears nowhere in the course.** OS-specific commands use `<hfoptions id="os">`. Budget ≤200 lines per page (assembly exempt: media-heavy).
| # | Slug | Title | Goal — the user can… | Content source | Length | Next |
|---|---|---|---|---|---|---|
| 0 | `learn/index` | Welcome: what you'll build | See the outcome (a trained ACT policy moving a real SO-101), the shopping list (~$200 kit, 2 cameras, GPU or Colab), time budget (~1 weekend) | NEW; framing from AGENT_GUIDE §23 | ~120 | 1 |
| 1 | `learn/install` | Install LeRobot | Working env via one happy path (`pip install 'lerobot[feetech]'`), verified with `lerobot-info` | `installation.mdx` steps 13, trimmed to one path | ~100 | 2 |
| 2 | `learn/assemble` | Assemble your SO-101 | Source/print parts, assemble leader + follower | `so101.mdx` sourcing + assembly sections (videos carry the load) | ~400 | 3 |
| 3 | `learn/motors` | Set up the motors | Find ports (`lerobot-find-port`), set IDs (`lerobot-setup-motors`) for both arms | `so101.mdx` motor config + `cheat-sheet.mdx` | ~150 | 4 |
| 4 | `learn/calibrate` | Calibrate | Run `lerobot-calibrate` on both arms; know where calibration files live | `so101.mdx` + `il_robots.mdx` calibration sections | ~120 | 5 |
| 5 | `learn/teleoperate` | Teleoperate your arms | Run `lerobot-teleoperate` — the first "wow" moment, deliberately **before** cameras | `il_robots.mdx` teleop (CLI only) | ~80 | 6 |
| 6 | `learn/cameras` | Connect your cameras | Detect with `lerobot-find-cameras`, position wrist + front cams, teleop with camera view | `cameras.mdx` + `il_robots.mdx` + AGENT_GUIDE §5.1 placement tips | ~120 | 7 |
| 7 | `learn/record` | Record a dataset | Record 50 episodes of one task with `lerobot-record`; keyboard controls, resume, Hub push | `il_robots.mdx` record + AGENT_GUIDE §5.7 defaults ("50 episodes, one task, fixed camera") + §5.2 | ~180 | 8 |
| 8 | `learn/train` | Train your first policy (ACT) | Launch `lerobot-train` with sane ACT defaults; know how long to wait and when to stop. Tabs: local GPU / Colab / HF Jobs (`<hfoptions id="train_env">`) | `il_robots.mdx` train + AGENT_GUIDE §7.1, §7.3, §7.7 | ~150 | 9 |
| 9 | `learn/evaluate` | Evaluate your policy | Run the policy, measure success over 10 trials, know what's "good" | `il_robots.mdx` eval + AGENT_GUIDE §8.1, §8.3, §5.8 | ~120 | 10 |
| 10 | `learn/next_steps` | Next steps | Pick a direction: better data → `data_collection_tips`; language/multi-task → `smolvla`; mobile → `lekiwi`; RL → `hilserl`; sim → `envhub`; Discord | NEW; card grid | ~60 | — |
**Parallel track — `quickstart_no_robot`:** one page mirroring steps 79 without hardware: pick a Hub dataset → `lerobot-dataset-viz` → train ACT (Colab button + local command) → evaluate in sim. Ends with "got a robot? start the course." Source: AGENT_GUIDE §3 Path B + `notebooks.mdx`. (Blessed dataset/env to be decided — §10 Q8.)
---
## 6. New pages — content briefs
**Hub / decision pages:**
- **`choose_a_robot`** — Comparison table of all 11 robots: photo, type (arm / mobile / humanoid / hand), DoF, motor family (Feetech / Dynamixel / Damiao), approx. price, sourcing (kit / 3D-print / buy), teleop options, docs maturity. Below: short decision paragraphs ("first robot → SO-101", "mobile manipulation → LeKiwi", "research platform → Reachy 2 / G1"). Ends with a card into the course.
- **`teleoperators_overview`** — Pairing matrix: all 15 teleoperators × compatible robots, with the exact `--teleop.type=` value for each. Inline usage sections for **keyboard** and **gamepad** (currently undocumented); leader arms link to their robot pages; phone links to `phone_teleop`.
- **`choose_a_policy`** — Table from AGENT_GUIDE §6.1: policy, params, min VRAM tier, single/multi-task, language-conditioned, inference speed, pretrained checkpoints, dataset-size sweet spot. Decision rules from §6.2 ("first policy → ACT; language + 16 GB → SmolVLA; ≥40 GB → π₀ / GR00T"). Legacy policies in a collapsed footnote.
- **`reward_models_overview`** — Half page: what reward models are for (success classification for RL, episode filtering, eval scoring), where they plug into HIL-SERL, 3-row table (SARM / ROBOMETER / TOPReward: input modality, output, use case).
- **`choose_a_benchmark`** — Table: benchmark, sim engine, # tasks, focus (long-horizon / language / generalization / bimanual), policies with reported results, GPU needs, Dockerfile availability (AGENT_GUIDE §8.2b). Guidance: match the benchmark to your policy's claim.
**Getting started:**
- **`index` (rewrite)** — Card grid replacing prose: "Build your first robot (course)" / "Quickstart without a robot" / "Choose a robot" / "Choose a policy" / "Browse datasets & models on the Hub" / "Join Discord". Keep the one-paragraph mission statement.
- **`troubleshooting`** — Symptom-organized, one H2 per failure area: installation (ffmpeg, CUDA) · ports & permissions (find-port fails, udev) · motors (wrong ID, no torque, LED codes from AGENT_GUIDE) · calibration drift · cameras (fps, USB bandwidth) · recording (crashes, resume) · training (flat loss, OOM → compute guide) · policy acts erratically (AGENT_GUIDE §5.8 signal table). Each entry: symptom → cause → fix command.
**How-to consolidations (Diátaxis splits):**
- **`record_dataset`** — Robot-agnostic deep version of course step 7: full `lerobot-record` flag reference, `<hfoptions id="robot">` per-robot commands, Python API in its own H2, Hub upload.
- **`data_collection_tips`** — AGENT_GUIDE §5 nearly verbatim: ergonomics, practice runs, consistency, the "start small" golden rule, troubleshooting signals.
- **`visualize_replay`** — `lerobot-dataset-viz`, the online Hub visualizer, `lerobot-replay`.
- **`train_policy`** — One canonical training how-to: `lerobot-train` anatomy, steps/batch/LR guidance (AGENT_GUIDE §7), resume, checkpoints, W&B, local/Colab/HF-Jobs tabs. Links out to multi-GPU, PEFT, compute guide.
- **`evaluate_policy`** — Real-robot eval protocol (n trials, success criteria, §8.3 targets), `lerobot-eval` for sim, comparing checkpoints.
- **`using_lerobot_dataset`** — Python API split out of `lerobot-dataset-v3`: load from Hub, random access, `delta_timestamps`, DataLoader, streaming, image transforms (+ `lerobot-imgtransform-viz`).
**Reference:**
- **`cli_reference`** — Replaces cheat-sheet. All 17 commands, one H2 each, grouped by workflow: *Setup* (find-port, setup-motors, setup-can, calibrate, find-cameras, info) · *Data* (record, replay, dataset-viz, edit-dataset, imgtransform-viz, find-joint-limits) · *Train* (train, train-tokenizer) · *Deploy* (eval, rollout, teleoperate). Each: one-line purpose, copy-paste SO-101 example, link to the relevant guide.
- **`diffusion`** — Standard policy-template page built from `policy_diffusion_README.md` (the only major policy with no page today).
- **`legacy_policies`** — VQ-BeT + TDMPC on one page with a status banner ("maintained for reproducibility, not recommended for new projects"), minimal train/eval commands, paper links.
---
## 7. Migration table
**Redirects needed: 2.** Orphan files aren't in the toctree (never built/served), so deleting them breaks no URLs. Toctree moves and retitles with unchanged slugs need no redirects.
### 7.1 Current toctree pages (76)
| Old slug | Disposition | New location | Redirect |
|---|---|---|---|
| `index` | rewrite (card grid) | Get started | no |
| `installation` | keep (reference matrix); happy path copied to `learn/install` | Get started | no |
| `cheat-sheet` | **merge** into `cli_reference` | Reference & resources | **yes** |
| `il_robots` | **split 8 ways**`learn/{calibrate,teleoperate,cameras,record,train,evaluate}`, `record_dataset`, `visualize_replay`, `train_policy`, `evaluate_policy` | — | **yes → `learn/index`** |
| `lelab` | move | Reference & resources | no |
| `bring_your_own_policies` | move, retitle "Add a policy" | Extending | no |
| `integrate_hardware` | move, retitle "Add a robot" | Extending | no |
| `hilserl`, `hilserl_sim`, `hil_data_collection` | move | Reinforcement learning | no |
| `multi_gpu_training`, `peft_training`, `torch_accelerators`, `rename_map` | move | Train policies | no |
| `hardware_guide` | move, retitle "Compute requirements" | Train policies | no |
| `lerobot-dataset-v3` | **split**: keeps slug as format spec; Python usage → `using_lerobot_dataset` | Datasets in depth | no |
| `porting_datasets_v3`, `language_and_recipes` | move | Collect data | no |
| `using_dataset_tools` | move, retitle "Edit datasets"; viz section → `visualize_replay` | Collect data | no |
| `tools` | move, retitle "Tool-calling columns" | Datasets in depth | no |
| `video_encoding_parameters`, `streaming_video_encoding` | move | Datasets in depth | no |
| `act`, `smolvla`, `pi0`, `pi0fast`, `pi05`, `molmoact2`, `vla_jepa`, `eo1`, `groot`, `xvla`, `multi_task_dit`, `walloss` | keep; normalize to policy template; absorb matching orphan READMEs | Policies (after `choose_a_policy`) | no |
| `sarm`, `robometer`, `topreward` | keep | Reward models (after overview) | no |
| `inference` | keep, retitle "Deploy a policy (lerobot-rollout)" | Deploy & evaluate | no |
| `async` | keep | Deploy & evaluate | no |
| `rtc` | keep; absorb `policy_rtc_README.md` | Deploy & evaluate | no |
| `envhub`, `envhub_leisaac` | keep | Simulation | no |
| `envhub_isaaclab_arena` | move | Simulation | no |
| `adding_benchmarks` | move, retitle "Add a benchmark" | Extending | no |
| `libero`, `libero_plus`, `metaworld`, `robotwin`, `robocasa`, `robocerebra`, `robomme`, `vlabench` | keep | Benchmarks (after `choose_a_benchmark`) | no |
| `introduction_processors` | keep; entry-level rewrite | Processors | no |
| `processors_robots_teleop`, `env_processor`, `action_representations`, `debug_processor_pipeline` | keep | Processors | no |
| `implement_your_own_processor` | move | Extending | no |
| `so101` | **split**: assembly → `learn/assemble`, motors → `learn/motors`, calibration → `learn/calibrate`; page becomes spec/sourcing/overview linking into the course | Robots | no (slug unchanged) |
| `so100` | keep + legacy banner ("SO-101 is the current generation") | Robots | no |
| `koch`, `lekiwi`, `hope_jr`, `reachy2`, `unitree_g1`, `earthrover_mini_plus`, `omx`, `openarm`, `rebot_b601` | keep; normalize to robot template | Robots (after `choose_a_robot`) | no |
| `phone_teleop` | keep | Teleoperators (after overview) | no |
| `cameras` | keep; expand (ZMQ + Reachy 2 backends) | Cameras & motors | no |
| `feetech`, `damiao` | move; document `lerobot-setup-can` in damiao | Cameras & motors | no |
| `notebooks`, `backwardcomp` | move | Reference & resources | no |
| `contributing` | keep | Community | no |
### 7.2 Orphan files (14 — delete from `docs/source/`, no redirects needed)
Canonical copies remain in `src/lerobot/policies/*/README.md`. Verified: the docs/source copies are byte-identical hand-copies with **no sync script anywhere in the repo** — deletion is safe once unique content is folded in.
| Orphan file | Disposition |
|---|---|
| `policy_act_README.md`, `policy_groot_README.md`, `policy_molmoact2_README.md`, `policy_multi_task_dit_README.md`, `policy_pi0_README.md`, `policy_pi05_README.md`, `policy_sarm_README.md`, `policy_smolvla_README.md`, `policy_vla_jepa_README.md`, `policy_walloss_README.md`, `policy_rtc_README.md` | Delete; fold any unique content into the corresponding `.mdx` page |
| `policy_diffusion_README.md` | Delete; content seeds the **new** `diffusion.mdx` |
| `policy_tdmpc_README.md`, `policy_vqbet_README.md` | Delete; content seeds the **new** `legacy_policies.mdx` |
### 7.3 New files (27)
`learn/index`, `learn/install`, `learn/assemble`, `learn/motors`, `learn/calibrate`, `learn/teleoperate`, `learn/cameras`, `learn/record`, `learn/train`, `learn/evaluate`, `learn/next_steps`, `quickstart_no_robot`, `troubleshooting`, `record_dataset`, `data_collection_tips`, `visualize_replay`, `train_policy`, `evaluate_policy`, `using_lerobot_dataset`, `choose_a_robot`, `teleoperators_overview`, `choose_a_policy`, `diffusion`, `legacy_policies`, `reward_models_overview`, `choose_a_benchmark`, `cli_reference` — plus `_redirects.yml`.
---
## 8. Conventions & style guide
**Naming**
- Product names in prose: **SO-101**, SO-100, LeKiwi, π₀ (display) / `pi0` (slug/code), HIL-SERL, LeRobotDataset. Never "so101"/"SO101" in prose.
- Slugs: lowercase with underscores for new files (`choose_a_robot`); course under `learn/`. Existing hyphenated slugs (`lerobot-dataset-v3`, `cheat-sheet`) are grandfathered, not propagated.
- Titles: sentence case. How-to titles start with a verb ("Record a dataset"); reference titles are nouns ("CLI reference"); course titles are numbered ("3. Set up the motors").
- Placeholders in commands: consistent `<angle_brackets>` (e.g. `--robot.port=<your_port>`).
**Components**
- `<hfoptions>` with **fixed, site-wide ids** so a choice persists across pages: `id="robot"` · `id="os"` · `id="install"` · `id="train_env"` (local/colab/jobs). Tabs only for structurally parallel content; never hide unique content inside a tab.
- `<Tip>` for actionable advice; `<Tip warning={true}>` for anything that can damage hardware, lose data, or cost money. Standardize on `<Tip>` in `.mdx`; `> [!NOTE]` only in plain `.md`. Max ~3 tips per page.
- Card grids only on `index`, `learn/next_steps`, and section overview pages.
- Every assembly/calibration video must have the same steps in text below it (accessibility + searchability).
**Page templates & length budgets**
| Type | Skeleton | Budget |
|---|---|---|
| Course page | goal → "you need" → numbered steps w/ checkpoints → mini-troubleshooting → Next | ≤200 lines (assembly exempt) |
| How-to | problem statement → minimal working command → variations → edge cases → related links | ≤300 |
| Robot page | hero photo → spec table → buy/build → calibration quirks → compatible teleops → known issues | ≤400 |
| Policy page | summary card (params, VRAM, license, paper, checkpoints) → when to use → install → train → finetune → results → citation | ≤250 |
| Concept page | what & why in 3 sentences → diagram → details → pointers to how-tos | ≤400 |
Hard cap **500 lines** for any page → must split (`il_robots` at ~650 and `hilserl` at ~950 are the cautionary examples).
---
## 9. Phased rollout plan
| Phase | Scope | PRs | Effort | Parallel? |
|---|---|---|---|---|
| **0. Sign-off** | This proposal reviewed; open questions (§10) decided | — | ~0.5 wk elapsed | — |
| **1. Skeleton & cleanup** | New `_toctree.yml` (all KEPT/MOVED pages in final positions), `_redirects.yml`, delete 14 orphans, retitles, SO-100 legacy banner | 1 mechanical PR (no content changes — easy review) | 12 days | no |
| **2. Beginner course** | 11 `learn/` pages, `so101` + `il_robots` splits, `quickstart_no_robot`, `troubleshooting`; `il_robots` redirect ships here | 3 PRs (steps 04 / 510 / quickstart+troubleshooting) | 58 days | partially — single author for voice; troubleshooting separable |
| **3. Hub / decision pages** | `choose_a_robot`, `choose_a_policy`, `choose_a_benchmark`, `teleoperators_overview`, `reward_models_overview`, `index` rewrite | 6 independent PRs | 46 days | **fully** — good first issues per area owner |
| **4. How-to consolidation** | `record_dataset`, `data_collection_tips`, `visualize_replay`, `train_policy`, `evaluate_policy`, dataset split, cheat-sheet → `cli_reference` (+ its redirect) | 34 PRs (data / train / deploy / datasets) | 57 days | parallel across clusters |
| **5. Reference fill-in** | `diffusion`, `legacy_policies`, ZMQ camera docs, 6 undocumented CLI commands, keyboard/gamepad teleop content, policy/robot pages normalized to templates | many small PRs | 46 days | fully — community-friendly |
Total ≈ **45 person-weeks**. Phases 2 and 3 can run concurrently after Phase 1 merges. Each URL-breaking change ships in the same PR as its redirect.
---
## 10. Open questions for maintainers
1. **Legacy policies (TDMPC, VQ-BeT):** combined `legacy_policies` page (proposed), full individual pages, or src READMEs only?
2. **Orphan READMEs:** verified byte-identical hand-copies of `src/lerobot/policies/*/README.md` with no sync script — OK to delete from `docs/source/` in Phase 1? Which copy is canonical going forward (proposed: src)?
3. **AGENT_GUIDE.md future:** large parts become docs pages (§5–§8). Slim it to a pointer file for AI agents (avoids divergence), or keep self-contained and accept dual maintenance?
4. **Course URL namespace:** `learn/` (proposed) vs `getting_started/` vs `tutorial/` — permanent once shipped, decide before Phase 2.
5. **LeLab status:** first-party and maintained? If yes, add a "prefer a GUI?" callout inside the course (teleop/record steps); if experimental, keep in Reference & resources as proposed.
6. **SO-100 messaging:** proposed banner "SO-101 is the current generation" + full docs retained. Strong enough, or formally deprecate?
7. **HIL-SERL split:** `hilserl.mdx` is ~950 lines. Split into "Setup & demonstrations" + "Reward classifier & training" in a later phase, or keep monolithic for its expert audience?
8. **Blessed no-hardware quickstart:** PushT, a Hub SO-101 dataset, gym-hil, or an EnvHub env? Determines `quickstart_no_robot` content.
9. **`<hfoptions>` id taxonomy:** agree on the canonical ids (`robot`, `os`, `install`, `train_env`) — they persist site-wide once shipped.
10. **Reward models placement:** standalone section after Policies (proposed) vs nested under Reinforcement learning — depends on whether SARM/ROBOMETER/TOPReward are positioned as general eval tools or RL components.
---
## Appendix: sources & verification
- Current nav: `docs/source/_toctree.yml` (76 pages, 15 sections — verified June 2026).
- File census: 90 content files in `docs/source/`; orphans confirmed by toctree diff; orphan↔src byte-identity confirmed by `diff`; absence of a sync script confirmed by repo-wide grep.
- CLI commands: `pyproject.toml [project.scripts]` (17 entries).
- AGENT_GUIDE.md section references (§5 data tips, §6 policy choice, §7 training duration, §8 evaluation) verified against its headings.
- Patterns referenced: Diátaxis (diataxis.fr) · FastAPI docs · Ultralytics YOLO docs · HF Transformers docs (doc-builder feature ceiling).
+172
View File
@@ -0,0 +1,172 @@
- sections:
- local: index
title: LeRobot
- local: installation
title: Installation
- local: cheat-sheet
title: Cheat sheet
title: Get started
- sections:
- local: il_robots
title: Imitation Learning for Robots
- local: bring_your_own_policies
title: Adding a Policy
- local: integrate_hardware
title: Bring Your Own Hardware
- local: hilserl
title: Train a Robot with RL
- local: hilserl_sim
title: Train RL in Simulation
- local: multi_gpu_training
title: Multi GPU training
- local: hil_data_collection
title: Human In the Loop Data Collection
- local: peft_training
title: Training with PEFT (e.g., LoRA)
- local: rename_map
title: Using Rename Map and Empty Cameras
title: "Tutorials"
- sections:
- local: hardware_guide
title: Compute Hardware Guide
- local: torch_accelerators
title: PyTorch accelerators
title: "Compute & Hardware"
- sections:
- local: lerobot-dataset-v3
title: Using LeRobotDataset
- local: porting_datasets_v3
title: Porting Large Datasets
- local: using_dataset_tools
title: Using the Dataset Tools
- local: language_and_recipes
title: Language Columns and Recipes
- local: tools
title: Tools
- local: video_encoding_parameters
title: Video encoding parameters
- local: streaming_video_encoding
title: Streaming Video Encoding
title: "Datasets"
- sections:
- local: act
title: ACT
- local: smolvla
title: SmolVLA
- local: pi0
title: π₀ (Pi0)
- local: pi0fast
title: π₀-FAST (Pi0Fast)
- local: pi05
title: π₀.₅ (Pi05)
- local: eo1
title: EO-1
- local: groot
title: NVIDIA GR00T N1.5
- local: xvla
title: X-VLA
- local: multi_task_dit
title: Multitask DiT Policy
- local: walloss
title: WALL-OSS
title: "Policies"
- sections:
- local: sarm
title: SARM
title: "Reward Models"
- sections:
- local: inference
title: Policy Deployment (lerobot-rollout)
- local: async
title: Use Async Inference
- local: rtc
title: Real-Time Chunking (RTC)
title: "Inference"
- sections:
- local: envhub
title: Environments from the Hub
- local: envhub_leisaac
title: Control & Train Robots in Sim (LeIsaac)
title: "Simulation"
- sections:
- local: adding_benchmarks
title: Adding a New Benchmark
- local: libero
title: LIBERO
- local: libero_plus
title: LIBERO-plus
- local: metaworld
title: Meta-World
- local: robotwin
title: RoboTwin 2.0
- local: robocasa
title: RoboCasa365
- local: robocerebra
title: RoboCerebra
- local: robomme
title: RoboMME
- local: envhub_isaaclab_arena
title: NVIDIA IsaacLab Arena Environments
- local: vlabench
title: VLABench
title: "Benchmarks"
- sections:
- local: introduction_processors
title: Introduction to Robot Processors
- local: debug_processor_pipeline
title: Debug your processor pipeline
- local: implement_your_own_processor
title: Implement your own processor
- local: processors_robots_teleop
title: Processors for Robots and Teleoperators
- local: env_processor
title: Environment Processors
- local: action_representations
title: Action Representations
title: "Robot Processors"
- sections:
- local: so101
title: SO-101
- local: so100
title: SO-100
- local: koch
title: Koch v1.1
- local: lekiwi
title: LeKiwi
- local: hope_jr
title: Hope Jr
- local: reachy2
title: Reachy 2
- local: unitree_g1
title: Unitree G1
- local: earthrover_mini_plus
title: Earth Rover Mini
- local: omx
title: OMX
- local: openarm
title: OpenArm
- local: rebot_b601
title: reBot B601-DM
title: "Robots"
- sections:
- local: phone_teleop
title: Phone
title: "Teleoperators"
- sections:
- local: cameras
title: Cameras
title: "Sensors"
- sections:
- local: notebooks
title: Notebooks
- local: feetech
title: Updating Feetech Firmware
- local: damiao
title: Damiao Motors and CAN Bus
title: "Resources"
- sections:
- local: contributing
title: Contribute to LeRobot
- local: backwardcomp
title: Backward compatibility
title: "About"
+120 -78
View File
@@ -1,54 +1,68 @@
# LeRobot documentation table of contents
#
# Ordering principle: gentle onboarding first, advanced/custom work last.
# Within each top-level section the same rule applies — concept/overview pages
# before reference/per-item pages.
#
# Pages marked "NEW (to create)" do not yet exist as .mdx files; they are
# placeholders for the redesign and must be authored before the docs build.
- sections: - sections:
- local: index - local: index
title: LeRobot title: 🤗 LeRobot
- local: quickstart # NEW (to create) — 15-min zero-to-trained-ACT path
title: Quickstart
- local: installation - local: installation
title: Installation title: Installation
- local: core_concepts # NEW (to create) — datasets, policies, processors, robots, envs in one mental model
title: Core concepts
- local: cheat-sheet - local: cheat-sheet
title: Cheat sheet title: Command cheat sheet
title: Get started title: Get started
- sections: - sections:
- local: il_robots - local: il_robots
title: Imitation Learning for Robots title: Imitation learning end-to-end
- local: bring_your_own_policies
title: Adding a Policy
- local: integrate_hardware
title: Bring Your Own Hardware
- local: hilserl
title: Train a Robot with RL
- local: hilserl_sim
title: Train RL in Simulation
- local: multi_gpu_training
title: Multi GPU training
- local: hil_data_collection - local: hil_data_collection
title: Human In the Loop Data Collection title: Human-in-the-loop data collection
- local: peft_training - local: inference
title: Training with PEFT (e.g., LoRA) title: Deploying a trained policy
- local: rename_map - local: rename_map
title: Using Rename Map and Empty Cameras title: Matching dataset keys to a policy (rename map)
title: "Tutorials" title: Your first project
- sections: - sections:
- local: hardware_guide - local: hardware_guide
title: Compute Hardware Guide title: Compute hardware guide
- local: torch_accelerators - local: torch_accelerators
title: PyTorch accelerators title: PyTorch accelerators
title: "Compute & Hardware" - local: multi_gpu_training
title: Multi-GPU training
- local: peft_training
title: Parameter-efficient fine-tuning (LoRA)
title: Training
- sections: - sections:
- local: lerobot-dataset-v3 - local: lerobot-dataset-v3
title: Using LeRobotDataset title: Using LeRobotDataset
- local: porting_datasets_v3
title: Porting Large Datasets
- local: using_dataset_tools - local: using_dataset_tools
title: Using the Dataset Tools title: Dataset tools
- local: language_and_recipes - local: language_and_recipes
title: Language Columns and Recipes title: Language columns & recipes
- local: tools - local: tools
title: Tools title: Tool calls in datasets
- local: video_encoding_parameters - local: video_encoding_parameters
title: Video encoding parameters title: Video encoding parameters
- local: streaming_video_encoding - local: streaming_video_encoding
title: Streaming Video Encoding title: Streaming video encoding
title: "Datasets" - local: porting_datasets_v3
title: Porting datasets to v3
title: Datasets
- sections: - sections:
- local: policies_overview # NEW (to create) — concept hub + "choose a policy" decision guide
title: Choosing a policy
- sections:
- local: act - local: act
title: ACT title: ACT
- local: smolvla - local: smolvla
@@ -56,7 +70,7 @@
- local: pi0 - local: pi0
title: π₀ (Pi0) title: π₀ (Pi0)
- local: pi0fast - local: pi0fast
title: π₀-FAST (Pi0Fast) title: π₀-FAST
- local: pi05 - local: pi05
title: π₀.₅ (Pi05) title: π₀.₅ (Pi05)
- local: eo1 - local: eo1
@@ -65,32 +79,37 @@
title: NVIDIA GR00T N1.5 title: NVIDIA GR00T N1.5
- local: xvla - local: xvla
title: X-VLA title: X-VLA
- local: multi_task_dit
title: Multitask DiT Policy
- local: walloss - local: walloss
title: WALL-OSS title: WALL-OSS
title: "Policies" - local: multi_task_dit
title: Multitask DiT
title: Policy reference
title: Policies
- sections: - sections:
- local: sarm
title: SARM
title: "Reward Models"
- sections:
- local: inference
title: Policy Deployment (lerobot-rollout)
- local: async - local: async
title: Use Async Inference title: Async inference
- local: rtc - local: rtc
title: Real-Time Chunking (RTC) title: Real-time chunking (RTC)
title: "Inference" title: Real-time deployment
- sections:
- local: hilserl
title: Train a robot with RL (HIL-SERL)
- local: hilserl_sim
title: Train RL in simulation
- local: sarm
title: SARM reward model
title: Reinforcement learning
- sections: - sections:
- local: envhub - local: envhub
title: Environments from the Hub title: Environments from the Hub
- local: envhub_leisaac - local: envhub_leisaac
title: Control & Train Robots in Sim (LeIsaac) title: LeIsaac — control & train in sim
title: "Simulation" - local: envhub_isaaclab_arena
- sections: title: NVIDIA IsaacLab Arena environments
- local: adding_benchmarks - sections:
title: Adding a New Benchmark
- local: libero - local: libero
title: LIBERO title: LIBERO
- local: libero_plus - local: libero_plus
@@ -105,68 +124,91 @@
title: RoboCerebra title: RoboCerebra
- local: robomme - local: robomme
title: RoboMME title: RoboMME
- local: envhub_isaaclab_arena
title: NVIDIA IsaacLab Arena Environments
- local: vlabench - local: vlabench
title: VLABench title: VLABench
title: "Benchmarks" title: Benchmark suites
title: Simulation & benchmarks
- sections: - sections:
- local: introduction_processors - local: introduction_processors
title: Introduction to Robot Processors title: Introduction to processors
- local: debug_processor_pipeline
title: Debug your processor pipeline
- local: implement_your_own_processor
title: Implement your own processor
- local: processors_robots_teleop - local: processors_robots_teleop
title: Processors for Robots and Teleoperators title: Processors for robots & teleoperators
- local: env_processor - local: env_processor
title: Environment Processors title: Environment processors
- local: action_representations - local: action_representations
title: Action Representations title: Action representations
title: "Robot Processors" - local: debug_processor_pipeline
title: Debugging a pipeline
- local: implement_your_own_processor
title: Implementing your own processor
title: Processors
- sections: - sections:
- sections:
- local: so101 - local: so101
title: SO-101 title: SO-101
- local: so100 - local: so100
title: SO-100 title: SO-100
- local: koch - local: koch
title: Koch v1.1 title: Koch v1.1
- local: omx
title: OMX
- local: openarm
title: OpenArm
title: Low-cost arms
- sections:
- local: lekiwi - local: lekiwi
title: LeKiwi title: LeKiwi
- local: earthrover_mini_plus
title: Earth Rover Mini
title: Mobile platforms
- sections:
- local: hope_jr - local: hope_jr
title: Hope Jr title: Hope Jr
- local: reachy2 - local: reachy2
title: Reachy 2 title: Reachy 2
- local: unitree_g1 - local: unitree_g1
title: Unitree G1 title: Unitree G1
- local: earthrover_mini_plus title: Bimanual & humanoid
title: Earth Rover Mini - sections:
- local: omx
title: OMX
- local: openarm
title: OpenArm
- local: rebot_b601 - local: rebot_b601
title: reBot B601-DM title: reBot B601-DM
title: "Robots" title: Research & industrial
- sections: title: Supported robots
- local: phone_teleop
title: Phone
title: "Teleoperators"
- sections: - sections:
- local: cameras - local: cameras
title: Cameras title: Cameras
title: "Sensors" - local: phone_teleop
- sections: title: Phone teleoperation
- local: notebooks
title: Notebooks
- local: feetech - local: feetech
title: Updating Feetech Firmware title: Feetech firmware update
- local: damiao - local: damiao
title: Damiao Motors and CAN Bus title: Damiao motors & CAN bus
title: "Resources" title: Sensors, teleop & motors
- sections: - sections:
- local: contributing - local: integrate_hardware
title: Contribute to LeRobot title: Bring your own hardware
- local: bring_your_own_policies
title: Add a new policy
- local: adding_benchmarks
title: Add a new benchmark
title: Extend LeRobot
- sections:
- local: troubleshooting # NEW (to create) — common errors: USB, calibration drift, CUDA OOM, video decoding…
title: Troubleshooting & FAQ
- local: glossary # NEW (to create) — episode, action chunk, leader/follower, teleop, processor…
title: Glossary
- local: notebooks
title: Example notebooks
- local: backwardcomp - local: backwardcomp
title: Backward compatibility title: Backward compatibility
title: "About" title: Reference
- sections:
- local: contributing
title: Contributing to LeRobot
title: About
+219
View File
@@ -0,0 +1,219 @@
# Quickstart
This is the **shortest path** from an unboxed SO-101 to a policy that drives your own robot. Every step is copy-paste; replace the **`<placeholders>`** with the values for your setup.
By the end you will have:
- A calibrated SO-101 leader + follower pair.
- A dataset of 30 episodes pushed to the Hugging Face Hub.
- A trained ACT policy (~20k steps) running on your robot via `lerobot-rollout`.
> [!NOTE]
> **How long will this take?**
> Recording 30 episodes is roughly 3060 minutes of teleoperation. Training ACT for 20k steps takes ~1.5h on an A100, a few hours on a laptop RTX 3060, longer on Apple Silicon (`mps`). The commands themselves are quick — most of the wall-clock is data collection and training.
> [!TIP]
> If you only want to **understand the codebase** or **train on an existing dataset without hardware**, this page isn't for you. Read [Core concepts](./core_concepts) first, then jump to [Imitation learning end-to-end](./il_robots).
---
## Before you start
You need:
- An **assembled SO-101 leader + follower pair**. If your robot is not assembled yet, follow the [SO-101 assembly guide](./so101) and come back here.
- **One or two cameras** (USB webcam works fine).
- A **CUDA GPU with ≥ 6 GB VRAM** (ACT is light — a laptop RTX 3060 works). Apple Silicon (`mps`) and CPU are supported but slower. See the [compute hardware guide](./hardware_guide) for sizing.
- A **Hugging Face account** — datasets and the trained policy will be pushed to your Hub.
If any of the above is missing, fix it first; the rest of the page assumes it.
---
## Step 1 — Install LeRobot
Follow the full [Installation Guide](./installation) for environment setup, then add the SO-101 motor stack and log in to the Hub:
```bash
pip install 'lerobot[feetech]'
git lfs install && git lfs pull
hf auth login # paste a token from https://huggingface.co/settings/tokens
```
Sanity check — the CLI entry points should be available:
```bash
lerobot-find-port --help
```
---
## Step 2 — Identify USB ports and motor IDs
Plug **only the follower arm** in (USB + power) and run:
```bash
lerobot-find-port
```
When prompted, unplug it and press Enter. Note the printed port — that's your `<FOLLOWER_PORT>`. Repeat with only the **leader arm** plugged in to get `<LEADER_PORT>`.
> [!TIP]
> On Linux, USB ports look like `/dev/ttyACM0`; on macOS like `/dev/tty.usbmodem...`. On Linux you may need `sudo chmod 666 /dev/ttyACM0` to grant access.
If your motors are brand-new (or repurposed), set their IDs and baudrate **once per arm**:
```bash
lerobot-setup-motors --robot.type=so101_follower --robot.port=<FOLLOWER_PORT>
lerobot-setup-motors --teleop.type=so101_leader --teleop.port=<LEADER_PORT>
```
The script walks you through connecting motors one at a time. Full details: [SO-101 → Configure the motors](./so101#configure-the-motors).
---
## Step 3 — Calibrate
Center every joint roughly in the middle of its range, then run:
```bash
lerobot-calibrate \
--robot.type=so101_follower \
--robot.port=<FOLLOWER_PORT> \
--robot.id=my_follower
lerobot-calibrate \
--teleop.type=so101_leader \
--teleop.port=<LEADER_PORT> \
--teleop.id=my_leader
```
After pressing Enter, sweep each joint through its full range of motion, then press Enter again to finish.
> [!WARNING]
> The `--robot.id` / `--teleop.id` values (`my_follower`, `my_leader`) become the **calibration keys**. Reuse the same IDs in every later command — that's how LeRobot finds the calibration on disk.
Watch the [calibration video](./so101#calibrate) if anything is unclear.
---
## Step 4 — Teleoperate (sanity check, no recording)
Before recording anything, confirm the leader drives the follower correctly:
```bash
lerobot-teleoperate \
--robot.type=so101_follower \
--robot.port=<FOLLOWER_PORT> \
--robot.id=my_follower \
--robot.cameras="{ top: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30} }" \
--teleop.type=so101_leader \
--teleop.port=<LEADER_PORT> \
--teleop.id=my_leader \
--display_data=true
```
A Rerun window should open showing the camera feed and joint angles. Move the leader — the follower should mirror it in real time. If it doesn't, see [Troubleshooting & FAQ](./troubleshooting).
Don't know which camera index is which? Run `lerobot-find-cameras` — it saves a frame from each detected camera so you can pick the right one.
---
## Step 5 — Record a dataset (30 episodes)
Now record demonstrations. Pick a short, repeatable task (e.g. *"put the red brick in the bowl"*). The dataset is pushed to the Hub under your username:
```bash
export HF_USER=<your-hf-username>
lerobot-record \
--robot.type=so101_follower \
--robot.port=<FOLLOWER_PORT> \
--robot.id=my_follower \
--robot.cameras="{ top: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30} }" \
--teleop.type=so101_leader \
--teleop.port=<LEADER_PORT> \
--teleop.id=my_leader \
--dataset.repo_id=${HF_USER}/so101_quickstart \
--dataset.num_episodes=30 \
--dataset.single_task="Put the red brick in the bowl" \
--dataset.streaming_encoding=true \
--display_data=true
```
**Keyboard controls during recording:**
- **`→` (Right Arrow)** — save the current episode and move to the next.
- **`←` (Left Arrow)** — discard the current episode and retry.
- **`Esc`** — stop, encode videos, and upload to the Hub.
> [!TIP]
> **Quality beats quantity.** 30 clean, varied episodes (different brick positions, lighting, camera shake) train a much better policy than 100 identical ones. Move the object around. Vary your speed slightly.
When you're done, your dataset lives at `https://huggingface.co/datasets/${HF_USER}/so101_quickstart`. You can preview it in the browser. For deeper recording options (resume, multiple tasks, custom processors), see [Imitation learning end-to-end → Record](./il_robots#record-a-dataset).
---
## Step 6 — Train ACT
ACT (Action Chunking Transformer) is the right default for a first run — small, fast, and works well on 30 episodes.
```bash
lerobot-train \
--dataset.repo_id=${HF_USER}/so101_quickstart \
--policy.type=act \
--output_dir=outputs/train/act_so101_quickstart \
--job_name=act_so101_quickstart \
--policy.device=cuda \
--policy.repo_id=${HF_USER}/act_so101_quickstart \
--steps=20000 \
--wandb.enable=true
```
A few notes:
- Replace `--policy.device=cuda` with `mps` on Apple Silicon, or `cpu` if you have no GPU (very slow — not recommended for a real run).
- `--wandb.enable=true` is optional. If you use it, run `wandb login` first. Otherwise drop the flag.
- Checkpoints land in `outputs/train/act_so101_quickstart/checkpoints/`. The final model is also pushed to the Hub at the `--policy.repo_id` you specified.
- To resume from an interruption: `lerobot-train --config_path=outputs/train/act_so101_quickstart/checkpoints/last/pretrained_model/train_config.json --resume=true`.
> [!TIP]
> **No GPU locally?** Train on Google Colab using the [ACT notebook](./notebooks#training-act), or rent a GPU via [Hugging Face Jobs](./il_robots#train-using-hugging-face-jobs) — pay-as-you-go, no setup.
For why ACT is the default and when to switch to SmolVLA, Pi0, or another policy, see [Choosing a policy](./policies_overview).
---
## Step 7 — Run your policy on the robot
Deploy with `lerobot-rollout`. **Use the same camera layout you used while recording** — keys and resolutions must match.
```bash
lerobot-rollout \
--strategy.type=base \
--policy.path=${HF_USER}/act_so101_quickstart \
--robot.type=so101_follower \
--robot.port=<FOLLOWER_PORT> \
--robot.id=my_follower \
--robot.cameras="{ top: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30} }" \
--task="Put the red brick in the bowl" \
--duration=60
```
`--duration` is in seconds — leave it off to run until you stop the script. You should see the follower arm move on its own, attempting the task.
If observations from the robot use different keys than the policy expects, you'll need a [rename map](./rename_map). If latency matters, look at [async inference](./async) and [real-time chunking](./rtc).
---
## You're done 🎉
You now have a working IL pipeline end-to-end. From here, the natural next steps are:
- **Improve the policy** — record more diverse episodes, train longer, or try a stronger model. See [Choosing a policy](./policies_overview).
- **Go deeper on imitation learning** — [Imitation learning end-to-end](./il_robots) covers multi-camera setups, multi-task datasets, episode replay, evaluation, and Hugging Face Jobs.
- **Try RL with a human in the loop** — [HIL-SERL](./hilserl) trains a policy that improves while you correct it.
- **Use a different robot** — see [Supported robots](./so101) for low-cost arms, mobile platforms, bimanual, and humanoid.
- **Build something new** — [Bring your own hardware](./integrate_hardware) and [Add a new policy](./bring_your_own_policies).
Stuck on something? Check [Troubleshooting & FAQ](./troubleshooting), or ask on [Discord](https://discord.gg/s3KuuzsPFb).