Compare commits

..

1 Commits

Author SHA1 Message Date
hf-dependantbot-rollout[bot] bec7d668a6 chore: enable Dependabot weekly GitHub Actions bumps 2026-05-26 10:32:49 +00:00
4 changed files with 140 additions and 562 deletions
+11
View File
@@ -0,0 +1,11 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
cooldown:
default-days: 7
groups:
actions:
patterns: ["*"]
-172
View File
@@ -1,172 +0,0 @@
- sections:
- local: index
title: LeRobot
- local: installation
title: Installation
- local: cheat-sheet
title: Cheat sheet
title: Get started
- sections:
- local: il_robots
title: Imitation Learning for Robots
- local: bring_your_own_policies
title: Adding a Policy
- local: integrate_hardware
title: Bring Your Own Hardware
- local: hilserl
title: Train a Robot with RL
- local: hilserl_sim
title: Train RL in Simulation
- local: multi_gpu_training
title: Multi GPU training
- local: hil_data_collection
title: Human In the Loop Data Collection
- local: peft_training
title: Training with PEFT (e.g., LoRA)
- local: rename_map
title: Using Rename Map and Empty Cameras
title: "Tutorials"
- sections:
- local: hardware_guide
title: Compute Hardware Guide
- local: torch_accelerators
title: PyTorch accelerators
title: "Compute & Hardware"
- sections:
- local: lerobot-dataset-v3
title: Using LeRobotDataset
- local: porting_datasets_v3
title: Porting Large Datasets
- local: using_dataset_tools
title: Using the Dataset Tools
- local: language_and_recipes
title: Language Columns and Recipes
- local: tools
title: Tools
- local: video_encoding_parameters
title: Video encoding parameters
- local: streaming_video_encoding
title: Streaming Video Encoding
title: "Datasets"
- sections:
- local: act
title: ACT
- local: smolvla
title: SmolVLA
- local: pi0
title: π₀ (Pi0)
- local: pi0fast
title: π₀-FAST (Pi0Fast)
- local: pi05
title: π₀.₅ (Pi05)
- local: eo1
title: EO-1
- local: groot
title: NVIDIA GR00T N1.5
- local: xvla
title: X-VLA
- local: multi_task_dit
title: Multitask DiT Policy
- local: walloss
title: WALL-OSS
title: "Policies"
- sections:
- local: sarm
title: SARM
title: "Reward Models"
- sections:
- local: inference
title: Policy Deployment (lerobot-rollout)
- local: async
title: Use Async Inference
- local: rtc
title: Real-Time Chunking (RTC)
title: "Inference"
- sections:
- local: envhub
title: Environments from the Hub
- local: envhub_leisaac
title: Control & Train Robots in Sim (LeIsaac)
title: "Simulation"
- sections:
- local: adding_benchmarks
title: Adding a New Benchmark
- local: libero
title: LIBERO
- local: libero_plus
title: LIBERO-plus
- local: metaworld
title: Meta-World
- local: robotwin
title: RoboTwin 2.0
- local: robocasa
title: RoboCasa365
- local: robocerebra
title: RoboCerebra
- local: robomme
title: RoboMME
- local: envhub_isaaclab_arena
title: NVIDIA IsaacLab Arena Environments
- local: vlabench
title: VLABench
title: "Benchmarks"
- sections:
- local: introduction_processors
title: Introduction to Robot Processors
- local: debug_processor_pipeline
title: Debug your processor pipeline
- local: implement_your_own_processor
title: Implement your own processor
- local: processors_robots_teleop
title: Processors for Robots and Teleoperators
- local: env_processor
title: Environment Processors
- local: action_representations
title: Action Representations
title: "Robot Processors"
- sections:
- local: so101
title: SO-101
- local: so100
title: SO-100
- local: koch
title: Koch v1.1
- local: lekiwi
title: LeKiwi
- local: hope_jr
title: Hope Jr
- local: reachy2
title: Reachy 2
- local: unitree_g1
title: Unitree G1
- local: earthrover_mini_plus
title: Earth Rover Mini
- local: omx
title: OMX
- local: openarm
title: OpenArm
- local: rebot_b601
title: reBot B601-DM
title: "Robots"
- sections:
- local: phone_teleop
title: Phone
title: "Teleoperators"
- sections:
- local: cameras
title: Cameras
title: "Sensors"
- sections:
- local: notebooks
title: Notebooks
- local: feetech
title: Updating Feetech Firmware
- local: damiao
title: Damiao Motors and CAN Bus
title: "Resources"
- sections:
- local: contributing
title: Contribute to LeRobot
- local: backwardcomp
title: Backward compatibility
title: "About"
+129 -171
View File
@@ -1,214 +1,172 @@
# LeRobot documentation table of contents
#
# Ordering principle: gentle onboarding first, advanced/custom work last.
# Within each top-level section the same rule applies — concept/overview pages
# before reference/per-item pages.
#
# Pages marked "NEW (to create)" do not yet exist as .mdx files; they are
# placeholders for the redesign and must be authored before the docs build.
- sections: - sections:
- local: index - local: index
title: 🤗 LeRobot title: LeRobot
- local: quickstart # NEW (to create) — 15-min zero-to-trained-ACT path
title: Quickstart
- local: installation - local: installation
title: Installation title: Installation
- local: core_concepts # NEW (to create) — datasets, policies, processors, robots, envs in one mental model
title: Core concepts
- local: cheat-sheet - local: cheat-sheet
title: Command cheat sheet title: Cheat sheet
title: Get started title: Get started
- sections: - sections:
- local: il_robots - local: il_robots
title: Imitation learning end-to-end title: Imitation Learning for Robots
- local: bring_your_own_policies
title: Adding a Policy
- local: integrate_hardware
title: Bring Your Own Hardware
- local: hilserl
title: Train a Robot with RL
- local: hilserl_sim
title: Train RL in Simulation
- local: multi_gpu_training
title: Multi GPU training
- local: hil_data_collection - local: hil_data_collection
title: Human-in-the-loop data collection title: Human In the Loop Data Collection
- local: inference - local: peft_training
title: Deploying a trained policy title: Training with PEFT (e.g., LoRA)
- local: rename_map - local: rename_map
title: Matching dataset keys to a policy (rename map) title: Using Rename Map and Empty Cameras
title: Your first project title: "Tutorials"
- sections: - sections:
- local: hardware_guide - local: hardware_guide
title: Compute hardware guide title: Compute Hardware Guide
- local: torch_accelerators - local: torch_accelerators
title: PyTorch accelerators title: PyTorch accelerators
- local: multi_gpu_training title: "Compute & Hardware"
title: Multi-GPU training
- local: peft_training
title: Parameter-efficient fine-tuning (LoRA)
title: Training
- sections: - sections:
- local: lerobot-dataset-v3 - local: lerobot-dataset-v3
title: Using LeRobotDataset title: Using LeRobotDataset
- local: porting_datasets_v3
title: Porting Large Datasets
- local: using_dataset_tools - local: using_dataset_tools
title: Dataset tools title: Using the Dataset Tools
- local: language_and_recipes - local: language_and_recipes
title: Language columns & recipes title: Language Columns and Recipes
- local: tools - local: tools
title: Tool calls in datasets title: Tools
- local: video_encoding_parameters - local: video_encoding_parameters
title: Video encoding parameters title: Video encoding parameters
- local: streaming_video_encoding - local: streaming_video_encoding
title: Streaming video encoding title: Streaming Video Encoding
- local: porting_datasets_v3 title: "Datasets"
title: Porting datasets to v3
title: Datasets
- sections: - sections:
- local: policies_overview # NEW (to create) — concept hub + "choose a policy" decision guide - local: act
title: Choosing a policy title: ACT
- sections: - local: smolvla
- local: act title: SmolVLA
title: ACT - local: pi0
- local: smolvla title: π₀ (Pi0)
title: SmolVLA - local: pi0fast
- local: pi0 title: π₀-FAST (Pi0Fast)
title: π₀ (Pi0) - local: pi05
- local: pi0fast title: π₀.₅ (Pi05)
title: π₀-FAST - local: eo1
- local: pi05 title: EO-1
title: π₀.₅ (Pi05) - local: groot
- local: eo1 title: NVIDIA GR00T N1.5
title: EO-1 - local: xvla
- local: groot title: X-VLA
title: NVIDIA GR00T N1.5 - local: multi_task_dit
- local: xvla title: Multitask DiT Policy
title: X-VLA - local: walloss
- local: walloss title: WALL-OSS
title: WALL-OSS title: "Policies"
- local: multi_task_dit
title: Multitask DiT
title: Policy reference
title: Policies
- sections: - sections:
- local: async
title: Async inference
- local: rtc
title: Real-time chunking (RTC)
title: Real-time deployment
- sections:
- local: hilserl
title: Train a robot with RL (HIL-SERL)
- local: hilserl_sim
title: Train RL in simulation
- local: sarm - local: sarm
title: SARM reward model title: SARM
title: Reinforcement learning title: "Reward Models"
- sections:
- local: inference
title: Policy Deployment (lerobot-rollout)
- local: async
title: Use Async Inference
- local: rtc
title: Real-Time Chunking (RTC)
title: "Inference"
- sections: - sections:
- local: envhub - local: envhub
title: Environments from the Hub title: Environments from the Hub
- local: envhub_leisaac - local: envhub_leisaac
title: LeIsaac — control & train in sim title: Control & Train Robots in Sim (LeIsaac)
title: "Simulation"
- sections:
- local: adding_benchmarks
title: Adding a New Benchmark
- local: libero
title: LIBERO
- local: libero_plus
title: LIBERO-plus
- local: metaworld
title: Meta-World
- local: robotwin
title: RoboTwin 2.0
- local: robocasa
title: RoboCasa365
- local: robocerebra
title: RoboCerebra
- local: robomme
title: RoboMME
- local: envhub_isaaclab_arena - local: envhub_isaaclab_arena
title: NVIDIA IsaacLab Arena environments title: NVIDIA IsaacLab Arena Environments
- sections: - local: vlabench
- local: libero title: VLABench
title: LIBERO title: "Benchmarks"
- local: libero_plus
title: LIBERO-plus
- local: metaworld
title: Meta-World
- local: robotwin
title: RoboTwin 2.0
- local: robocasa
title: RoboCasa365
- local: robocerebra
title: RoboCerebra
- local: robomme
title: RoboMME
- local: vlabench
title: VLABench
title: Benchmark suites
title: Simulation & benchmarks
- sections: - sections:
- local: introduction_processors - local: introduction_processors
title: Introduction to processors title: Introduction to Robot Processors
- local: processors_robots_teleop
title: Processors for robots & teleoperators
- local: env_processor
title: Environment processors
- local: action_representations
title: Action representations
- local: debug_processor_pipeline - local: debug_processor_pipeline
title: Debugging a pipeline title: Debug your processor pipeline
- local: implement_your_own_processor - local: implement_your_own_processor
title: Implementing your own processor title: Implement your own processor
title: Processors - local: processors_robots_teleop
title: Processors for Robots and Teleoperators
- local: env_processor
title: Environment Processors
- local: action_representations
title: Action Representations
title: "Robot Processors"
- sections: - sections:
- sections: - local: so101
- local: so101 title: SO-101
title: SO-101 - local: so100
- local: so100 title: SO-100
title: SO-100 - local: koch
- local: koch title: Koch v1.1
title: Koch v1.1 - local: lekiwi
- local: omx title: LeKiwi
title: OMX - local: hope_jr
- local: openarm title: Hope Jr
title: OpenArm - local: reachy2
title: Low-cost arms title: Reachy 2
- sections: - local: unitree_g1
- local: lekiwi title: Unitree G1
title: LeKiwi - local: earthrover_mini_plus
- local: earthrover_mini_plus title: Earth Rover Mini
title: Earth Rover Mini - local: omx
title: Mobile platforms title: OMX
- sections: - local: openarm
- local: hope_jr title: OpenArm
title: Hope Jr - local: rebot_b601
- local: reachy2 title: reBot B601-DM
title: Reachy 2 title: "Robots"
- local: unitree_g1 - sections:
title: Unitree G1 - local: phone_teleop
title: Bimanual & humanoid title: Phone
- sections: title: "Teleoperators"
- local: rebot_b601
title: reBot B601-DM
title: Research & industrial
title: Supported robots
- sections: - sections:
- local: cameras - local: cameras
title: Cameras title: Cameras
- local: phone_teleop title: "Sensors"
title: Phone teleoperation
- local: feetech
title: Feetech firmware update
- local: damiao
title: Damiao motors & CAN bus
title: Sensors, teleop & motors
- sections: - sections:
- local: integrate_hardware
title: Bring your own hardware
- local: bring_your_own_policies
title: Add a new policy
- local: adding_benchmarks
title: Add a new benchmark
title: Extend LeRobot
- sections:
- local: troubleshooting # NEW (to create) — common errors: USB, calibration drift, CUDA OOM, video decoding…
title: Troubleshooting & FAQ
- local: glossary # NEW (to create) — episode, action chunk, leader/follower, teleop, processor…
title: Glossary
- local: notebooks - local: notebooks
title: Example notebooks title: Notebooks
- local: backwardcomp - local: feetech
title: Backward compatibility title: Updating Feetech Firmware
title: Reference - local: damiao
title: Damiao Motors and CAN Bus
title: "Resources"
- sections: - sections:
- local: contributing - local: contributing
title: Contributing to LeRobot title: Contribute to LeRobot
title: About - local: backwardcomp
title: Backward compatibility
title: "About"
-219
View File
@@ -1,219 +0,0 @@
# Quickstart
This is the **shortest path** from an unboxed SO-101 to a policy that drives your own robot. Every step is copy-paste; replace the **`<placeholders>`** with the values for your setup.
By the end you will have:
- A calibrated SO-101 leader + follower pair.
- A dataset of 30 episodes pushed to the Hugging Face Hub.
- A trained ACT policy (~20k steps) running on your robot via `lerobot-rollout`.
> [!NOTE]
> **How long will this take?**
> Recording 30 episodes is roughly 3060 minutes of teleoperation. Training ACT for 20k steps takes ~1.5h on an A100, a few hours on a laptop RTX 3060, longer on Apple Silicon (`mps`). The commands themselves are quick — most of the wall-clock is data collection and training.
> [!TIP]
> If you only want to **understand the codebase** or **train on an existing dataset without hardware**, this page isn't for you. Read [Core concepts](./core_concepts) first, then jump to [Imitation learning end-to-end](./il_robots).
---
## Before you start
You need:
- An **assembled SO-101 leader + follower pair**. If your robot is not assembled yet, follow the [SO-101 assembly guide](./so101) and come back here.
- **One or two cameras** (USB webcam works fine).
- A **CUDA GPU with ≥ 6 GB VRAM** (ACT is light — a laptop RTX 3060 works). Apple Silicon (`mps`) and CPU are supported but slower. See the [compute hardware guide](./hardware_guide) for sizing.
- A **Hugging Face account** — datasets and the trained policy will be pushed to your Hub.
If any of the above is missing, fix it first; the rest of the page assumes it.
---
## Step 1 — Install LeRobot
Follow the full [Installation Guide](./installation) for environment setup, then add the SO-101 motor stack and log in to the Hub:
```bash
pip install 'lerobot[feetech]'
git lfs install && git lfs pull
hf auth login # paste a token from https://huggingface.co/settings/tokens
```
Sanity check — the CLI entry points should be available:
```bash
lerobot-find-port --help
```
---
## Step 2 — Identify USB ports and motor IDs
Plug **only the follower arm** in (USB + power) and run:
```bash
lerobot-find-port
```
When prompted, unplug it and press Enter. Note the printed port — that's your `<FOLLOWER_PORT>`. Repeat with only the **leader arm** plugged in to get `<LEADER_PORT>`.
> [!TIP]
> On Linux, USB ports look like `/dev/ttyACM0`; on macOS like `/dev/tty.usbmodem...`. On Linux you may need `sudo chmod 666 /dev/ttyACM0` to grant access.
If your motors are brand-new (or repurposed), set their IDs and baudrate **once per arm**:
```bash
lerobot-setup-motors --robot.type=so101_follower --robot.port=<FOLLOWER_PORT>
lerobot-setup-motors --teleop.type=so101_leader --teleop.port=<LEADER_PORT>
```
The script walks you through connecting motors one at a time. Full details: [SO-101 → Configure the motors](./so101#configure-the-motors).
---
## Step 3 — Calibrate
Center every joint roughly in the middle of its range, then run:
```bash
lerobot-calibrate \
--robot.type=so101_follower \
--robot.port=<FOLLOWER_PORT> \
--robot.id=my_follower
lerobot-calibrate \
--teleop.type=so101_leader \
--teleop.port=<LEADER_PORT> \
--teleop.id=my_leader
```
After pressing Enter, sweep each joint through its full range of motion, then press Enter again to finish.
> [!WARNING]
> The `--robot.id` / `--teleop.id` values (`my_follower`, `my_leader`) become the **calibration keys**. Reuse the same IDs in every later command — that's how LeRobot finds the calibration on disk.
Watch the [calibration video](./so101#calibrate) if anything is unclear.
---
## Step 4 — Teleoperate (sanity check, no recording)
Before recording anything, confirm the leader drives the follower correctly:
```bash
lerobot-teleoperate \
--robot.type=so101_follower \
--robot.port=<FOLLOWER_PORT> \
--robot.id=my_follower \
--robot.cameras="{ top: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30} }" \
--teleop.type=so101_leader \
--teleop.port=<LEADER_PORT> \
--teleop.id=my_leader \
--display_data=true
```
A Rerun window should open showing the camera feed and joint angles. Move the leader — the follower should mirror it in real time. If it doesn't, see [Troubleshooting & FAQ](./troubleshooting).
Don't know which camera index is which? Run `lerobot-find-cameras` — it saves a frame from each detected camera so you can pick the right one.
---
## Step 5 — Record a dataset (30 episodes)
Now record demonstrations. Pick a short, repeatable task (e.g. *"put the red brick in the bowl"*). The dataset is pushed to the Hub under your username:
```bash
export HF_USER=<your-hf-username>
lerobot-record \
--robot.type=so101_follower \
--robot.port=<FOLLOWER_PORT> \
--robot.id=my_follower \
--robot.cameras="{ top: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30} }" \
--teleop.type=so101_leader \
--teleop.port=<LEADER_PORT> \
--teleop.id=my_leader \
--dataset.repo_id=${HF_USER}/so101_quickstart \
--dataset.num_episodes=30 \
--dataset.single_task="Put the red brick in the bowl" \
--dataset.streaming_encoding=true \
--display_data=true
```
**Keyboard controls during recording:**
- **`→` (Right Arrow)** — save the current episode and move to the next.
- **`←` (Left Arrow)** — discard the current episode and retry.
- **`Esc`** — stop, encode videos, and upload to the Hub.
> [!TIP]
> **Quality beats quantity.** 30 clean, varied episodes (different brick positions, lighting, camera shake) train a much better policy than 100 identical ones. Move the object around. Vary your speed slightly.
When you're done, your dataset lives at `https://huggingface.co/datasets/${HF_USER}/so101_quickstart`. You can preview it in the browser. For deeper recording options (resume, multiple tasks, custom processors), see [Imitation learning end-to-end → Record](./il_robots#record-a-dataset).
---
## Step 6 — Train ACT
ACT (Action Chunking Transformer) is the right default for a first run — small, fast, and works well on 30 episodes.
```bash
lerobot-train \
--dataset.repo_id=${HF_USER}/so101_quickstart \
--policy.type=act \
--output_dir=outputs/train/act_so101_quickstart \
--job_name=act_so101_quickstart \
--policy.device=cuda \
--policy.repo_id=${HF_USER}/act_so101_quickstart \
--steps=20000 \
--wandb.enable=true
```
A few notes:
- Replace `--policy.device=cuda` with `mps` on Apple Silicon, or `cpu` if you have no GPU (very slow — not recommended for a real run).
- `--wandb.enable=true` is optional. If you use it, run `wandb login` first. Otherwise drop the flag.
- Checkpoints land in `outputs/train/act_so101_quickstart/checkpoints/`. The final model is also pushed to the Hub at the `--policy.repo_id` you specified.
- To resume from an interruption: `lerobot-train --config_path=outputs/train/act_so101_quickstart/checkpoints/last/pretrained_model/train_config.json --resume=true`.
> [!TIP]
> **No GPU locally?** Train on Google Colab using the [ACT notebook](./notebooks#training-act), or rent a GPU via [Hugging Face Jobs](./il_robots#train-using-hugging-face-jobs) — pay-as-you-go, no setup.
For why ACT is the default and when to switch to SmolVLA, Pi0, or another policy, see [Choosing a policy](./policies_overview).
---
## Step 7 — Run your policy on the robot
Deploy with `lerobot-rollout`. **Use the same camera layout you used while recording** — keys and resolutions must match.
```bash
lerobot-rollout \
--strategy.type=base \
--policy.path=${HF_USER}/act_so101_quickstart \
--robot.type=so101_follower \
--robot.port=<FOLLOWER_PORT> \
--robot.id=my_follower \
--robot.cameras="{ top: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30} }" \
--task="Put the red brick in the bowl" \
--duration=60
```
`--duration` is in seconds — leave it off to run until you stop the script. You should see the follower arm move on its own, attempting the task.
If observations from the robot use different keys than the policy expects, you'll need a [rename map](./rename_map). If latency matters, look at [async inference](./async) and [real-time chunking](./rtc).
---
## You're done 🎉
You now have a working IL pipeline end-to-end. From here, the natural next steps are:
- **Improve the policy** — record more diverse episodes, train longer, or try a stronger model. See [Choosing a policy](./policies_overview).
- **Go deeper on imitation learning** — [Imitation learning end-to-end](./il_robots) covers multi-camera setups, multi-task datasets, episode replay, evaluation, and Hugging Face Jobs.
- **Try RL with a human in the loop** — [HIL-SERL](./hilserl) trains a policy that improves while you correct it.
- **Use a different robot** — see [Supported robots](./so101) for low-cost arms, mobile platforms, bimanual, and humanoid.
- **Build something new** — [Bring your own hardware](./integrate_hardware) and [Add a new policy](./bring_your_own_policies).
Stuck on something? Check [Troubleshooting & FAQ](./troubleshooting), or ask on [Discord](https://discord.gg/s3KuuzsPFb).