mirror of
https://github.com/huggingface/lerobot.git
synced 2026-05-28 06:59:44 +00:00
Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| bec7d668a6 |
@@ -0,0 +1,11 @@
|
|||||||
|
version: 2
|
||||||
|
updates:
|
||||||
|
- package-ecosystem: "github-actions"
|
||||||
|
directory: "/"
|
||||||
|
schedule:
|
||||||
|
interval: "weekly"
|
||||||
|
cooldown:
|
||||||
|
default-days: 7
|
||||||
|
groups:
|
||||||
|
actions:
|
||||||
|
patterns: ["*"]
|
||||||
@@ -1,172 +0,0 @@
|
|||||||
- sections:
|
|
||||||
- local: index
|
|
||||||
title: LeRobot
|
|
||||||
- local: installation
|
|
||||||
title: Installation
|
|
||||||
- local: cheat-sheet
|
|
||||||
title: Cheat sheet
|
|
||||||
title: Get started
|
|
||||||
- sections:
|
|
||||||
- local: il_robots
|
|
||||||
title: Imitation Learning for Robots
|
|
||||||
- local: bring_your_own_policies
|
|
||||||
title: Adding a Policy
|
|
||||||
- local: integrate_hardware
|
|
||||||
title: Bring Your Own Hardware
|
|
||||||
- local: hilserl
|
|
||||||
title: Train a Robot with RL
|
|
||||||
- local: hilserl_sim
|
|
||||||
title: Train RL in Simulation
|
|
||||||
- local: multi_gpu_training
|
|
||||||
title: Multi GPU training
|
|
||||||
- local: hil_data_collection
|
|
||||||
title: Human In the Loop Data Collection
|
|
||||||
- local: peft_training
|
|
||||||
title: Training with PEFT (e.g., LoRA)
|
|
||||||
- local: rename_map
|
|
||||||
title: Using Rename Map and Empty Cameras
|
|
||||||
title: "Tutorials"
|
|
||||||
- sections:
|
|
||||||
- local: hardware_guide
|
|
||||||
title: Compute Hardware Guide
|
|
||||||
- local: torch_accelerators
|
|
||||||
title: PyTorch accelerators
|
|
||||||
title: "Compute & Hardware"
|
|
||||||
- sections:
|
|
||||||
- local: lerobot-dataset-v3
|
|
||||||
title: Using LeRobotDataset
|
|
||||||
- local: porting_datasets_v3
|
|
||||||
title: Porting Large Datasets
|
|
||||||
- local: using_dataset_tools
|
|
||||||
title: Using the Dataset Tools
|
|
||||||
- local: language_and_recipes
|
|
||||||
title: Language Columns and Recipes
|
|
||||||
- local: tools
|
|
||||||
title: Tools
|
|
||||||
- local: video_encoding_parameters
|
|
||||||
title: Video encoding parameters
|
|
||||||
- local: streaming_video_encoding
|
|
||||||
title: Streaming Video Encoding
|
|
||||||
title: "Datasets"
|
|
||||||
- sections:
|
|
||||||
- local: act
|
|
||||||
title: ACT
|
|
||||||
- local: smolvla
|
|
||||||
title: SmolVLA
|
|
||||||
- local: pi0
|
|
||||||
title: π₀ (Pi0)
|
|
||||||
- local: pi0fast
|
|
||||||
title: π₀-FAST (Pi0Fast)
|
|
||||||
- local: pi05
|
|
||||||
title: π₀.₅ (Pi05)
|
|
||||||
- local: eo1
|
|
||||||
title: EO-1
|
|
||||||
- local: groot
|
|
||||||
title: NVIDIA GR00T N1.5
|
|
||||||
- local: xvla
|
|
||||||
title: X-VLA
|
|
||||||
- local: multi_task_dit
|
|
||||||
title: Multitask DiT Policy
|
|
||||||
- local: walloss
|
|
||||||
title: WALL-OSS
|
|
||||||
title: "Policies"
|
|
||||||
- sections:
|
|
||||||
- local: sarm
|
|
||||||
title: SARM
|
|
||||||
title: "Reward Models"
|
|
||||||
- sections:
|
|
||||||
- local: inference
|
|
||||||
title: Policy Deployment (lerobot-rollout)
|
|
||||||
- local: async
|
|
||||||
title: Use Async Inference
|
|
||||||
- local: rtc
|
|
||||||
title: Real-Time Chunking (RTC)
|
|
||||||
title: "Inference"
|
|
||||||
- sections:
|
|
||||||
- local: envhub
|
|
||||||
title: Environments from the Hub
|
|
||||||
- local: envhub_leisaac
|
|
||||||
title: Control & Train Robots in Sim (LeIsaac)
|
|
||||||
title: "Simulation"
|
|
||||||
- sections:
|
|
||||||
- local: adding_benchmarks
|
|
||||||
title: Adding a New Benchmark
|
|
||||||
- local: libero
|
|
||||||
title: LIBERO
|
|
||||||
- local: libero_plus
|
|
||||||
title: LIBERO-plus
|
|
||||||
- local: metaworld
|
|
||||||
title: Meta-World
|
|
||||||
- local: robotwin
|
|
||||||
title: RoboTwin 2.0
|
|
||||||
- local: robocasa
|
|
||||||
title: RoboCasa365
|
|
||||||
- local: robocerebra
|
|
||||||
title: RoboCerebra
|
|
||||||
- local: robomme
|
|
||||||
title: RoboMME
|
|
||||||
- local: envhub_isaaclab_arena
|
|
||||||
title: NVIDIA IsaacLab Arena Environments
|
|
||||||
- local: vlabench
|
|
||||||
title: VLABench
|
|
||||||
title: "Benchmarks"
|
|
||||||
- sections:
|
|
||||||
- local: introduction_processors
|
|
||||||
title: Introduction to Robot Processors
|
|
||||||
- local: debug_processor_pipeline
|
|
||||||
title: Debug your processor pipeline
|
|
||||||
- local: implement_your_own_processor
|
|
||||||
title: Implement your own processor
|
|
||||||
- local: processors_robots_teleop
|
|
||||||
title: Processors for Robots and Teleoperators
|
|
||||||
- local: env_processor
|
|
||||||
title: Environment Processors
|
|
||||||
- local: action_representations
|
|
||||||
title: Action Representations
|
|
||||||
title: "Robot Processors"
|
|
||||||
- sections:
|
|
||||||
- local: so101
|
|
||||||
title: SO-101
|
|
||||||
- local: so100
|
|
||||||
title: SO-100
|
|
||||||
- local: koch
|
|
||||||
title: Koch v1.1
|
|
||||||
- local: lekiwi
|
|
||||||
title: LeKiwi
|
|
||||||
- local: hope_jr
|
|
||||||
title: Hope Jr
|
|
||||||
- local: reachy2
|
|
||||||
title: Reachy 2
|
|
||||||
- local: unitree_g1
|
|
||||||
title: Unitree G1
|
|
||||||
- local: earthrover_mini_plus
|
|
||||||
title: Earth Rover Mini
|
|
||||||
- local: omx
|
|
||||||
title: OMX
|
|
||||||
- local: openarm
|
|
||||||
title: OpenArm
|
|
||||||
- local: rebot_b601
|
|
||||||
title: reBot B601-DM
|
|
||||||
title: "Robots"
|
|
||||||
- sections:
|
|
||||||
- local: phone_teleop
|
|
||||||
title: Phone
|
|
||||||
title: "Teleoperators"
|
|
||||||
- sections:
|
|
||||||
- local: cameras
|
|
||||||
title: Cameras
|
|
||||||
title: "Sensors"
|
|
||||||
- sections:
|
|
||||||
- local: notebooks
|
|
||||||
title: Notebooks
|
|
||||||
- local: feetech
|
|
||||||
title: Updating Feetech Firmware
|
|
||||||
- local: damiao
|
|
||||||
title: Damiao Motors and CAN Bus
|
|
||||||
title: "Resources"
|
|
||||||
- sections:
|
|
||||||
- local: contributing
|
|
||||||
title: Contribute to LeRobot
|
|
||||||
- local: backwardcomp
|
|
||||||
title: Backward compatibility
|
|
||||||
title: "About"
|
|
||||||
+80
-122
@@ -1,67 +1,53 @@
|
|||||||
# LeRobot documentation table of contents
|
|
||||||
#
|
|
||||||
# Ordering principle: gentle onboarding first, advanced/custom work last.
|
|
||||||
# Within each top-level section the same rule applies — concept/overview pages
|
|
||||||
# before reference/per-item pages.
|
|
||||||
#
|
|
||||||
# Pages marked "NEW (to create)" do not yet exist as .mdx files; they are
|
|
||||||
# placeholders for the redesign and must be authored before the docs build.
|
|
||||||
|
|
||||||
- sections:
|
- sections:
|
||||||
- local: index
|
- local: index
|
||||||
title: 🤗 LeRobot
|
title: LeRobot
|
||||||
- local: quickstart # NEW (to create) — 15-min zero-to-trained-ACT path
|
|
||||||
title: Quickstart
|
|
||||||
- local: installation
|
- local: installation
|
||||||
title: Installation
|
title: Installation
|
||||||
- local: core_concepts # NEW (to create) — datasets, policies, processors, robots, envs in one mental model
|
|
||||||
title: Core concepts
|
|
||||||
- local: cheat-sheet
|
- local: cheat-sheet
|
||||||
title: Command cheat sheet
|
title: Cheat sheet
|
||||||
title: Get started
|
title: Get started
|
||||||
|
|
||||||
- sections:
|
- sections:
|
||||||
- local: il_robots
|
- local: il_robots
|
||||||
title: Imitation learning end-to-end
|
title: Imitation Learning for Robots
|
||||||
|
- local: bring_your_own_policies
|
||||||
|
title: Adding a Policy
|
||||||
|
- local: integrate_hardware
|
||||||
|
title: Bring Your Own Hardware
|
||||||
|
- local: hilserl
|
||||||
|
title: Train a Robot with RL
|
||||||
|
- local: hilserl_sim
|
||||||
|
title: Train RL in Simulation
|
||||||
|
- local: multi_gpu_training
|
||||||
|
title: Multi GPU training
|
||||||
- local: hil_data_collection
|
- local: hil_data_collection
|
||||||
title: Human-in-the-loop data collection
|
title: Human In the Loop Data Collection
|
||||||
- local: inference
|
- local: peft_training
|
||||||
title: Deploying a trained policy
|
title: Training with PEFT (e.g., LoRA)
|
||||||
- local: rename_map
|
- local: rename_map
|
||||||
title: Matching dataset keys to a policy (rename map)
|
title: Using Rename Map and Empty Cameras
|
||||||
title: Your first project
|
title: "Tutorials"
|
||||||
|
|
||||||
- sections:
|
- sections:
|
||||||
- local: hardware_guide
|
- local: hardware_guide
|
||||||
title: Compute hardware guide
|
title: Compute Hardware Guide
|
||||||
- local: torch_accelerators
|
- local: torch_accelerators
|
||||||
title: PyTorch accelerators
|
title: PyTorch accelerators
|
||||||
- local: multi_gpu_training
|
title: "Compute & Hardware"
|
||||||
title: Multi-GPU training
|
|
||||||
- local: peft_training
|
|
||||||
title: Parameter-efficient fine-tuning (LoRA)
|
|
||||||
title: Training
|
|
||||||
|
|
||||||
- sections:
|
- sections:
|
||||||
- local: lerobot-dataset-v3
|
- local: lerobot-dataset-v3
|
||||||
title: Using LeRobotDataset
|
title: Using LeRobotDataset
|
||||||
|
- local: porting_datasets_v3
|
||||||
|
title: Porting Large Datasets
|
||||||
- local: using_dataset_tools
|
- local: using_dataset_tools
|
||||||
title: Dataset tools
|
title: Using the Dataset Tools
|
||||||
- local: language_and_recipes
|
- local: language_and_recipes
|
||||||
title: Language columns & recipes
|
title: Language Columns and Recipes
|
||||||
- local: tools
|
- local: tools
|
||||||
title: Tool calls in datasets
|
title: Tools
|
||||||
- local: video_encoding_parameters
|
- local: video_encoding_parameters
|
||||||
title: Video encoding parameters
|
title: Video encoding parameters
|
||||||
- local: streaming_video_encoding
|
- local: streaming_video_encoding
|
||||||
title: Streaming video encoding
|
title: Streaming Video Encoding
|
||||||
- local: porting_datasets_v3
|
title: "Datasets"
|
||||||
title: Porting datasets to v3
|
|
||||||
title: Datasets
|
|
||||||
|
|
||||||
- sections:
|
|
||||||
- local: policies_overview # NEW (to create) — concept hub + "choose a policy" decision guide
|
|
||||||
title: Choosing a policy
|
|
||||||
- sections:
|
- sections:
|
||||||
- local: act
|
- local: act
|
||||||
title: ACT
|
title: ACT
|
||||||
@@ -70,7 +56,7 @@
|
|||||||
- local: pi0
|
- local: pi0
|
||||||
title: π₀ (Pi0)
|
title: π₀ (Pi0)
|
||||||
- local: pi0fast
|
- local: pi0fast
|
||||||
title: π₀-FAST
|
title: π₀-FAST (Pi0Fast)
|
||||||
- local: pi05
|
- local: pi05
|
||||||
title: π₀.₅ (Pi05)
|
title: π₀.₅ (Pi05)
|
||||||
- local: eo1
|
- local: eo1
|
||||||
@@ -79,37 +65,32 @@
|
|||||||
title: NVIDIA GR00T N1.5
|
title: NVIDIA GR00T N1.5
|
||||||
- local: xvla
|
- local: xvla
|
||||||
title: X-VLA
|
title: X-VLA
|
||||||
|
- local: multi_task_dit
|
||||||
|
title: Multitask DiT Policy
|
||||||
- local: walloss
|
- local: walloss
|
||||||
title: WALL-OSS
|
title: WALL-OSS
|
||||||
- local: multi_task_dit
|
title: "Policies"
|
||||||
title: Multitask DiT
|
|
||||||
title: Policy reference
|
|
||||||
title: Policies
|
|
||||||
|
|
||||||
- sections:
|
- sections:
|
||||||
- local: async
|
|
||||||
title: Async inference
|
|
||||||
- local: rtc
|
|
||||||
title: Real-time chunking (RTC)
|
|
||||||
title: Real-time deployment
|
|
||||||
|
|
||||||
- sections:
|
|
||||||
- local: hilserl
|
|
||||||
title: Train a robot with RL (HIL-SERL)
|
|
||||||
- local: hilserl_sim
|
|
||||||
title: Train RL in simulation
|
|
||||||
- local: sarm
|
- local: sarm
|
||||||
title: SARM reward model
|
title: SARM
|
||||||
title: Reinforcement learning
|
title: "Reward Models"
|
||||||
|
- sections:
|
||||||
|
- local: inference
|
||||||
|
title: Policy Deployment (lerobot-rollout)
|
||||||
|
- local: async
|
||||||
|
title: Use Async Inference
|
||||||
|
- local: rtc
|
||||||
|
title: Real-Time Chunking (RTC)
|
||||||
|
title: "Inference"
|
||||||
- sections:
|
- sections:
|
||||||
- local: envhub
|
- local: envhub
|
||||||
title: Environments from the Hub
|
title: Environments from the Hub
|
||||||
- local: envhub_leisaac
|
- local: envhub_leisaac
|
||||||
title: LeIsaac — control & train in sim
|
title: Control & Train Robots in Sim (LeIsaac)
|
||||||
- local: envhub_isaaclab_arena
|
title: "Simulation"
|
||||||
title: NVIDIA IsaacLab Arena environments
|
|
||||||
- sections:
|
- sections:
|
||||||
|
- local: adding_benchmarks
|
||||||
|
title: Adding a New Benchmark
|
||||||
- local: libero
|
- local: libero
|
||||||
title: LIBERO
|
title: LIBERO
|
||||||
- local: libero_plus
|
- local: libero_plus
|
||||||
@@ -124,27 +105,25 @@
|
|||||||
title: RoboCerebra
|
title: RoboCerebra
|
||||||
- local: robomme
|
- local: robomme
|
||||||
title: RoboMME
|
title: RoboMME
|
||||||
|
- local: envhub_isaaclab_arena
|
||||||
|
title: NVIDIA IsaacLab Arena Environments
|
||||||
- local: vlabench
|
- local: vlabench
|
||||||
title: VLABench
|
title: VLABench
|
||||||
title: Benchmark suites
|
title: "Benchmarks"
|
||||||
title: Simulation & benchmarks
|
|
||||||
|
|
||||||
- sections:
|
- sections:
|
||||||
- local: introduction_processors
|
- local: introduction_processors
|
||||||
title: Introduction to processors
|
title: Introduction to Robot Processors
|
||||||
- local: processors_robots_teleop
|
|
||||||
title: Processors for robots & teleoperators
|
|
||||||
- local: env_processor
|
|
||||||
title: Environment processors
|
|
||||||
- local: action_representations
|
|
||||||
title: Action representations
|
|
||||||
- local: debug_processor_pipeline
|
- local: debug_processor_pipeline
|
||||||
title: Debugging a pipeline
|
title: Debug your processor pipeline
|
||||||
- local: implement_your_own_processor
|
- local: implement_your_own_processor
|
||||||
title: Implementing your own processor
|
title: Implement your own processor
|
||||||
title: Processors
|
- local: processors_robots_teleop
|
||||||
|
title: Processors for Robots and Teleoperators
|
||||||
- sections:
|
- local: env_processor
|
||||||
|
title: Environment Processors
|
||||||
|
- local: action_representations
|
||||||
|
title: Action Representations
|
||||||
|
title: "Robot Processors"
|
||||||
- sections:
|
- sections:
|
||||||
- local: so101
|
- local: so101
|
||||||
title: SO-101
|
title: SO-101
|
||||||
@@ -152,63 +131,42 @@
|
|||||||
title: SO-100
|
title: SO-100
|
||||||
- local: koch
|
- local: koch
|
||||||
title: Koch v1.1
|
title: Koch v1.1
|
||||||
- local: omx
|
|
||||||
title: OMX
|
|
||||||
- local: openarm
|
|
||||||
title: OpenArm
|
|
||||||
title: Low-cost arms
|
|
||||||
- sections:
|
|
||||||
- local: lekiwi
|
- local: lekiwi
|
||||||
title: LeKiwi
|
title: LeKiwi
|
||||||
- local: earthrover_mini_plus
|
|
||||||
title: Earth Rover Mini
|
|
||||||
title: Mobile platforms
|
|
||||||
- sections:
|
|
||||||
- local: hope_jr
|
- local: hope_jr
|
||||||
title: Hope Jr
|
title: Hope Jr
|
||||||
- local: reachy2
|
- local: reachy2
|
||||||
title: Reachy 2
|
title: Reachy 2
|
||||||
- local: unitree_g1
|
- local: unitree_g1
|
||||||
title: Unitree G1
|
title: Unitree G1
|
||||||
title: Bimanual & humanoid
|
- local: earthrover_mini_plus
|
||||||
- sections:
|
title: Earth Rover Mini
|
||||||
|
- local: omx
|
||||||
|
title: OMX
|
||||||
|
- local: openarm
|
||||||
|
title: OpenArm
|
||||||
- local: rebot_b601
|
- local: rebot_b601
|
||||||
title: reBot B601-DM
|
title: reBot B601-DM
|
||||||
title: Research & industrial
|
title: "Robots"
|
||||||
title: Supported robots
|
- sections:
|
||||||
|
- local: phone_teleop
|
||||||
|
title: Phone
|
||||||
|
title: "Teleoperators"
|
||||||
- sections:
|
- sections:
|
||||||
- local: cameras
|
- local: cameras
|
||||||
title: Cameras
|
title: Cameras
|
||||||
- local: phone_teleop
|
title: "Sensors"
|
||||||
title: Phone teleoperation
|
|
||||||
- local: feetech
|
|
||||||
title: Feetech firmware update
|
|
||||||
- local: damiao
|
|
||||||
title: Damiao motors & CAN bus
|
|
||||||
title: Sensors, teleop & motors
|
|
||||||
|
|
||||||
- sections:
|
- sections:
|
||||||
- local: integrate_hardware
|
|
||||||
title: Bring your own hardware
|
|
||||||
- local: bring_your_own_policies
|
|
||||||
title: Add a new policy
|
|
||||||
- local: adding_benchmarks
|
|
||||||
title: Add a new benchmark
|
|
||||||
title: Extend LeRobot
|
|
||||||
|
|
||||||
- sections:
|
|
||||||
- local: troubleshooting # NEW (to create) — common errors: USB, calibration drift, CUDA OOM, video decoding…
|
|
||||||
title: Troubleshooting & FAQ
|
|
||||||
- local: glossary # NEW (to create) — episode, action chunk, leader/follower, teleop, processor…
|
|
||||||
title: Glossary
|
|
||||||
- local: notebooks
|
- local: notebooks
|
||||||
title: Example notebooks
|
title: Notebooks
|
||||||
- local: backwardcomp
|
- local: feetech
|
||||||
title: Backward compatibility
|
title: Updating Feetech Firmware
|
||||||
title: Reference
|
- local: damiao
|
||||||
|
title: Damiao Motors and CAN Bus
|
||||||
|
title: "Resources"
|
||||||
- sections:
|
- sections:
|
||||||
- local: contributing
|
- local: contributing
|
||||||
title: Contributing to LeRobot
|
title: Contribute to LeRobot
|
||||||
title: About
|
- local: backwardcomp
|
||||||
|
title: Backward compatibility
|
||||||
|
title: "About"
|
||||||
|
|||||||
@@ -1,219 +0,0 @@
|
|||||||
# Quickstart
|
|
||||||
|
|
||||||
This is the **shortest path** from an unboxed SO-101 to a policy that drives your own robot. Every step is copy-paste; replace the **`<placeholders>`** with the values for your setup.
|
|
||||||
|
|
||||||
By the end you will have:
|
|
||||||
|
|
||||||
- A calibrated SO-101 leader + follower pair.
|
|
||||||
- A dataset of 30 episodes pushed to the Hugging Face Hub.
|
|
||||||
- A trained ACT policy (~20k steps) running on your robot via `lerobot-rollout`.
|
|
||||||
|
|
||||||
> [!NOTE]
|
|
||||||
> **How long will this take?**
|
|
||||||
> Recording 30 episodes is roughly 30–60 minutes of teleoperation. Training ACT for 20k steps takes ~1.5h on an A100, a few hours on a laptop RTX 3060, longer on Apple Silicon (`mps`). The commands themselves are quick — most of the wall-clock is data collection and training.
|
|
||||||
|
|
||||||
> [!TIP]
|
|
||||||
> If you only want to **understand the codebase** or **train on an existing dataset without hardware**, this page isn't for you. Read [Core concepts](./core_concepts) first, then jump to [Imitation learning end-to-end](./il_robots).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Before you start
|
|
||||||
|
|
||||||
You need:
|
|
||||||
|
|
||||||
- An **assembled SO-101 leader + follower pair**. If your robot is not assembled yet, follow the [SO-101 assembly guide](./so101) and come back here.
|
|
||||||
- **One or two cameras** (USB webcam works fine).
|
|
||||||
- A **CUDA GPU with ≥ 6 GB VRAM** (ACT is light — a laptop RTX 3060 works). Apple Silicon (`mps`) and CPU are supported but slower. See the [compute hardware guide](./hardware_guide) for sizing.
|
|
||||||
- A **Hugging Face account** — datasets and the trained policy will be pushed to your Hub.
|
|
||||||
|
|
||||||
If any of the above is missing, fix it first; the rest of the page assumes it.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Step 1 — Install LeRobot
|
|
||||||
|
|
||||||
Follow the full [Installation Guide](./installation) for environment setup, then add the SO-101 motor stack and log in to the Hub:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install 'lerobot[feetech]'
|
|
||||||
git lfs install && git lfs pull
|
|
||||||
hf auth login # paste a token from https://huggingface.co/settings/tokens
|
|
||||||
```
|
|
||||||
|
|
||||||
Sanity check — the CLI entry points should be available:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
lerobot-find-port --help
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Step 2 — Identify USB ports and motor IDs
|
|
||||||
|
|
||||||
Plug **only the follower arm** in (USB + power) and run:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
lerobot-find-port
|
|
||||||
```
|
|
||||||
|
|
||||||
When prompted, unplug it and press Enter. Note the printed port — that's your `<FOLLOWER_PORT>`. Repeat with only the **leader arm** plugged in to get `<LEADER_PORT>`.
|
|
||||||
|
|
||||||
> [!TIP]
|
|
||||||
> On Linux, USB ports look like `/dev/ttyACM0`; on macOS like `/dev/tty.usbmodem...`. On Linux you may need `sudo chmod 666 /dev/ttyACM0` to grant access.
|
|
||||||
|
|
||||||
If your motors are brand-new (or repurposed), set their IDs and baudrate **once per arm**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
lerobot-setup-motors --robot.type=so101_follower --robot.port=<FOLLOWER_PORT>
|
|
||||||
lerobot-setup-motors --teleop.type=so101_leader --teleop.port=<LEADER_PORT>
|
|
||||||
```
|
|
||||||
|
|
||||||
The script walks you through connecting motors one at a time. Full details: [SO-101 → Configure the motors](./so101#configure-the-motors).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Step 3 — Calibrate
|
|
||||||
|
|
||||||
Center every joint roughly in the middle of its range, then run:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
lerobot-calibrate \
|
|
||||||
--robot.type=so101_follower \
|
|
||||||
--robot.port=<FOLLOWER_PORT> \
|
|
||||||
--robot.id=my_follower
|
|
||||||
|
|
||||||
lerobot-calibrate \
|
|
||||||
--teleop.type=so101_leader \
|
|
||||||
--teleop.port=<LEADER_PORT> \
|
|
||||||
--teleop.id=my_leader
|
|
||||||
```
|
|
||||||
|
|
||||||
After pressing Enter, sweep each joint through its full range of motion, then press Enter again to finish.
|
|
||||||
|
|
||||||
> [!WARNING]
|
|
||||||
> The `--robot.id` / `--teleop.id` values (`my_follower`, `my_leader`) become the **calibration keys**. Reuse the same IDs in every later command — that's how LeRobot finds the calibration on disk.
|
|
||||||
|
|
||||||
Watch the [calibration video](./so101#calibrate) if anything is unclear.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Step 4 — Teleoperate (sanity check, no recording)
|
|
||||||
|
|
||||||
Before recording anything, confirm the leader drives the follower correctly:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
lerobot-teleoperate \
|
|
||||||
--robot.type=so101_follower \
|
|
||||||
--robot.port=<FOLLOWER_PORT> \
|
|
||||||
--robot.id=my_follower \
|
|
||||||
--robot.cameras="{ top: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30} }" \
|
|
||||||
--teleop.type=so101_leader \
|
|
||||||
--teleop.port=<LEADER_PORT> \
|
|
||||||
--teleop.id=my_leader \
|
|
||||||
--display_data=true
|
|
||||||
```
|
|
||||||
|
|
||||||
A Rerun window should open showing the camera feed and joint angles. Move the leader — the follower should mirror it in real time. If it doesn't, see [Troubleshooting & FAQ](./troubleshooting).
|
|
||||||
|
|
||||||
Don't know which camera index is which? Run `lerobot-find-cameras` — it saves a frame from each detected camera so you can pick the right one.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Step 5 — Record a dataset (30 episodes)
|
|
||||||
|
|
||||||
Now record demonstrations. Pick a short, repeatable task (e.g. *"put the red brick in the bowl"*). The dataset is pushed to the Hub under your username:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
export HF_USER=<your-hf-username>
|
|
||||||
|
|
||||||
lerobot-record \
|
|
||||||
--robot.type=so101_follower \
|
|
||||||
--robot.port=<FOLLOWER_PORT> \
|
|
||||||
--robot.id=my_follower \
|
|
||||||
--robot.cameras="{ top: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30} }" \
|
|
||||||
--teleop.type=so101_leader \
|
|
||||||
--teleop.port=<LEADER_PORT> \
|
|
||||||
--teleop.id=my_leader \
|
|
||||||
--dataset.repo_id=${HF_USER}/so101_quickstart \
|
|
||||||
--dataset.num_episodes=30 \
|
|
||||||
--dataset.single_task="Put the red brick in the bowl" \
|
|
||||||
--dataset.streaming_encoding=true \
|
|
||||||
--display_data=true
|
|
||||||
```
|
|
||||||
|
|
||||||
**Keyboard controls during recording:**
|
|
||||||
|
|
||||||
- **`→` (Right Arrow)** — save the current episode and move to the next.
|
|
||||||
- **`←` (Left Arrow)** — discard the current episode and retry.
|
|
||||||
- **`Esc`** — stop, encode videos, and upload to the Hub.
|
|
||||||
|
|
||||||
> [!TIP]
|
|
||||||
> **Quality beats quantity.** 30 clean, varied episodes (different brick positions, lighting, camera shake) train a much better policy than 100 identical ones. Move the object around. Vary your speed slightly.
|
|
||||||
|
|
||||||
When you're done, your dataset lives at `https://huggingface.co/datasets/${HF_USER}/so101_quickstart`. You can preview it in the browser. For deeper recording options (resume, multiple tasks, custom processors), see [Imitation learning end-to-end → Record](./il_robots#record-a-dataset).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Step 6 — Train ACT
|
|
||||||
|
|
||||||
ACT (Action Chunking Transformer) is the right default for a first run — small, fast, and works well on 30 episodes.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
lerobot-train \
|
|
||||||
--dataset.repo_id=${HF_USER}/so101_quickstart \
|
|
||||||
--policy.type=act \
|
|
||||||
--output_dir=outputs/train/act_so101_quickstart \
|
|
||||||
--job_name=act_so101_quickstart \
|
|
||||||
--policy.device=cuda \
|
|
||||||
--policy.repo_id=${HF_USER}/act_so101_quickstart \
|
|
||||||
--steps=20000 \
|
|
||||||
--wandb.enable=true
|
|
||||||
```
|
|
||||||
|
|
||||||
A few notes:
|
|
||||||
|
|
||||||
- Replace `--policy.device=cuda` with `mps` on Apple Silicon, or `cpu` if you have no GPU (very slow — not recommended for a real run).
|
|
||||||
- `--wandb.enable=true` is optional. If you use it, run `wandb login` first. Otherwise drop the flag.
|
|
||||||
- Checkpoints land in `outputs/train/act_so101_quickstart/checkpoints/`. The final model is also pushed to the Hub at the `--policy.repo_id` you specified.
|
|
||||||
- To resume from an interruption: `lerobot-train --config_path=outputs/train/act_so101_quickstart/checkpoints/last/pretrained_model/train_config.json --resume=true`.
|
|
||||||
|
|
||||||
> [!TIP]
|
|
||||||
> **No GPU locally?** Train on Google Colab using the [ACT notebook](./notebooks#training-act), or rent a GPU via [Hugging Face Jobs](./il_robots#train-using-hugging-face-jobs) — pay-as-you-go, no setup.
|
|
||||||
|
|
||||||
For why ACT is the default and when to switch to SmolVLA, Pi0, or another policy, see [Choosing a policy](./policies_overview).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Step 7 — Run your policy on the robot
|
|
||||||
|
|
||||||
Deploy with `lerobot-rollout`. **Use the same camera layout you used while recording** — keys and resolutions must match.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
lerobot-rollout \
|
|
||||||
--strategy.type=base \
|
|
||||||
--policy.path=${HF_USER}/act_so101_quickstart \
|
|
||||||
--robot.type=so101_follower \
|
|
||||||
--robot.port=<FOLLOWER_PORT> \
|
|
||||||
--robot.id=my_follower \
|
|
||||||
--robot.cameras="{ top: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30} }" \
|
|
||||||
--task="Put the red brick in the bowl" \
|
|
||||||
--duration=60
|
|
||||||
```
|
|
||||||
|
|
||||||
`--duration` is in seconds — leave it off to run until you stop the script. You should see the follower arm move on its own, attempting the task.
|
|
||||||
|
|
||||||
If observations from the robot use different keys than the policy expects, you'll need a [rename map](./rename_map). If latency matters, look at [async inference](./async) and [real-time chunking](./rtc).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## You're done 🎉
|
|
||||||
|
|
||||||
You now have a working IL pipeline end-to-end. From here, the natural next steps are:
|
|
||||||
|
|
||||||
- **Improve the policy** — record more diverse episodes, train longer, or try a stronger model. See [Choosing a policy](./policies_overview).
|
|
||||||
- **Go deeper on imitation learning** — [Imitation learning end-to-end](./il_robots) covers multi-camera setups, multi-task datasets, episode replay, evaluation, and Hugging Face Jobs.
|
|
||||||
- **Try RL with a human in the loop** — [HIL-SERL](./hilserl) trains a policy that improves while you correct it.
|
|
||||||
- **Use a different robot** — see [Supported robots](./so101) for low-cost arms, mobile platforms, bimanual, and humanoid.
|
|
||||||
- **Build something new** — [Bring your own hardware](./integrate_hardware) and [Add a new policy](./bring_your_own_policies).
|
|
||||||
|
|
||||||
Stuck on something? Check [Troubleshooting & FAQ](./troubleshooting), or ask on [Discord](https://discord.gg/s3KuuzsPFb).
|
|
||||||
Reference in New Issue
Block a user