mirror of
https://github.com/huggingface/lerobot.git
synced 2026-05-17 09:39:47 +00:00
Move build your robot section
This commit is contained in:
Simon Alibert
@@ -22,6 +22,95 @@
|
|||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
🤗 LeRobot aims to provide models, datasets, and tools for real-world robotics in PyTorch. The goal is to lower the barrier to entry to robotics so that everyone can contribute and benefit from sharing datasets and pretrained models.
|
||||||
|
|
||||||
|
🤗 LeRobot contains state-of-the-art approaches that have been shown to transfer to the real-world with a focus on imitation learning and reinforcement learning.
|
||||||
|
|
||||||
|
🤗 LeRobot already provides a set of pretrained models, datasets with human collected demonstrations, and simulation environments to get started without assembling a robot. In the coming weeks, the plan is to add more and more support for real-world robotics on the most affordable and capable robots out there.
|
||||||
|
|
||||||
|
🤗 LeRobot hosts pretrained models and datasets on this Hugging Face community page: [huggingface.co/lerobot](https://huggingface.co/lerobot)
|
||||||
|
|
||||||
|
#### Examples of pretrained models on simulation environments
|
||||||
|
|
||||||
|
<table>
|
||||||
|
<tr>
|
||||||
|
<td><img src="media/gym/aloha_act.gif" width="100%" alt="ACT policy on ALOHA env"/></td>
|
||||||
|
<td><img src="media/gym/simxarm_tdmpc.gif" width="100%" alt="TDMPC policy on SimXArm env"/></td>
|
||||||
|
<td><img src="media/gym/pusht_diffusion.gif" width="100%" alt="Diffusion policy on PushT env"/></td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<td align="center">ACT policy on ALOHA env</td>
|
||||||
|
<td align="center">TDMPC policy on SimXArm env</td>
|
||||||
|
<td align="center">Diffusion policy on PushT env</td>
|
||||||
|
</tr>
|
||||||
|
</table>
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
Download our source code:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/huggingface/lerobot.git
|
||||||
|
cd lerobot
|
||||||
|
```
|
||||||
|
|
||||||
|
LeRobot works with Python 3.10+ and PyTorch 2.2+.
|
||||||
|
Create a virtual environment with Python 3.10 and activate it, e.g. with [`miniconda`](https://docs.anaconda.com/free/miniconda/index.html):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
conda create -y -n lerobot python=3.10
|
||||||
|
conda activate lerobot
|
||||||
|
```
|
||||||
|
|
||||||
|
When using `miniconda`, install `ffmpeg` in your environment:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
conda install ffmpeg -c conda-forge
|
||||||
|
```
|
||||||
|
|
||||||
|
> **NOTE:** This usually installs `ffmpeg 7.X` for your platform compiled with the `libsvtav1` encoder. If `libsvtav1` is not supported (check supported encoders with `ffmpeg -encoders`), you can:
|
||||||
|
>
|
||||||
|
> - _[On any platform]_ Explicitly install `ffmpeg 7.X` using:
|
||||||
|
>
|
||||||
|
> ```bash
|
||||||
|
> conda install ffmpeg=7.1.1 -c conda-forge
|
||||||
|
> ```
|
||||||
|
>
|
||||||
|
> - _[On Linux only]_ Install [ffmpeg build dependencies](https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu#GettheDependencies) and [compile ffmpeg from source with libsvtav1](https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu#libsvtav1), and make sure you use the corresponding ffmpeg binary to your install with `which ffmpeg`.
|
||||||
|
|
||||||
|
Install 🤗 LeRobot:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install lerobot
|
||||||
|
```
|
||||||
|
|
||||||
|
> **NOTE:** If you encounter build errors, you may need to install additional dependencies (`cmake`, `build-essential`, and `ffmpeg libs`). On Linux, run:
|
||||||
|
> `sudo apt-get install cmake build-essential python3-dev pkg-config libavformat-dev libavcodec-dev libavdevice-dev libavutil-dev libswscale-dev libswresample-dev libavfilter-dev`. For other systems, see: [Compiling PyAV](https://pyav.org/docs/develop/overview/installation.html#bring-your-own-ffmpeg)
|
||||||
|
|
||||||
|
For simulations, 🤗 LeRobot comes with gymnasium environments that can be installed as extras:
|
||||||
|
|
||||||
|
- [aloha](https://github.com/huggingface/gym-aloha)
|
||||||
|
- [xarm](https://github.com/huggingface/gym-xarm)
|
||||||
|
- [pusht](https://github.com/huggingface/gym-pusht)
|
||||||
|
|
||||||
|
For instance, to install 🤗 LeRobot with aloha and pusht, use:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -e ".[aloha, pusht]"
|
||||||
|
```
|
||||||
|
|
||||||
|
To use [Weights and Biases](https://docs.wandb.ai/quickstart) for experiment tracking, log in with
|
||||||
|
|
||||||
|
```bash
|
||||||
|
wandb login
|
||||||
|
```
|
||||||
|
|
||||||
|
(note: you will also need to enable WandB in the configuration. See below.)
|
||||||
|
|
||||||
|
### Build your own robot
|
||||||
|
|
||||||
<h2 align="center">
|
<h2 align="center">
|
||||||
<p><a href="https://huggingface.co/docs/lerobot/hope_jr">
|
<p><a href="https://huggingface.co/docs/lerobot/hope_jr">
|
||||||
Build Your Own HopeJR Robot!</a></p>
|
Build Your Own HopeJR Robot!</a></p>
|
||||||
@@ -85,92 +174,6 @@
|
|||||||
<p>LeRobot: State-of-the-art AI for real-world robotics</p>
|
<p>LeRobot: State-of-the-art AI for real-world robotics</p>
|
||||||
</h3>
|
</h3>
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
🤗 LeRobot aims to provide models, datasets, and tools for real-world robotics in PyTorch. The goal is to lower the barrier to entry to robotics so that everyone can contribute and benefit from sharing datasets and pretrained models.
|
|
||||||
|
|
||||||
🤗 LeRobot contains state-of-the-art approaches that have been shown to transfer to the real-world with a focus on imitation learning and reinforcement learning.
|
|
||||||
|
|
||||||
🤗 LeRobot already provides a set of pretrained models, datasets with human collected demonstrations, and simulation environments to get started without assembling a robot. In the coming weeks, the plan is to add more and more support for real-world robotics on the most affordable and capable robots out there.
|
|
||||||
|
|
||||||
🤗 LeRobot hosts pretrained models and datasets on this Hugging Face community page: [huggingface.co/lerobot](https://huggingface.co/lerobot)
|
|
||||||
|
|
||||||
#### Examples of pretrained models on simulation environments
|
|
||||||
|
|
||||||
<table>
|
|
||||||
<tr>
|
|
||||||
<td><img src="media/gym/aloha_act.gif" width="100%" alt="ACT policy on ALOHA env"/></td>
|
|
||||||
<td><img src="media/gym/simxarm_tdmpc.gif" width="100%" alt="TDMPC policy on SimXArm env"/></td>
|
|
||||||
<td><img src="media/gym/pusht_diffusion.gif" width="100%" alt="Diffusion policy on PushT env"/></td>
|
|
||||||
</tr>
|
|
||||||
<tr>
|
|
||||||
<td align="center">ACT policy on ALOHA env</td>
|
|
||||||
<td align="center">TDMPC policy on SimXArm env</td>
|
|
||||||
<td align="center">Diffusion policy on PushT env</td>
|
|
||||||
</tr>
|
|
||||||
</table>
|
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
Download our source code:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git clone https://github.com/huggingface/lerobot.git
|
|
||||||
cd lerobot
|
|
||||||
```
|
|
||||||
|
|
||||||
Create a virtual environment with Python 3.10 and activate it, e.g. with [`miniconda`](https://docs.anaconda.com/free/miniconda/index.html):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
conda create -y -n lerobot python=3.10
|
|
||||||
conda activate lerobot
|
|
||||||
```
|
|
||||||
|
|
||||||
When using `miniconda`, install `ffmpeg` in your environment:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
conda install ffmpeg -c conda-forge
|
|
||||||
```
|
|
||||||
|
|
||||||
> **NOTE:** This usually installs `ffmpeg 7.X` for your platform compiled with the `libsvtav1` encoder. If `libsvtav1` is not supported (check supported encoders with `ffmpeg -encoders`), you can:
|
|
||||||
>
|
|
||||||
> - _[On any platform]_ Explicitly install `ffmpeg 7.X` using:
|
|
||||||
>
|
|
||||||
> ```bash
|
|
||||||
> conda install ffmpeg=7.1.1 -c conda-forge
|
|
||||||
> ```
|
|
||||||
>
|
|
||||||
> - _[On Linux only]_ Install [ffmpeg build dependencies](https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu#GettheDependencies) and [compile ffmpeg from source with libsvtav1](https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu#libsvtav1), and make sure you use the corresponding ffmpeg binary to your install with `which ffmpeg`.
|
|
||||||
|
|
||||||
Install 🤗 LeRobot:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install -e .
|
|
||||||
```
|
|
||||||
|
|
||||||
> **NOTE:** If you encounter build errors, you may need to install additional dependencies (`cmake`, `build-essential`, and `ffmpeg libs`). On Linux, run:
|
|
||||||
> `sudo apt-get install cmake build-essential python3-dev pkg-config libavformat-dev libavcodec-dev libavdevice-dev libavutil-dev libswscale-dev libswresample-dev libavfilter-dev`. For other systems, see: [Compiling PyAV](https://pyav.org/docs/develop/overview/installation.html#bring-your-own-ffmpeg)
|
|
||||||
|
|
||||||
For simulations, 🤗 LeRobot comes with gymnasium environments that can be installed as extras:
|
|
||||||
|
|
||||||
- [aloha](https://github.com/huggingface/gym-aloha)
|
|
||||||
- [xarm](https://github.com/huggingface/gym-xarm)
|
|
||||||
- [pusht](https://github.com/huggingface/gym-pusht)
|
|
||||||
|
|
||||||
For instance, to install 🤗 LeRobot with aloha and pusht, use:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install -e ".[aloha, pusht]"
|
|
||||||
```
|
|
||||||
|
|
||||||
To use [Weights and Biases](https://docs.wandb.ai/quickstart) for experiment tracking, log in with
|
|
||||||
|
|
||||||
```bash
|
|
||||||
wandb login
|
|
||||||
```
|
|
||||||
|
|
||||||
(note: you will also need to enable WandB in the configuration. See below.)
|
|
||||||
|
|
||||||
### Visualize datasets
|
### Visualize datasets
|
||||||
|
|
||||||
Check out [example 1](./examples/1_load_lerobot_dataset.py) that illustrates how to use our dataset class which automatically downloads data from the Hugging Face hub.
|
Check out [example 1](./examples/1_load_lerobot_dataset.py) that illustrates how to use our dataset class which automatically downloads data from the Hugging Face hub.
|
||||||
|
|||||||
Reference in New Issue
Block a user