mirror of
https://github.com/huggingface/lerobot.git
synced 2026-05-16 17:20:05 +00:00
Remove open pi patch and use custom transformer branch for now
This commit is contained in:
+41
-1
@@ -127,4 +127,44 @@ LeRobot uses MuJoCo for simulation. You need to set the rendering backend before
|
||||
|
||||
## Reproducing π₀ and π₀.₅ results
|
||||
|
||||
We can also reproduce the results of π₀ and π₀.₅ on the Libero benchmark by using the finetuned libero models.
|
||||
We reproduce the results of π₀ and π₀.₅ on the LIBERO benchmark using the LeRobot implementation. We take the Physical Intelligence LIBERO base models (`pi0_libero` and `pi05_libero`) and finetune them for an additional 6k steps in bfloat16, with batch size of 256 on 8 H100 GPUs using the [HuggingFace LIBERO dataset](https://huggingface.co/datasets/HuggingFaceVLA/libero).
|
||||
|
||||
The finetuned models can be found here:
|
||||
|
||||
- **π₀ LIBERO**: [pepijn223/pi0_libero_fp32](https://huggingface.co/pepijn223/pi0_libero_fp32)
|
||||
- **π₀.₅ LIBERO**: [pepijn223/pi05_libero_fp32](https://huggingface.co/pepijn223/pi05_libero_fp32)
|
||||
|
||||
We then evaluate the finetuned models using the LeRobot LIBERO implementation, by running the following command:
|
||||
|
||||
```bash
|
||||
python src/lerobot/scripts/eval.py \
|
||||
--output_dir=/logs/ \
|
||||
--env.type=libero \
|
||||
--env.task=libero_spatial,libero_object,libero_goal,libero_10 \
|
||||
--eval.batch_size=1 \
|
||||
--eval.n_episodes=5 \
|
||||
--policy.path=pepijn223/pi0_libero_fp32 \
|
||||
--env.multitask_eval=true \
|
||||
--output_dir=./eval_logs/ \
|
||||
--policy.compile_model=false \
|
||||
--policy.gradient_checkpointing=false \
|
||||
--env.max_parallel_tasks=1
|
||||
```
|
||||
|
||||
**Note:** We set `n_action_steps=10` which normally has a default value of 50.
|
||||
|
||||
### Results
|
||||
|
||||
We obtain the following results on the LIBERO benchmark:
|
||||
|
||||
| Model | LIBERO Spatial | LIBERO Object | LIBERO Goal | LIBERO 10 | Average |
|
||||
| -------- | -------------- | ------------- | ----------- | --------- | ------- |
|
||||
| **π₀** | x | x | x | x | **x** |
|
||||
| **π₀.₅** | 98.0 | 99.0 | 97.0 | 93.0 | **x** |
|
||||
|
||||
These results are consistent with the original results reported by Physical Intelligence:
|
||||
|
||||
| Model | LIBERO Spatial | LIBERO Object | LIBERO Goal | LIBERO 10 | Average |
|
||||
| -------- | -------------- | ------------- | ----------- | --------- | --------- |
|
||||
| **π₀** | 96.8 | 98.8 | 95.8 | 85.2 | **94.15** |
|
||||
| **π₀.₅** | 98.8 | 98.2 | 98.0 | 92.4 | **96.85** |
|
||||
|
||||
+3
-34
@@ -21,44 +21,13 @@ As described by Physical Intelligence, while AI has achieved remarkable success
|
||||
|
||||
## Installation Requirements
|
||||
|
||||
⚠️ **Warning**: This policy requires patching the Hugging Face `transformers` library.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
1. Ensure you have the exact version installed:
|
||||
1. Install LeRobot by following our [Installation Guide](./installation).
|
||||
2. Install Pi0 dependencies by running:
|
||||
|
||||
```bash
|
||||
pip show transformers
|
||||
pip install -e ".[pi]"
|
||||
```
|
||||
|
||||
It must be version **4.53.2**.
|
||||
|
||||
2. Apply the custom patches:
|
||||
```bash
|
||||
cp -r ./src/lerobot/policies/pi0/transformers_replace/* \
|
||||
$(python -c "import transformers, os; print(os.path.dirname(transformers.__file__))")
|
||||
```
|
||||
|
||||
### What the patches do:
|
||||
|
||||
- Support the **AdaRMS optimizer**
|
||||
- Correctly control the precision of activations
|
||||
- Allow the KV cache to be used without updates
|
||||
|
||||
**Important Notes:**
|
||||
|
||||
- This permanently modifies your `transformers` installation
|
||||
- The changes survive reinstalls unless you explicitly remove the patched files or recreate the environment
|
||||
|
||||
### Restoring Clean State
|
||||
|
||||
To undo the patches and restore a clean state:
|
||||
|
||||
```bash
|
||||
pip uninstall transformers
|
||||
pip install transformers==4.53.2
|
||||
```
|
||||
|
||||
## Training Data and Capabilities
|
||||
|
||||
π₀ is trained on the largest robot interaction dataset to date, combining three key data sources:
|
||||
|
||||
+3
-14
@@ -29,22 +29,11 @@ This diverse training mixture creates a "curriculum" that enables generalization
|
||||
|
||||
## Installation Requirements
|
||||
|
||||
⚠️ **Warning**: This policy requires patching the Hugging Face `transformers` library.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
1. Ensure you have the exact version installed:
|
||||
1. Install LeRobot by following our [Installation Guide](./installation).
|
||||
2. Install Pi0.5 dependencies by running:
|
||||
|
||||
```bash
|
||||
pip show transformers
|
||||
```
|
||||
|
||||
It must be version **4.53.2**.
|
||||
|
||||
2. Apply the custom patches:
|
||||
```bash
|
||||
cp -r ./src/lerobot/policies/pi05/transformers_replace/* \
|
||||
$(python -c "import transformers, os; print(os.path.dirname(transformers.__file__))")
|
||||
pip install -e ".[pi]"
|
||||
```
|
||||
|
||||
### What the patches do:
|
||||
|
||||
Reference in New Issue
Block a user