Remove open pi patch and use custom transformer branch for now

This commit is contained in:
Pepijn
2025-09-23 17:03:23 +02:00
parent 969e8eeae1
commit d71f0d4a05
16 changed files with 60 additions and 6112 deletions
+41 -1
View File
@@ -127,4 +127,44 @@ LeRobot uses MuJoCo for simulation. You need to set the rendering backend before
## Reproducing π₀ and π₀.₅ results
We can also reproduce the results of π₀ and π₀.₅ on the Libero benchmark by using the finetuned libero models.
We reproduce the results of π₀ and π₀.₅ on the LIBERO benchmark using the LeRobot implementation. We take the Physical Intelligence LIBERO base models (`pi0_libero` and `pi05_libero`) and finetune them for an additional 6k steps in bfloat16, with batch size of 256 on 8 H100 GPUs using the [HuggingFace LIBERO dataset](https://huggingface.co/datasets/HuggingFaceVLA/libero).
The finetuned models can be found here:
- **π₀ LIBERO**: [pepijn223/pi0_libero_fp32](https://huggingface.co/pepijn223/pi0_libero_fp32)
- **π₀.₅ LIBERO**: [pepijn223/pi05_libero_fp32](https://huggingface.co/pepijn223/pi05_libero_fp32)
We then evaluate the finetuned models using the LeRobot LIBERO implementation, by running the following command:
```bash
python src/lerobot/scripts/eval.py \
--output_dir=/logs/ \
--env.type=libero \
--env.task=libero_spatial,libero_object,libero_goal,libero_10 \
--eval.batch_size=1 \
--eval.n_episodes=5 \
--policy.path=pepijn223/pi0_libero_fp32 \
--env.multitask_eval=true \
--output_dir=./eval_logs/ \
--policy.compile_model=false \
--policy.gradient_checkpointing=false \
--env.max_parallel_tasks=1
```
**Note:** We set `n_action_steps=10` which normally has a default value of 50.
### Results
We obtain the following results on the LIBERO benchmark:
| Model | LIBERO Spatial | LIBERO Object | LIBERO Goal | LIBERO 10 | Average |
| -------- | -------------- | ------------- | ----------- | --------- | ------- |
| **π₀** | x | x | x | x | **x** |
| **π₀.₅** | 98.0 | 99.0 | 97.0 | 93.0 | **x** |
These results are consistent with the original results reported by Physical Intelligence:
| Model | LIBERO Spatial | LIBERO Object | LIBERO Goal | LIBERO 10 | Average |
| -------- | -------------- | ------------- | ----------- | --------- | --------- |
| **π₀** | 96.8 | 98.8 | 95.8 | 85.2 | **94.15** |
| **π₀.₅** | 98.8 | 98.2 | 98.0 | 92.4 | **96.85** |
+3 -34
View File
@@ -21,44 +21,13 @@ As described by Physical Intelligence, while AI has achieved remarkable success
## Installation Requirements
⚠️ **Warning**: This policy requires patching the Hugging Face `transformers` library.
### Prerequisites
1. Ensure you have the exact version installed:
1. Install LeRobot by following our [Installation Guide](./installation).
2. Install Pi0 dependencies by running:
```bash
pip show transformers
pip install -e ".[pi]"
```
It must be version **4.53.2**.
2. Apply the custom patches:
```bash
cp -r ./src/lerobot/policies/pi0/transformers_replace/* \
$(python -c "import transformers, os; print(os.path.dirname(transformers.__file__))")
```
### What the patches do:
- Support the **AdaRMS optimizer**
- Correctly control the precision of activations
- Allow the KV cache to be used without updates
**Important Notes:**
- This permanently modifies your `transformers` installation
- The changes survive reinstalls unless you explicitly remove the patched files or recreate the environment
### Restoring Clean State
To undo the patches and restore a clean state:
```bash
pip uninstall transformers
pip install transformers==4.53.2
```
## Training Data and Capabilities
π₀ is trained on the largest robot interaction dataset to date, combining three key data sources:
+3 -14
View File
@@ -29,22 +29,11 @@ This diverse training mixture creates a "curriculum" that enables generalization
## Installation Requirements
⚠️ **Warning**: This policy requires patching the Hugging Face `transformers` library.
### Prerequisites
1. Ensure you have the exact version installed:
1. Install LeRobot by following our [Installation Guide](./installation).
2. Install Pi0.5 dependencies by running:
```bash
pip show transformers
```
It must be version **4.53.2**.
2. Apply the custom patches:
```bash
cp -r ./src/lerobot/policies/pi05/transformers_replace/* \
$(python -c "import transformers, os; print(os.path.dirname(transformers.__file__))")
pip install -e ".[pi]"
```
### What the patches do: