admin/lerobot
1
0
Fork 0
mirror of https://github.com/huggingface/lerobot.git synced 2026-05-15 00:29:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(policy): multi dit docs (#3285)

Browse Source 
* docs(policy): add libero results multi task dit + remove readme in src code

* docs(policy): add hyperlink to doc file in src code

* chore(style): pre-commit
This commit is contained in:
Steven Palma
2026-04-05 21:23:13 +02:00
committed by GitHub
parent 8c3d4cf900
commit d60a700d2b
2 changed files with 49 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/source/multi_task_dit.mdx
+48
View File
@@ -331,6 +331,54 @@ lerobot-train \
--wandb.project=multitask_dit
```
## Libero Results
```
python -m lerobot.scripts.lerobot_train \
--dataset.repo_id=HuggingFaceVLA/libero \
--policy.type=multi_task_dit \
--policy.push_to_hub=false \
--output_dir="./outputs/multitask_dit_libero" \
--job_name="multitask-dit-libero" \
--wandb.enable=true \
--wandb.project=multitask_dit_libero \
--dataset.image_transforms.enable=true \
--dataset.image_transforms.max_num_transforms=4 \
--dataset.image_transforms.tfs='{"brightness":{"type":"ColorJitter","kwargs":{"brightness":[0.75,1.25]}},"contrast":{"type":"ColorJitter","kwargs":{"contrast":[0.6,1.4]}},"saturation":{"type":"ColorJitter","kwargs":{"saturation":[0.8,1.2]}},"hue":{"type":"ColorJitter","kwargs":{"hue":[-0.05,0.05]}},"sharpness":{"type":"SharpnessJitter","kwargs":{"sharpness":[0.6,1.4]}},"rotation":{"type":"RandomRotation","kwargs":{"degrees":[-5,5]}},"translation":{"type":"RandomAffine","kwargs":{"degrees":0,"translate":[0.1,0.1]}}}' \
--dataset.video_backend=torchcodec \
--policy.use_amp=true \
--policy.horizon=48 \
--policy.n_obs_steps=2 \
--policy.use_rope=true \
--policy.use_positional_encoding=false \
--policy.hidden_dim=768 \
--policy.num_layers=8 \
--policy.num_heads=12 \
--policy.dropout=0.1 \
--policy.timestep_embed_dim=256 \
--policy.objective=diffusion \
--policy.optimizer_lr=3e-4 \
--policy.optimizer_weight_decay=0 \
--policy.scheduler_warmup_steps=0 \
--policy.vision_encoder_name=openai/clip-vit-base-patch16 \
--policy.image_resize_shape=[256,256] \
--policy.image_crop_is_random=true \
--policy.text_encoder_name=openai/clip-vit-base-patch16 \
--policy.vision_encoder_lr_multiplier=0.1 \
--policy.device=cuda \
--num_workers=8 \
--save_freq=4000 \
--log_freq=100 \
--steps=100000 \
--batch_size=320
```
Results:
| LIBERO Spatial | LIBERO Object | LIBERO Goal | LIBERO 10 | Average |
| -------------- | ------------- | ----------- | --------- | ------- |
| 87.0 | 98.2 | 93.8 | 83.2 | 90.6 |
## References
For more details on the technical implementation and architecture, see: