mirror of
https://github.com/huggingface/lerobot.git
synced 2026-07-06 17:41:47 +00:00
Maxime Ellerbach
141c353206
* Add FastWAM policy * Add FastWAM policy review updates * big refactor to use models from diffusers and transformers * changing reproducable results * preparing for training adding some temporary debug code aswell to visualize model output * re-parenting of some layers to enable proper zero-3 FSDP * linting * small fix for the preprocessor and padded images * removing some preprocessors * removing temporary debug code * cleaning up * updating uv lock after rebasing * adding lazy imports * linting * fixing stale assertion * make tokenizer/text-encoder model ids configurable + some nits * moving and renaming files to have a cleaner file tree * removed asserts from the model, added guard instead and completely removed useless asserts * cleaning up imports * removing is_main_process and custom logging logic * removing unused / stale attention path, removing some of the stale forwards within wan/models --------- Co-authored-by: ZibinDong <zibindong@outlook.com> Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
57 lines
1.9 KiB
Markdown
57 lines
1.9 KiB
Markdown
## Research Paper
|
|
|
|
Paper: https://arxiv.org/abs/2603.16666
|
|
|
|
## Repository
|
|
|
|
Code: https://github.com/yuantianyuan01/FastWAM
|
|
|
|
Project page: https://yuantianyuan01.github.io/FastWAM/
|
|
|
|
## Citation
|
|
|
|
```bibtex
|
|
@article{yuan2026fastwam,
|
|
title = {Fast-WAM: Do World Action Models Need Test-time Future Imagination?},
|
|
author = {Tianyuan Yuan and Zibin Dong and Yicheng Liu and Hang Zhao},
|
|
journal = {arXiv preprint arXiv:2603.16666},
|
|
year = {2026},
|
|
url = {https://arxiv.org/abs/2603.16666}
|
|
}
|
|
```
|
|
|
|
## Additional Resources
|
|
|
|
Base video model: https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B
|
|
|
|
Released upstream checkpoints: https://huggingface.co/yuanty/fastwam
|
|
|
|
## Results
|
|
|
|
Evaluated on LIBERO with [`ZibinDong/fastwam_libero_uncond_2cam224`](https://huggingface.co/ZibinDong/fastwam_libero_uncond_2cam224):
|
|
|
|
| Suite | Success rate | n_episodes |
|
|
| -------------- | -----------: | ---------: |
|
|
| libero_spatial | 97.6% | 500 |
|
|
| libero_object | 99.0% | 500 |
|
|
| libero_goal | 95.0% | 500 |
|
|
| libero_10 | 94.0% | 500 |
|
|
| **average** | **96.4%** | 2000 |
|
|
|
|
Reproduce: `lerobot-eval --policy.path=ZibinDong/fastwam_libero_uncond_2cam224 --policy.device=cuda --policy.torch_dtype=float32 --policy.n_action_steps=10 --env.type=libero --env.task=libero_spatial --env.observation_height=256 --env.observation_width=256 --eval.batch_size=1 --eval.n_episodes=50 --seed=0 --env.episode_length=300`.
|
|
|
|
For LIBERO-10, use `--env.task=libero_10 --env.episode_length=600`:
|
|
|
|
```bash
|
|
lerobot-eval \
|
|
--policy.path=ZibinDong/fastwam_libero_uncond_2cam224 \
|
|
--policy.device=cuda \
|
|
--policy.torch_dtype=float32 \
|
|
--policy.n_action_steps=10 \
|
|
--env.type=libero \
|
|
--env.task=libero_10 --env.observation_height=256 --env.observation_width=256 \
|
|
--eval.batch_size=1 \
|
|
--eval.n_episodes=50 \
|
|
--seed=0 --env.episode_length=600
|
|
```
|