d9b9cc80da
fix(rebase) reverting files to main
2025-07-02 11:47:07 +02:00
0a390de361
Merge remote-tracking branch 'origin/main' into user/rcadene/2025_04_11_dataset_v3
2025-07-02 11:41:53 +02:00
d4ee470b00
Package folder structure ( #1417 )
...
* Move files
* Replace imports & paths
* Update relative paths
* Update doc symlinks
* Update instructions paths
* Fix imports
* Update grpc files
* Update more instructions
* Downgrade grpc-tools
* Update manifest
* Update more paths
* Update config paths
* Update CI paths
* Update bandit exclusions
* Remove walkthrough section
2025-07-01 16:34:46 +02:00
f3d931e1b2
Add direct access to action chunks ( #1020 )
...
* fix: sharing predicted chunk with user
* [pre-commit.ci] pre-commit autoupdate (#1011 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Revert "[pre-commit.ci] pre-commit autoupdate" (#1025 )
* fix(ci): Pin draccus (<0.10.0) and torch (<2.7) to fix pipeline (#1022 )
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
* fix(ci): Pin `torchcodec` (==0.2.1) to fix pipeline temporarly (#1030 )
* Update tutorial (#1021 )
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
* Add description motor order SO-101 leader (#1051 )
* feat(encoding): switching to PyAV for ffmpeg related tasks (#983 )
* feat(docs): Add new docs build process (#1046 )
Co-authored-by: Mishig Davaadorj <dmishig@gmail.com >
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
* Docs: adapt text + fix video code (#1064 )
* Fix typos (#1070 )
* docs: minor corrections and clean-up (#1089 )
* Update 10_use_so100.md; use diff syntax (#944 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
* Update 12_use_so101.md (#1081 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
* bug fix for #1071 When --display_data=true, Failed running control_robot. (#1073 )
* Add editable -e for feetech install command (#1133 )
* Fix: emptying action queue between resets (#1117 )
* fix: typos and grammar (#1148 )
* Update README.md (#1160 )
* Update README.md (#1163 )
* [Fix] Unpin torch beyond 2.6.0 & torchcodec beyond 0.2.1 (#1127 )
* (hotfix): nightly CI by clipping pymunk version below 7.0.0 (#1182 )
* [pre-commit.ci] pre-commit autoupdate (#1048 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
* Add SmolVLA (#1175 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: fracapuano <francesco.capuano@huggingface.co >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: Dana Aubakirova <118912928+danaaubakirova@users.noreply.github.com >
Co-authored-by: Remi <remi.cadene@huggingface.co >
* Fix SmolVLA loss not sent to wandb (#1198 )
* Hardware API redesign (#777 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
Co-authored-by: Pepijn <pepijn@huggingface.co >
* fix(smolvla): update record.py, fix populate_queues and remove unused dependencies (#1208 )
* replaced OBS_ROBOT with OBS_STATE constant (#1211 )
* Fix test_teleoperate (#1216 )
* Fix LeKiwi example (#1217 )
* Fix smolVLA dependencies (#1218 )
* fix(pyserial): adding pyserial dependency to global ones (#1219 )
* Update SmolVLA README.md (#1228 )
* Fix unable to set camera width/height to non-default (#1225 )
* Update tutorial link (#1250 )
* update KochFollower.get_observation() so it returns same observation structure as SO101 (#1248 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [pre-commit.ci] pre-commit autoupdate (#1185 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
* Proposal for fix for enter_pressed on Windows (#1230 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
* fix: update pi0 dependency version constraint (#1247 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Match motor names with ids lekiwi (#1261 )
* fix issues: checkpoints keys mismatch and 'task' tokenisation in smolvla (#1256 )
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
* fix(docs): update realsense documentation (#1268 )
* Use HF Papers (#1120 )
* Skip normalization parameters in load_smolvla (#1274 )
* fix(record): no teleop needed when running with policy (#1284 )
* Port HIL SERL (#644 )
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Ke Wang <superwk1017@gmail.com >
Co-authored-by: Yoel Chornton <yoel.chornton@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
* fix(docs): SmolVLA fine-tuning getting started (#1201 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Francesco Capuano <francesco_capuano@aol.com >
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
* chore(teleop): print calibration path saved (#1286 )
* chore(dependencies): add gamepad support with pygame and hidapi (#1287 )
* Robot integration tutorial (#1285 )
* fix(docs): update send_feedback docstrings
* Add sim tutorial, fix lekiwi motor config, add notebook links (#1275 )
Co-authored-by: AdilZouitine <adilzouitinegm@gmail.com >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Michel Aractingi <michel.aractingi@gmail.com >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
* Fixes on robot integration tutorial (#1290 )
* Add keyboard teleop device to control the end effector robot (#1289 )
* Improve type hints (#1293 )
* fix(record): no teleop arg in reset environment (#1294 )
* `learner.py` import so101_leader instead of so100 (#1295 )
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
* Fixing `PI0` Policy (#1297 )
* `gym_manipulator.py` Remove None value action_intervention of BaseLeaderTeleoperator (#1299 )
* (chore): incorrect resume parameter in recording documentation (#1301 )
* Update lekiwi.mdx (#1229 )
* bump `pi0` and `hil` transformers version (#1298 )
* docs: fix imitation learning robots docs command (#1308 )
* fix(benchmarks): remove .numpy() from frame in benchmark script (#1354 )
* add smolvla to the supported policies to run tests (:
* add: chunk-level access for the policy
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* add: smolvla in availables
* remove: smolvla from library supported policies
* fix: change env for training, xarm is broken as of now
* add: predict_action_chunk to all supported policies
* fix: add robot type constants
* add: predict action chunk in base policy class
* restore original Makefile
* fix: minor
* fix: dict keys come from lerobot/constants
* fix: improve act encapsulation, properly supporting temporal ensembling
* fix: smolvla action chunking
* fix: very minor, but very annoying
* fix: minor
* fix minor naming
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Signed-off-by: Francesco Capuano <74058581+fracapuano@users.noreply.github.com >
* fix: refactoring inference for single actions and chunks into different components
* fix: minor
* fix: temporal ensembling
* fix: moving populate queues out of modular component for batch preparation
* fix: minor for CI
* fix: smovla debug
* fix: reward classifier, maybe the last policy lacking?
---------
Signed-off-by: Francesco Capuano <74058581+fracapuano@users.noreply.github.com >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: Caroline Pascal <caroline8.pascal@gmail.com >
Co-authored-by: Mishig Davaadorj <dmishig@gmail.com >
Co-authored-by: omahs <73983677+omahs@users.noreply.github.com >
Co-authored-by: CharlesCNorton <135471798+CharlesCNorton@users.noreply.github.com >
Co-authored-by: masato-ka <jp6uzv@gmail.com >
Co-authored-by: Ragnar <rodiondenmark@gmail.com >
Co-authored-by: mshukor <mustafa.shukor97@gmail.com >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: Dana Aubakirova <118912928+danaaubakirova@users.noreply.github.com >
Co-authored-by: Remi <remi.cadene@huggingface.co >
Co-authored-by: Ben Zhang <5977478+ben-z@users.noreply.github.com >
Co-authored-by: Pepijn <pepijn@huggingface.co >
Co-authored-by: Dhruva <51377003+utterwqlnut@users.noreply.github.com >
Co-authored-by: Daisuke Sato <tiryoh@gmail.com >
Co-authored-by: Sarunas Kalade <sarunas.kalade@amd.com >
Co-authored-by: koenvanwijk <koenvanwijk@users.noreply.github.com >
Co-authored-by: Yushun Xiang <73413365+YushunXiang@users.noreply.github.com >
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com >
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Ke Wang <superwk1017@gmail.com >
Co-authored-by: Yoel Chornton <yoel.chornton@gmail.com >
Co-authored-by: Michel Aractingi <michel.aractingi@gmail.com >
Co-authored-by: tidely <43219534+tidely@users.noreply.github.com >
Co-authored-by: David <17435126+DavidLMS@users.noreply.github.com >
2025-06-27 10:19:19 +02:00
0b2285d1ec
Feat: Improve hub integration ( #1382 )
...
* feat(policies): Initial setup to push policies to hub with tags and model card
* feat: add dataset that is used to train
* Add model template summary
* fix: Update link model_card template
* fix: remove print
* fix: change import name
* fix: add model summary in template
* fix: minor text
* fix: comments Lucain
* fix: feedback steven
* fix: restructure push to hub
* fix: remove unneeded changes
* fix: import
* fix: import 2
* Add MANIFEST.in
* fix: feedback pr
* Fix tests
* tests: Add smolvla end-to-end test
* Fix: smolvla test
* fix test name
* fix policy tests
* Add push to hub false policy tests
* Do push to hub cleaner
* fix(ci): add push_to_hub false in tests
---------
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
2025-06-26 14:36:16 +02:00
ce6a26deeb
Fixing PI0 Policy ( #1297 )
2025-06-14 19:25:50 +02:00
d8079587a2
Port HIL SERL ( #644 )
...
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Ke Wang <superwk1017@gmail.com >
Co-authored-by: Yoel Chornton <yoel.chornton@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
2025-06-13 13:15:47 +02:00
5c87365cc1
Skip normalization parameters in load_smolvla ( #1274 )
2025-06-13 11:06:45 +02:00
edfebd522c
Use HF Papers ( #1120 )
2025-06-12 09:58:59 +02:00
d0521189b1
fix issues: checkpoints keys mismatch and 'task' tokenisation in smolvla ( #1256 )
...
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
2025-06-11 16:56:55 +02:00
459c95197b
fix: update pi0 dependency version constraint ( #1247 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-06-10 18:46:41 +02:00
a28f02ecb3
replaced OBS_ROBOT with OBS_STATE constant ( #1211 )
2025-06-06 09:25:51 +02:00
09343acce7
fix(smolvla): update record.py, fix populate_queues and remove unused dependencies ( #1208 )
2025-06-06 09:17:02 +02:00
e23b41e79a
Hardware API redesign ( #777 )
...
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
Co-authored-by: Pepijn <pepijn@huggingface.co >
2025-06-05 17:48:43 +02:00
b536f47e3f
Fix SmolVLA loss not sent to wandb ( #1198 )
2025-06-05 11:13:03 +02:00
bfd26eef5a
Add SmolVLA ( #1175 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: fracapuano <francesco.capuano@huggingface.co >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: Dana Aubakirova <118912928+danaaubakirova@users.noreply.github.com >
Co-authored-by: Remi <remi.cadene@huggingface.co >
2025-06-03 17:11:50 +02:00
bed90e3a41
fix: typos and grammar ( #1148 )
2025-05-25 17:20:45 +02:00
eab5543750
Merge (No verify)
2025-04-17 04:46:09 +02:00
4041f57943
feat(visualization): replace cv2 GUI with Rerun (and solves ffmpeg versioning issues) ( #903 )
2025-04-09 17:33:01 +02:00
1c873df5c0
Support for PI0+FAST ( #921 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Dana Aubakirova <118912928+danaaubakirova@users.noreply.github.com >
Co-authored-by: Remi <re.cadene@gmail.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
2025-04-04 11:51:11 +02:00
78fd1a1e04
chore(docs): update docs ( #911 )
2025-03-27 09:55:06 +01:00
a774af2eab
fix pi0 action padding name ( #893 )
...
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
2025-03-25 11:24:46 +01:00
e8159997c7
User/pepijn/2025 03 17 act different image shapes ( #870 )
2025-03-18 11:09:05 +01:00
5e9473806c
refactor(config): Move device & amp args to PreTrainedConfig ( #812 )
...
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
2025-03-06 17:59:28 +01:00
5d24ce3160
chore(doc): add license header to all files ( #818 )
2025-03-05 17:56:51 +01:00
b80e55ca44
change "actions_id_pad" to "actions_is_pad"( 🐛 Bug) ( #774 )
...
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
2025-03-05 01:31:56 +01:00
a1809ad3de
Add typos checks ( #770 )
2025-02-25 23:51:15 +01:00
3354d919fc
LeRobotDataset v2.1 ( #711 )
...
Co-authored-by: Remi <remi.cadene@huggingface.co >
Co-authored-by: Remi Cadene <re.cadene@gmail.com >
2025-02-25 15:27:29 +01:00
c4c2ce04e7
Update pre-commits ( #733 )
2025-02-15 15:51:17 +01:00
e71095960f
Fixes following #670 ( #719 )
2025-02-12 12:53:55 +01:00
90e099b39f
Remove offline training, refactor train.py and logging/checkpointing ( #670 )
...
Co-authored-by: Remi <remi.cadene@huggingface.co >
2025-02-11 10:36:06 +01:00
638d411cd3
Add Pi0 ( #681 )
...
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Pablo <pablo.montalvo.leroux@gmail.com >
2025-02-04 18:01:04 +01:00
3c0a209f9f
Simplify configs ( #550 )
...
Co-authored-by: Remi <remi.cadene@huggingface.co >
Co-authored-by: HUANG TZU-CHUN <137322177+tc-huang@users.noreply.github.com >
2025-01-31 13:57:37 +01:00
538455a965
feat: enable to use multiple rgb encoders per camera in diffusion policy ( #484 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com >
2024-10-30 11:00:05 +01:00
a60d27b132
Raise ValueError if horizon is incompatible with downsampling ( #422 )
2024-09-09 17:22:46 +01:00
f17d9a2ba1
Bug: Fix VQ-Bet not working when n_action_pred_token=1 ( #420 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com >
2024-09-09 09:41:13 +01:00
b2896d38f5
fix(act): n_vae_encoder_layers config parameter wasn't being used ( #400 )
2024-09-02 18:29:27 +01:00
86bbd16d43
Improve discoverability on the hub ( #325 )
...
Co-authored-by: Lucain <lucainp@gmail.com >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
2024-08-19 15:16:46 +02:00
0f6e0f6d74
Fix input dim ( #365 )
2024-08-19 11:42:32 +01:00
7a3cb1ad34
Adjust the timestamps' description in Diffusion Policy ( #343 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com >
2024-07-26 12:47:03 +01:00
f8a6574698
Add online training with TD-MPC as proof of concept ( #338 )
2024-07-25 11:16:38 +01:00
abbb1d2367
Make sure policies don't mutate the batch ( #323 )
2024-07-22 20:38:33 +01:00
c0101f0948
Fix ACT temporal ensembling ( #319 )
2024-07-16 10:27:21 +01:00
471eab3d7e
Make ACT compatible with "observation.environment_state" ( #314 )
2024-07-11 13:12:22 +01:00
64425d5e00
Bug fix: fix error when setting select_target_actions_indices in vqbet ( #310 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com >
2024-07-10 17:56:11 +01:00
cc2f6e7404
Train diffusion pusht_keypoints ( #307 )
...
Co-authored-by: Remi <re.cadene@gmail.com >
2024-07-09 12:35:50 +01:00
74362ac453
Add VQ-BeT copyrights ( #299 )
2024-07-04 13:02:31 +02:00
342f429f1c
Add test to make sure policy dataclass configs match yaml configs ( #292 )
2024-06-26 09:09:40 +01:00
7d1542cae1
Add VQ-BeT ( #166 )
2024-06-26 08:55:02 +01:00
48951662f2
Bug fix: missing attention mask in VAE encoder in ACT policy ( #279 )
...
Co-authored-by: Alexander Soare <alexander.soare159@gmail.com >
2024-06-19 12:07:21 +01:00
Michel Aractingi