44245e714e
Merge branch 'main' of github.com:huggingface/lerobot into feature/basic-peft-support
2025-06-27 13:55:07 +02:00
2f9ba4e2cc
Add api examples IL docs ( #1391 )
...
* feat: add api examples for record, replay, eval for il
* fix: Add typings utils.py
* fix: Add inference to text eval
* fix: Add placeholders dataset and policy repo_ids
* fix: Improve text
* fix: Add type to 3rd ;)
* chore(docs): update API examples for replay, eval and record
---------
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
2025-06-27 11:57:24 +02:00
f3d931e1b2
Add direct access to action chunks ( #1020 )
...
* fix: sharing predicted chunk with user
* [pre-commit.ci] pre-commit autoupdate (#1011 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Revert "[pre-commit.ci] pre-commit autoupdate" (#1025 )
* fix(ci): Pin draccus (<0.10.0) and torch (<2.7) to fix pipeline (#1022 )
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
* fix(ci): Pin `torchcodec` (==0.2.1) to fix pipeline temporarly (#1030 )
* Update tutorial (#1021 )
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
* Add description motor order SO-101 leader (#1051 )
* feat(encoding): switching to PyAV for ffmpeg related tasks (#983 )
* feat(docs): Add new docs build process (#1046 )
Co-authored-by: Mishig Davaadorj <dmishig@gmail.com >
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
* Docs: adapt text + fix video code (#1064 )
* Fix typos (#1070 )
* docs: minor corrections and clean-up (#1089 )
* Update 10_use_so100.md; use diff syntax (#944 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
* Update 12_use_so101.md (#1081 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
* bug fix for #1071 When --display_data=true, Failed running control_robot. (#1073 )
* Add editable -e for feetech install command (#1133 )
* Fix: emptying action queue between resets (#1117 )
* fix: typos and grammar (#1148 )
* Update README.md (#1160 )
* Update README.md (#1163 )
* [Fix] Unpin torch beyond 2.6.0 & torchcodec beyond 0.2.1 (#1127 )
* (hotfix): nightly CI by clipping pymunk version below 7.0.0 (#1182 )
* [pre-commit.ci] pre-commit autoupdate (#1048 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
* Add SmolVLA (#1175 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: fracapuano <francesco.capuano@huggingface.co >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: Dana Aubakirova <118912928+danaaubakirova@users.noreply.github.com >
Co-authored-by: Remi <remi.cadene@huggingface.co >
* Fix SmolVLA loss not sent to wandb (#1198 )
* Hardware API redesign (#777 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
Co-authored-by: Pepijn <pepijn@huggingface.co >
* fix(smolvla): update record.py, fix populate_queues and remove unused dependencies (#1208 )
* replaced OBS_ROBOT with OBS_STATE constant (#1211 )
* Fix test_teleoperate (#1216 )
* Fix LeKiwi example (#1217 )
* Fix smolVLA dependencies (#1218 )
* fix(pyserial): adding pyserial dependency to global ones (#1219 )
* Update SmolVLA README.md (#1228 )
* Fix unable to set camera width/height to non-default (#1225 )
* Update tutorial link (#1250 )
* update KochFollower.get_observation() so it returns same observation structure as SO101 (#1248 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [pre-commit.ci] pre-commit autoupdate (#1185 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
* Proposal for fix for enter_pressed on Windows (#1230 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
* fix: update pi0 dependency version constraint (#1247 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Match motor names with ids lekiwi (#1261 )
* fix issues: checkpoints keys mismatch and 'task' tokenisation in smolvla (#1256 )
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
* fix(docs): update realsense documentation (#1268 )
* Use HF Papers (#1120 )
* Skip normalization parameters in load_smolvla (#1274 )
* fix(record): no teleop needed when running with policy (#1284 )
* Port HIL SERL (#644 )
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Ke Wang <superwk1017@gmail.com >
Co-authored-by: Yoel Chornton <yoel.chornton@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
* fix(docs): SmolVLA fine-tuning getting started (#1201 )
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Francesco Capuano <francesco_capuano@aol.com >
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
* chore(teleop): print calibration path saved (#1286 )
* chore(dependencies): add gamepad support with pygame and hidapi (#1287 )
* Robot integration tutorial (#1285 )
* fix(docs): update send_feedback docstrings
* Add sim tutorial, fix lekiwi motor config, add notebook links (#1275 )
Co-authored-by: AdilZouitine <adilzouitinegm@gmail.com >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Michel Aractingi <michel.aractingi@gmail.com >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
* Fixes on robot integration tutorial (#1290 )
* Add keyboard teleop device to control the end effector robot (#1289 )
* Improve type hints (#1293 )
* fix(record): no teleop arg in reset environment (#1294 )
* `learner.py` import so101_leader instead of so100 (#1295 )
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
* Fixing `PI0` Policy (#1297 )
* `gym_manipulator.py` Remove None value action_intervention of BaseLeaderTeleoperator (#1299 )
* (chore): incorrect resume parameter in recording documentation (#1301 )
* Update lekiwi.mdx (#1229 )
* bump `pi0` and `hil` transformers version (#1298 )
* docs: fix imitation learning robots docs command (#1308 )
* fix(benchmarks): remove .numpy() from frame in benchmark script (#1354 )
* add smolvla to the supported policies to run tests (:
* add: chunk-level access for the policy
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* add: smolvla in availables
* remove: smolvla from library supported policies
* fix: change env for training, xarm is broken as of now
* add: predict_action_chunk to all supported policies
* fix: add robot type constants
* add: predict action chunk in base policy class
* restore original Makefile
* fix: minor
* fix: dict keys come from lerobot/constants
* fix: improve act encapsulation, properly supporting temporal ensembling
* fix: smolvla action chunking
* fix: very minor, but very annoying
* fix: minor
* fix minor naming
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Signed-off-by: Francesco Capuano <74058581+fracapuano@users.noreply.github.com >
* fix: refactoring inference for single actions and chunks into different components
* fix: minor
* fix: temporal ensembling
* fix: moving populate queues out of modular component for batch preparation
* fix: minor for CI
* fix: smovla debug
* fix: reward classifier, maybe the last policy lacking?
---------
Signed-off-by: Francesco Capuano <74058581+fracapuano@users.noreply.github.com >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: Caroline Pascal <caroline8.pascal@gmail.com >
Co-authored-by: Mishig Davaadorj <dmishig@gmail.com >
Co-authored-by: omahs <73983677+omahs@users.noreply.github.com >
Co-authored-by: CharlesCNorton <135471798+CharlesCNorton@users.noreply.github.com >
Co-authored-by: masato-ka <jp6uzv@gmail.com >
Co-authored-by: Ragnar <rodiondenmark@gmail.com >
Co-authored-by: mshukor <mustafa.shukor97@gmail.com >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: Dana Aubakirova <118912928+danaaubakirova@users.noreply.github.com >
Co-authored-by: Remi <remi.cadene@huggingface.co >
Co-authored-by: Ben Zhang <5977478+ben-z@users.noreply.github.com >
Co-authored-by: Pepijn <pepijn@huggingface.co >
Co-authored-by: Dhruva <51377003+utterwqlnut@users.noreply.github.com >
Co-authored-by: Daisuke Sato <tiryoh@gmail.com >
Co-authored-by: Sarunas Kalade <sarunas.kalade@amd.com >
Co-authored-by: koenvanwijk <koenvanwijk@users.noreply.github.com >
Co-authored-by: Yushun Xiang <73413365+YushunXiang@users.noreply.github.com >
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com >
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Ke Wang <superwk1017@gmail.com >
Co-authored-by: Yoel Chornton <yoel.chornton@gmail.com >
Co-authored-by: Michel Aractingi <michel.aractingi@gmail.com >
Co-authored-by: tidely <43219534+tidely@users.noreply.github.com >
Co-authored-by: David <17435126+DavidLMS@users.noreply.github.com >
2025-06-27 10:19:19 +02:00
0b2285d1ec
Feat: Improve hub integration ( #1382 )
...
* feat(policies): Initial setup to push policies to hub with tags and model card
* feat: add dataset that is used to train
* Add model template summary
* fix: Update link model_card template
* fix: remove print
* fix: change import name
* fix: add model summary in template
* fix: minor text
* fix: comments Lucain
* fix: feedback steven
* fix: restructure push to hub
* fix: remove unneeded changes
* fix: import
* fix: import 2
* Add MANIFEST.in
* fix: feedback pr
* Fix tests
* tests: Add smolvla end-to-end test
* Fix: smolvla test
* fix test name
* fix policy tests
* Add push to hub false policy tests
* Do push to hub cleaner
* fix(ci): add push_to_hub false in tests
---------
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
2025-06-26 14:36:16 +02:00
a989c79558
docs: Fix the SO-100 documentation, the motors configuration step should be before the assembly instructions ( #1315 )
...
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
2025-06-26 13:31:32 +02:00
06450c6777
update assembly instructions to match outputs from setup motors 'python -m lerobot.setup_motors' script ( #1384 )
2025-06-26 12:15:35 +02:00
fe88c5942c
There can be only one!! ( #1343 )
...
pkg-config appears twice in the package list.
Co-authored-by: Caroline Pascal <caroline8.pascal@gmail.com >
2025-06-25 14:43:14 +02:00
a5727e37b4
Fix teleop disconnect during eval ( #1364 )
2025-06-23 16:49:14 +02:00
dc67b2ff3f
Store policy config alongside PEFT checkpoint
...
Before this change the PEFT-wrapped policy did not save the policy's config
alongside the adapter config / weights which prevented us from changing the
policy config. Now the policy config is saved both in full training and PEFT
training.
This change makes loading the PEFT policy adapter much easier as well.
2025-06-22 19:54:10 +02:00
7fd8b4c773
Implement loading of PEFT adapters
...
Loading a PEFT adapter is currently done by initializing a policy with default config
and then applying the adapter on the resulting model. This has the obvious drawback
that any configurations done during training are not applied in the adapted model.
Currently the `use_peft` attribute of `PreTrainedConfig` is only set during loading
to signal the following code that it has to deal with a PEFT adapter. However
we could imagine a scenario where this is already set at training time and stored
alongside the adapter.
2025-06-22 19:10:10 +02:00
98856662c1
Add basic support for PEFT adapter methods
...
This changes adds support for training policies with much less parameters
by applying adapter methods such as LoRA on specific parts of the policies
and therefore possibly higher learning rates / batch sizes.
To make this as accessible as possible I thought it useful to provide
defaults for `target_modules` and `modules_to_save`. Currently only SmolVLA
has such defaults but when we agree that this change is useful I will set
out to generate more such defaults. While the user can override these
settings, they are expected to only change the peft_method, rank and init_type
parameters.
2025-06-22 13:45:07 +02:00
c940676bdd
fix(benchmarks): remove .numpy() from frame in benchmark script ( #1354 )
2025-06-19 17:07:13 +02:00
2b71789e15
docs: fix imitation learning robots docs command ( #1308 )
2025-06-15 11:47:48 +02:00
7c8be7fb9b
bump pi0 and hil transformers version ( #1298 )
2025-06-15 08:57:08 +02:00
b8637c09ec
Update lekiwi.mdx ( #1229 )
2025-06-14 23:41:45 +02:00
1688fa3a88
(chore): incorrect resume parameter in recording documentation ( #1301 )
2025-06-14 23:38:10 +02:00
b852d15774
gym_manipulator.py Remove None value action_intervention of BaseLeaderTeleoperator (#1299 )
2025-06-14 20:53:40 +02:00
ce6a26deeb
Fixing PI0 Policy ( #1297 )
2025-06-14 19:25:50 +02:00
697c76f75e
learner.py import so101_leader instead of so100 (#1295 )
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
2025-06-14 15:30:19 +02:00
8d7969e7cb
fix(record): no teleop arg in reset environment ( #1294 )
2025-06-14 14:23:07 +02:00
dcc0c234dd
Improve type hints ( #1293 )
2025-06-14 14:06:22 +02:00
6007a221f0
Add keyboard teleop device to control the end effector robot ( #1289 )
2025-06-14 09:10:09 +02:00
35e67585bf
Fixes on robot integration tutorial ( #1290 )
2025-06-14 01:47:22 +02:00
438334d58e
Add sim tutorial, fix lekiwi motor config, add notebook links ( #1275 )
...
Co-authored-by: AdilZouitine <adilzouitinegm@gmail.com >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Michel Aractingi <michel.aractingi@gmail.com >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
2025-06-13 18:48:39 +02:00
69e8946480
fix(docs): update send_feedback docstrings
2025-06-13 18:29:19 +02:00
96fa48b5ec
Robot integration tutorial ( #1285 )
2025-06-13 18:23:07 +02:00
8fc18be065
chore(dependencies): add gamepad support with pygame and hidapi ( #1287 )
2025-06-13 17:07:11 +02:00
5350a02dc1
chore(teleop): print calibration path saved ( #1286 )
2025-06-13 15:29:10 +02:00
58afa2fbb0
fix(docs): SmolVLA fine-tuning getting started ( #1201 )
...
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Francesco Capuano <francesco_capuano@aol.com >
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
2025-06-13 14:17:59 +02:00
d8079587a2
Port HIL SERL ( #644 )
...
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Ke Wang <superwk1017@gmail.com >
Co-authored-by: Yoel Chornton <yoel.chornton@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
2025-06-13 13:15:47 +02:00
f976935ba1
fix(record): no teleop needed when running with policy ( #1284 )
2025-06-13 12:41:30 +02:00
5c87365cc1
Skip normalization parameters in load_smolvla ( #1274 )
2025-06-13 11:06:45 +02:00
edfebd522c
Use HF Papers ( #1120 )
2025-06-12 09:58:59 +02:00
2de93a8000
fix(docs): update realsense documentation ( #1268 )
2025-06-11 23:16:37 +02:00
d0521189b1
fix issues: checkpoints keys mismatch and 'task' tokenisation in smolvla ( #1256 )
...
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
2025-06-11 16:56:55 +02:00
10b7b35325
Match motor names with ids lekiwi ( #1261 )
2025-06-11 14:21:30 +02:00
459c95197b
fix: update pi0 dependency version constraint ( #1247 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-06-10 18:46:41 +02:00
37748c83ca
Proposal for fix for enter_pressed on Windows ( #1230 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
2025-06-10 18:36:02 +02:00
3fb04efec1
[pre-commit.ci] pre-commit autoupdate ( #1185 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
2025-06-10 18:04:09 +02:00
2889f3a06a
update KochFollower.get_observation() so it returns same observation structure as SO101 ( #1248 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-06-10 12:42:54 +02:00
f5335fe696
Update tutorial link ( #1250 )
2025-06-10 11:05:08 +02:00
f0a903c98f
Fix unable to set camera width/height to non-default ( #1225 )
2025-06-10 10:23:33 +02:00
0e7caae714
Update SmolVLA README.md ( #1228 )
2025-06-08 23:15:26 +02:00
1ee2ca5c26
fix(pyserial): adding pyserial dependency to global ones ( #1219 )
2025-06-06 14:38:33 +02:00
4e4eec92dc
Fix smolVLA dependencies ( #1218 )
2025-06-06 11:28:47 +02:00
95df341b4f
Fix LeKiwi example ( #1217 )
2025-06-06 10:08:03 +02:00
9e6f49f507
Fix test_teleoperate ( #1216 )
2025-06-06 09:38:37 +02:00
a28f02ecb3
replaced OBS_ROBOT with OBS_STATE constant ( #1211 )
2025-06-06 09:25:51 +02:00
09343acce7
fix(smolvla): update record.py, fix populate_queues and remove unused dependencies ( #1208 )
2025-06-06 09:17:02 +02:00
e23b41e79a
Hardware API redesign ( #777 )
...
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
Co-authored-by: Pepijn <pepijn@huggingface.co >
2025-06-05 17:48:43 +02:00
nemo