dc67b2ff3f
Store policy config alongside PEFT checkpoint
...
Before this change the PEFT-wrapped policy did not save the policy's config
alongside the adapter config / weights which prevented us from changing the
policy config. Now the policy config is saved both in full training and PEFT
training.
This change makes loading the PEFT policy adapter much easier as well.
2025-06-22 19:54:10 +02:00
7fd8b4c773
Implement loading of PEFT adapters
...
Loading a PEFT adapter is currently done by initializing a policy with default config
and then applying the adapter on the resulting model. This has the obvious drawback
that any configurations done during training are not applied in the adapted model.
Currently the `use_peft` attribute of `PreTrainedConfig` is only set during loading
to signal the following code that it has to deal with a PEFT adapter. However
we could imagine a scenario where this is already set at training time and stored
alongside the adapter.
2025-06-22 19:10:10 +02:00
98856662c1
Add basic support for PEFT adapter methods
...
This changes adds support for training policies with much less parameters
by applying adapter methods such as LoRA on specific parts of the policies
and therefore possibly higher learning rates / batch sizes.
To make this as accessible as possible I thought it useful to provide
defaults for `target_modules` and `modules_to_save`. Currently only SmolVLA
has such defaults but when we agree that this change is useful I will set
out to generate more such defaults. While the user can override these
settings, they are expected to only change the peft_method, rank and init_type
parameters.
2025-06-22 13:45:07 +02:00
c940676bdd
fix(benchmarks): remove .numpy() from frame in benchmark script ( #1354 )
2025-06-19 17:07:13 +02:00
2b71789e15
docs: fix imitation learning robots docs command ( #1308 )
2025-06-15 11:47:48 +02:00
7c8be7fb9b
bump pi0 and hil transformers version ( #1298 )
2025-06-15 08:57:08 +02:00
b8637c09ec
Update lekiwi.mdx ( #1229 )
2025-06-14 23:41:45 +02:00
1688fa3a88
(chore): incorrect resume parameter in recording documentation ( #1301 )
2025-06-14 23:38:10 +02:00
b852d15774
gym_manipulator.py Remove None value action_intervention of BaseLeaderTeleoperator (#1299 )
2025-06-14 20:53:40 +02:00
ce6a26deeb
Fixing PI0 Policy ( #1297 )
2025-06-14 19:25:50 +02:00
697c76f75e
learner.py import so101_leader instead of so100 (#1295 )
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
2025-06-14 15:30:19 +02:00
8d7969e7cb
fix(record): no teleop arg in reset environment ( #1294 )
2025-06-14 14:23:07 +02:00
dcc0c234dd
Improve type hints ( #1293 )
2025-06-14 14:06:22 +02:00
6007a221f0
Add keyboard teleop device to control the end effector robot ( #1289 )
2025-06-14 09:10:09 +02:00
35e67585bf
Fixes on robot integration tutorial ( #1290 )
2025-06-14 01:47:22 +02:00
438334d58e
Add sim tutorial, fix lekiwi motor config, add notebook links ( #1275 )
...
Co-authored-by: AdilZouitine <adilzouitinegm@gmail.com >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Michel Aractingi <michel.aractingi@gmail.com >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
2025-06-13 18:48:39 +02:00
69e8946480
fix(docs): update send_feedback docstrings
2025-06-13 18:29:19 +02:00
96fa48b5ec
Robot integration tutorial ( #1285 )
2025-06-13 18:23:07 +02:00
8fc18be065
chore(dependencies): add gamepad support with pygame and hidapi ( #1287 )
2025-06-13 17:07:11 +02:00
5350a02dc1
chore(teleop): print calibration path saved ( #1286 )
2025-06-13 15:29:10 +02:00
58afa2fbb0
fix(docs): SmolVLA fine-tuning getting started ( #1201 )
...
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Francesco Capuano <francesco_capuano@aol.com >
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
2025-06-13 14:17:59 +02:00
d8079587a2
Port HIL SERL ( #644 )
...
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Ke Wang <superwk1017@gmail.com >
Co-authored-by: Yoel Chornton <yoel.chornton@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
2025-06-13 13:15:47 +02:00
f976935ba1
fix(record): no teleop needed when running with policy ( #1284 )
2025-06-13 12:41:30 +02:00
5c87365cc1
Skip normalization parameters in load_smolvla ( #1274 )
2025-06-13 11:06:45 +02:00
edfebd522c
Use HF Papers ( #1120 )
2025-06-12 09:58:59 +02:00
2de93a8000
fix(docs): update realsense documentation ( #1268 )
2025-06-11 23:16:37 +02:00
d0521189b1
fix issues: checkpoints keys mismatch and 'task' tokenisation in smolvla ( #1256 )
...
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
2025-06-11 16:56:55 +02:00
10b7b35325
Match motor names with ids lekiwi ( #1261 )
2025-06-11 14:21:30 +02:00
459c95197b
fix: update pi0 dependency version constraint ( #1247 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-06-10 18:46:41 +02:00
37748c83ca
Proposal for fix for enter_pressed on Windows ( #1230 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
2025-06-10 18:36:02 +02:00
3fb04efec1
[pre-commit.ci] pre-commit autoupdate ( #1185 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
2025-06-10 18:04:09 +02:00
2889f3a06a
update KochFollower.get_observation() so it returns same observation structure as SO101 ( #1248 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-06-10 12:42:54 +02:00
f5335fe696
Update tutorial link ( #1250 )
2025-06-10 11:05:08 +02:00
f0a903c98f
Fix unable to set camera width/height to non-default ( #1225 )
2025-06-10 10:23:33 +02:00
0e7caae714
Update SmolVLA README.md ( #1228 )
2025-06-08 23:15:26 +02:00
1ee2ca5c26
fix(pyserial): adding pyserial dependency to global ones ( #1219 )
2025-06-06 14:38:33 +02:00
4e4eec92dc
Fix smolVLA dependencies ( #1218 )
2025-06-06 11:28:47 +02:00
95df341b4f
Fix LeKiwi example ( #1217 )
2025-06-06 10:08:03 +02:00
9e6f49f507
Fix test_teleoperate ( #1216 )
2025-06-06 09:38:37 +02:00
a28f02ecb3
replaced OBS_ROBOT with OBS_STATE constant ( #1211 )
2025-06-06 09:25:51 +02:00
09343acce7
fix(smolvla): update record.py, fix populate_queues and remove unused dependencies ( #1208 )
2025-06-06 09:17:02 +02:00
e23b41e79a
Hardware API redesign ( #777 )
...
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
Co-authored-by: Pepijn <pepijn@huggingface.co >
2025-06-05 17:48:43 +02:00
b536f47e3f
Fix SmolVLA loss not sent to wandb ( #1198 )
2025-06-05 11:13:03 +02:00
bfd26eef5a
Add SmolVLA ( #1175 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: fracapuano <francesco.capuano@huggingface.co >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: Dana Aubakirova <118912928+danaaubakirova@users.noreply.github.com >
Co-authored-by: Remi <remi.cadene@huggingface.co >
2025-06-03 17:11:50 +02:00
1537d0ab90
[pre-commit.ci] pre-commit autoupdate ( #1048 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
2025-06-02 19:30:39 +02:00
2be7f3a3ff
(hotfix): nightly CI by clipping pymunk version below 7.0.0 ( #1182 )
2025-06-02 13:18:02 +02:00
0cf864870c
[Fix] Unpin torch beyond 2.6.0 & torchcodec beyond 0.2.1 ( #1127 )
2025-05-28 16:54:20 +02:00
1786916a16
Update README.md ( #1163 )
2025-05-27 11:50:43 +02:00
0507ad4f68
Update README.md ( #1160 )
2025-05-27 11:45:07 +02:00
bed90e3a41
fix: typos and grammar ( #1148 )
2025-05-25 17:20:45 +02:00
nemo