Commit Graph

1204 Commits

Author SHA1 Message Date
Bryson Jones 632c778a2b Merge branch 'main' into feature/add-multitask-dit 2025-12-18 15:01:40 -08:00
Steven Palma 0bd1969d0a feat(docs): modernize readme (#2660) 2025-12-18 19:45:13 +01:00
Bryson Jones 77dbc951cd Merge branch 'main' into feature/add-multitask-dit 2025-12-18 08:36:25 -08:00
Pepijn f04958527e Add sarm (#2639)
* add initial modeling

* make rewind pretrained policy

* add annotation

* small fix

* add sarm

* subtasks

* fix spawn

* fix rewind discrepancies

* Add script to generate embedding for dataset (#2138)

* Add generate and validate script

* fix precommit

* Improve generate embeddings function by using dataset tools (#2206)

---------

Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>

* cleanup

* change order train log

* print batch size

* update sarm processor

* add reward output

* change expected features

* add image validation

* change validation

* get state input from dataset stats

* raise if no state key is found

* pass stats

* cleanup and refactor

* add episode inddex to complementary data

* add subtask init and detection

* revert lerobot_train changes

* pass dataset metadata to policy

* change loadig subtasks

* add small logging

* fix progress conversion and adding initial frame

* use large offset for initial frame (ugly)

* Remove rewind, use clip tokenizer

* add tests, implement formula 1,2 correctly and cleanup

* use task from dataset, cleanup visualizer

* simplify

* simplify and cleanup code and move compute_temporal_proportions to utils

* fix normalization in visualization

* Fix visualization and change prompt

* fix formatting

* add visualize subtask annotations

* use qwen thinking

* try different prompt

* format

* update prompt

* higher temp, long output

* different settings

* use instruct

* show full resp

* split message

* Temp: increase tolerance dataset

* Fix RA-BC (#2572)

* Add next observation loading for RA-BC progress deltas

* Compute weights based on temporal progress deltas instead of static rewards

* Add hard-masking for negative progress deltas in weight computation

* Feat/add dual head (#2582)

* Add dual dense sparse head and annotation

* Add docs

* add dual to procesor

* cleanup

* change sampling in visualize and cleanup

* remove validation

* remove compile

* Feat/test uniform (#2587)

* test uniform

* add different string for misaligned

* Fix rewind and add tests

* uncomment text implementation

* run precommit

* Add head mode for ra-bc

* fix visalization of single task

* add

* return per sample loss

* Fix RA_BC (#2602)

* update rabc implementation

* compute rabc beforehand

* fix import

* add only progress calulation

* use precomputed progress

* multi gpu processing

* import

* fix dataset meta data extraction

* add logging

* logging

* log

* progress per episode

* split differently

* move clip to gpu

* pre decode frames for an episode

* fix cuda initalization

* fix import

* multi processing

* rename

* fix import

* fix

* fix rabc

* use last known progress if oob

* use last known progress if oob

* add misalignment loss with random embeddings

* discard previous changes

* add selection of models to docs for ra_bc

* add transformers dep

* extend tolerance

* initial commit with new codebase

* add tests

* fix

* remove temporal sampler

* drop last frame for sampler

* use original ref

* some fixes

* fix visualization

* remove smoothing and fix order subtasks

* add stride rabc computation

* add push to hub

* add explanation

* add kappa expllaination

* better rabc logging

* feedback pr

* remove dataset tolerance

* revert dataset tool

* revert dataset changes

* add credit

* run precommit

* change path for generate ra_bc

* fix type

* include sarm in all in pyproject

* fix precommit

* lazy import matplotlib

* lazy import qwen

* remove rich console

* skip if transformers is not installed?

* run only when we have faker

* place transformer lazy loading

* Dont test if low transformer version

* fix

* increase transformer

* increase as 4.57.0 is yanked

* remove pi from all

* go back

---------

Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com>
2025-12-18 12:50:32 +01:00
Bryson Jones 5b9f98169c Merge branch 'main' into feature/add-multitask-dit 2025-12-17 20:22:11 -08:00
Steven Palma 4a151a9682 chore(ci): minor improvement bug-report template & pr auto label (#2676)
* chore(ci): minor improvement bug-report template

* chore(ci): change triggers for PR auto label
2025-12-18 00:23:23 +01:00
Steven Palma 8667b9ef08 chore(ci): minor improvements auto labeling (#2675) 2025-12-17 22:54:47 +01:00
Steven Palma 86eee5c1e2 fix(ci): close bracket pattern (#2674) 2025-12-17 22:40:33 +01:00
Steven Palma 469b855e42 fix(ci): better heuristic + issue type template fix (#2672)
* fix(ci): better heuristic + issue type template fix

* chore(ci): remove keywords in performance tag
2025-12-17 22:31:22 +01:00
Bryson Jones 2128dece82 Merge branch 'main' into feature/add-multitask-dit 2025-12-17 09:25:57 -08:00
Steven Palma 292333cafc chore(ci): update issue template (#2666) 2025-12-17 18:02:20 +01:00
Steven Palma f0c98e23f1 feat(ci): simple automatic labelling (#2667)
* ci: add pr labeler

* ci: add issue labeler

* ci: minor fixes for labelers

* fix(ci): add explicit path for pr labeler
2025-12-17 17:52:45 +01:00
Steven Palma 7621af5acd chore(ci): update PR template (#2665)
* chore: update code of conduct to transformers one

* chore: update PR template
2025-12-17 17:10:04 +01:00
Bryson Jones b575632f4f Merge branch 'main' into feature/add-multitask-dit 2025-12-17 07:37:09 -08:00
Steven Palma 92fdbe9bbf docs(dataset): add visualization section (#2664) 2025-12-17 14:14:31 +01:00
Steven Palma b303d1ab38 feat(scripts): add more info to lerobot-info (#2663) 2025-12-17 14:14:23 +01:00
Steven Palma b1d162f333 fix(policies): add device back to smolvlm expert (#2662) 2025-12-17 12:12:03 +01:00
Bryson Jones 8a2f5aa6cb remove cropping of images smaller than the crop size 2025-12-15 22:20:20 -08:00
Bryson Jones 25ecd16b67 skip tests without transformers 2025-12-15 21:52:42 -08:00
Bryson Jones 1e049fbef7 add test handling for multitask dit when transformers isnt available 2025-12-15 21:21:35 -08:00
Bryson Jones afe2c4d3aa Merge branch 'main' into feature/add-multitask-dit 2025-12-15 18:08:21 -08:00
Steven Palma 2b304eeb84 feat(dataset): expose tolerance_s argument to training config (#2653) 2025-12-16 00:53:19 +01:00
Bryson Jones 534e143b0c add conditional transformers import to match all other policies that use transformers lib 2025-12-15 10:08:47 -08:00
Bryson Jones 23382c0ac5 add multitask dit to toc for docs 2025-12-15 10:02:00 -08:00
Bryson Jones 4eda54c7fe Merge branch 'main' into feature/add-multitask-dit 2025-12-15 09:57:27 -08:00
Sota Nakamura 4e6048a221 finalize the dataset after recording (#2496)
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
2025-12-15 17:57:04 +01:00
./c² 81ebcac8d7 docs: update IL robots API example and add OpenCV workaround (#2648)
* docs: update IL robots API example and add OpenCV workaround

- Fix import path from lerobot.record to lerobot.scripts.lerobot_record
- Add required processor parameters to record_loop calls
- Document fourcc="MJPG" workaround for OpenCV async errors
- Improve code formatting in robot configuration examples

Fixes compatibility issues for users following the tutorial on embedded systems
and ensures API examples match current codebase requirements.

* Update il_robots.mdx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: ./c² <cagataycali@icloud.com>

---------

Signed-off-by: ./c² <cagataycali@icloud.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
2025-12-15 17:56:33 +01:00
Bryson Jones a632dd3af4 Merge branch 'main' into feature/add-multitask-dit 2025-12-15 08:22:05 -08:00
Martino Russi a6c3a0fa09 Feat/add mj env (#2613)
* add sim support

* close fix threading issues
2025-12-15 16:22:27 +01:00
Woojin Wie c2fb644613 feat(robot): Add support for OMX robot (#2614)
* upload

* feat(omx): simplify motor initialization and remove default calibration files

* feat(omx): read motor positions without normalization for improved accuracy

* update calibration method for return factory value

Signed-off-by: Junha Cha <ckwnsgk1@gachon.ac.kr>

* change the drive mode

* refactor: clean up code by removing unnecessary blank lines in omx_follower and omx_leader modules

* feat(omx): update calibration method to set drive modes for motors

* feat(pyproject): add 'ROBOTIS' to extend-ignore-identifiers-re list

* feat(omx): enhance calibration method to write default drive modes to motors

* Update src/lerobot/robots/omx_follower/__init__.py

Add informations about the robot

Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
Signed-off-by: Woojin Wie <dnldnwls1123@gmail.com>

---------

Signed-off-by: Junha Cha <ckwnsgk1@gachon.ac.kr>
Signed-off-by: Woojin Wie <dnldnwls1123@gmail.com>
Co-authored-by: Junha02 <chajunha2023@naver.com>
Co-authored-by: Junha Cha <ckwnsgk1@gachon.ac.kr>
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
2025-12-15 15:50:29 +01:00
Bryson Jones e4a1b27fd3 Merge branch 'main' into feature/add-multitask-dit 2025-12-14 09:10:48 -08:00
Jade Choghari 1d07a4aefd add auto in docs (#2645)
Signed-off-by: Jade Choghari <chogharijade@gmail.com>
2025-12-13 17:11:19 +01:00
Bryson Jones 71f359ca6e refactor code to perform task tokenization in the processor instead of in the modeling code for multitask dit 2025-12-11 12:09:54 -08:00
Bryson Jones 51dfee43f4 rename config param for multiple vision encoders 2025-12-11 09:58:57 -08:00
Bryson Jones 1f74982469 note origins of each training objective 2025-12-11 09:55:41 -08:00
Bryson Jones 8e3a1e8945 add more descriptions and depth to multitask dit tutorial 2025-12-11 09:45:53 -08:00
Bryson Jones 43c335d0d7 reformat and clean up tutorial for multitask dit policy 2025-12-11 09:33:30 -08:00
Bryson Jones dd4ef1383f fix nit formatting in generate actions fcn 2025-12-11 09:21:26 -08:00
Bryson Jones f3823e8bcd remove the base classes since we don't need to be able to extend 2025-12-11 09:20:25 -08:00
Bryson Jones c398a146b3 use constants for indexing into batches and remove env state references 2025-12-11 09:13:38 -08:00
Bryson Jones 9b47c5fac9 move policy to top of file 2025-12-11 09:04:38 -08:00
Bryson Jones 56dbeed89f add processor tests to multitask dit tests 2025-12-11 09:04:23 -08:00
Bryson Jones 67b1a9eeb1 update typo in test instruction comment 2025-12-11 08:28:35 -08:00
Bryson Jones 86e0ee787d remove environment state conditioning 2025-12-11 08:26:21 -08:00
Bryson Jones ba968e84f1 fix bugs when testing on hardware 2025-12-10 16:26:57 -08:00
Bryson Jones d49d3390f6 Merge branch 'main' into feature/add-multitask-dit 2025-12-10 14:44:04 -08:00
Bryson Jones f1ac454800 add tutorial to training with multi_task_dit 2025-12-10 14:43:27 -08:00
Bryson Jones 10cfc17705 remove redundant asserts 2025-12-10 14:43:27 -08:00
Bryson Jones 5524a0d7a7 split up select action return statement 2025-12-10 14:43:27 -08:00
Bryson Jones 3a16a002f8 add torch.no_grad decorators 2025-12-10 14:43:27 -08:00