Switching embodiment on a packaged checkpoint keeps the previous
embodiment's camera_sizes entries because dict config values merge
instead of replacing, which tripped the exact-key validation. Rebuild
sizes from the named profile whenever the keys diverge from
camera_order, and add regression tests for both fixes.
Co-authored-by: Cursor <cursoragent@cursor.com>
LeRobot dataset stats include a scalar per-feature sample count that is
not a per-dimension statistic, so projecting it to the embodiment index
map raised a width-mismatch ValueError for every real dataset.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add dataloader_multiprocessing_context, default to spawn
Make the DataLoader multiprocessing start method configurable on
TrainPipelineConfig and default it to 'spawn'.
The previous default (fork on Linux) is unsafe with libraries that hold
non-fork-safe state in the parent process — common ones in this codebase
are PyAV, torchcodec, and the ffmpeg shared libs they wrap. Symptoms
reported in #2488, #2209, and observed locally include:
- multiprocessing.context.AuthenticationError: digest received was wrong
- RuntimeError: Pin memory thread exited unexpectedly
- RuntimeError: DataLoader worker exited unexpectedly
- Random SIGSEGV inside worker processes during video decode
Switching to spawn re-imports modules cleanly in each worker and
eliminates these failure modes. Added the setting as a config field
rather than hard-coding so users on platforms where fork is preferred
can opt back in via --dataloader-multiprocessing-context=fork.
* Address review: shorten config comment, note spawn startup tradeoff
Per @jashshah999, mention that spawn workers re-import modules and so
add some startup time vs fork. Also trim the failure-mode dump from
the inline comment — the linked issue covers the symptoms in detail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(scripts): add multiprocessing_context safeguards
* chore(config): add libs note
---------
Co-authored-by: 0o8o0-blip <0o8o0-blip@users.noreply.github.com>
Register the `lekiwi` robot module in `lerobot_replay.py` so episodes can be
replayed on a LeKiwi via `--robot.type=lekiwi_client`. The module is already
registered in `lerobot_calibrate.py` and `lerobot_setup_motors.py`; this fills
the gap so the replay CLI recognizes the same robot.
Replayed actions are loaded from the dataset as torch tensors, which
`json.dumps` cannot serialize when `LeKiwiClient.send_action` ships them over
ZMQ. Coerce each action value to a plain float before sending. This is scoped
to the LeKiwi network client and does not affect any other robot.
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
* fix(record): connect teleoperator before robot to avoid watchdog jump
lerobot-record connected the robot before the teleoperator. A robot's
connect()/reset() can leave it holding a default pose under a firmware
watchdog (e.g. Unitree G1); if teleop.connect() (model loading, IK init,
network setup) then takes longer than that watchdog, the joints drop to
damping and the first send_action() makes the robot jump.
Swap the order so the teleoperator connects first, matching the ordering
already used in lerobot_teleoperate.py. Pure ordering fix, no API change.
Fixes#3684
* fix(record): trim comment and add connect-order regression test
Address review feedback on #3684:
- Trim the verbose ordering comment down to two lines.
- Add test_record_connects_teleop_before_robot to tests/test_control_robot.py,
asserting teleop.connect() runs before robot.connect() in record().
* chore(test): remove test
---------
Co-authored-by: Jaimin Patel <jpatel@tuvalabs.com>
Co-authored-by: Martino Russi <77496684+nepyope@users.noreply.github.com>
Pepijn