pepijn223 98ee5cdc22 feat(lingbot_va): implement training / fine-tuning (flow-matching loss) ...

- Implement LingBotVAPolicy.forward(): dual-stream flow-matching training loss
  (latent + action, timestep-weighted, action-masked) ported from upstream train.py;
  VAE-encodes camera clips, UMT5-encodes the task, noises both streams, runs the
  block-causal flex-attention training pass (forward_train).
- training_loss_from_streams() core + _build_training_streams() data prep (action
  scatter into the 30-d space, multi-frame VAE encode incl. robotwin_tshape).
- get_optim_params returns only trainable transformer params (LoRA/PEFT friendly);
  VAE/UMT5 stay frozen. Training needs attn_mode='flex'.
- Add a tiny-config single-training-step test (forward->loss->backward->AdamW) and a
  Training/fine-tuning section in the docs.

Co-authored-by: Cursor <cursoragent@cursor.com>

2026-06-23 17:31:28 +00:00

..

_toctree.yml

feat(policies): add LingBot-VA autoregressive video-action world model

2026-06-23 17:30:31 +00:00

act.mdx

fix examples (#3623)

2026-05-21 22:14:07 +02:00

action_representations.mdx

feat(policies): add relative action support for pi0, pi0.5, and pi0_fast (#2970)

2026-04-01 12:59:12 +02:00

adding_benchmarks.mdx

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

annotation_pipeline.mdx

feat: language annotation pipeline (#3471)

2026-06-12 15:12:33 +02:00

async.mdx

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

backwardcomp.mdx

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

bring_your_own_policies.mdx

docs: add policy & compute guide (#3534)

2026-05-11 15:19:12 +02:00

cameras.mdx

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

cheat-sheet.mdx

docs(cheat sheet): create cheat sheet (#3602)

2026-05-14 15:11:35 +02:00

contributing.md

Hardware API redesign (#777)

2025-06-05 17:48:43 +02:00

damiao.mdx

feat(motors): add damiao motors & can bus (#2788)

2026-01-26 17:53:25 +01:00

debug_processor_pipeline.mdx

feat(processors): use pipelines across the codebase (#1452)

2025-09-18 15:25:26 +02:00

earthrover_mini_plus.mdx

feat(encoding parameters): adding support for user provided video encoding parameters (#3455)

2026-05-14 23:46:42 +02:00

env_processor.mdx

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

envhub_isaaclab_arena.mdx

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

envhub_leisaac.mdx

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

envhub.mdx

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

eo1.mdx

feat(policies): add EO-1 model (#3403)

2026-05-06 18:01:16 +02:00

feetech.mdx

Add feetech firmware update docs (#1793)

2025-08-28 11:18:54 +02:00

groot.mdx

fix examples (#3623)

2026-05-21 22:14:07 +02:00

hardware_guide.mdx

docs: add policy & compute guide (#3534)

2026-05-11 15:19:12 +02:00

hil_data_collection.mdx

refactor(robots): homogenize bi-manual setups implementations (#3772)

2026-06-15 16:28:54 +02:00

hilserl_sim.mdx

chore(rl): move rl related code to its directory at top level (#2002)

2025-09-23 16:32:34 +02:00

hilserl.mdx

RL stack refactoring (#3075)

2026-05-12 15:49:54 +02:00

hope_jr.mdx

feat(encoding parameters): adding support for user provided video encoding parameters (#3455)

2026-05-14 23:46:42 +02:00

il_robots.mdx

feat(rollout): adding episodic strategy (#3717)

2026-06-06 00:32:38 +02:00

implement_your_own_processor.mdx

feat(processors): use pipelines across the codebase (#1452)

2025-09-18 15:25:26 +02:00

index.mdx

Update pre-commit-config.yaml + pyproject.toml + ceil rerun & transformer dependencies version (#1520)

2025-07-17 14:30:20 +02:00

inference.mdx

refactor(robots): homogenize bi-manual setups implementations (#3772)

2026-06-15 16:28:54 +02:00

installation.mdx

chore(deps): cap torch ceiling at <2.12, pin Linux wheels to cu128 (#3570)

2026-05-11 19:47:55 +02:00

integrate_hardware.mdx

feat(robots): consolidate SO arms implementation (#2763)

2026-01-08 13:04:30 +01:00

introduction_processors.mdx

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

koch.mdx

fix(docs): update outdated links (#2026)

2025-09-24 16:17:39 +02:00

language_and_recipes.mdx

Add extensive language support (#3467)

2026-05-19 14:46:11 +02:00

lekiwi.mdx

chore(docs): updating deprecated huggingface-cli to hf (#3071)

2026-03-04 15:08:49 +01:00

lelab.mdx

Docs/add lelab (#3707)

2026-06-03 14:22:05 +02:00

lerobot-dataset-v3.mdx

docs: fix broken dataset script paths (datasets/v30 -> scripts) (#3695)

2026-06-03 14:48:19 +02:00

libero_plus.mdx

feat(envs): add LIBERO-plus robustness benchmark (#3313)

2026-04-20 21:07:21 +02:00

libero.mdx

docs(benchmarks): add benchmark integration guide and standardize benchmark docs (#3270)

2026-04-03 14:44:53 +02:00

lingbot_va.mdx

feat(lingbot_va): implement training / fine-tuning (flow-matching loss)

2026-06-23 17:31:28 +00:00

metaworld.mdx

feat(envs): lazy env init + AsyncVectorEnv as default for n_envs > 1 (#3274)

2026-04-09 10:29:20 +02:00

molmoact2.mdx

docs: fix broken dataset script paths (datasets/v30 -> scripts) (#3695)

2026-06-03 14:48:19 +02:00

multi_gpu_training.mdx

feat(train): FSDP checkpoint saving (#3810)

2026-06-22 13:51:21 +02:00

multi_task_dit.mdx

chore(policy): multi dit docs (#3285)

2026-04-05 21:23:13 +02:00

notebooks.mdx

Update pre-commit-config.yaml + pyproject.toml + ceil rerun & transformer dependencies version (#1520)

2025-07-17 14:30:20 +02:00

omx.mdx

fix(robots): update gripper configuration and calibration settings for OMX (#2815)

2026-01-25 22:29:37 +01:00

openarm.mdx

feat(robots): add bi manual openarm follower and leader (#2835)

2026-01-28 17:25:57 +01:00

peft_training.mdx

fix(config): add lora_alpha to PeftConfig (#3573)

2026-05-13 11:09:19 +02:00

phone_teleop.mdx

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

pi0.mdx

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

pi0fast.mdx

chore(docs): remove pi installation note (#3095)

2026-03-06 15:52:54 +01:00

pi05.mdx

docs: fix broken dataset script paths (datasets/v30 -> scripts) (#3695)

2026-06-03 14:48:19 +02:00

policy_act_README.md

Update readme (#1570)

2025-08-01 17:39:39 +02:00

policy_diffusion_README.md

Update readme (#1570)

2025-08-01 17:39:39 +02:00

policy_groot_README.md

feat(policies): add Nvidia Gr00t N1.5 model (#2292)

2025-10-23 13:50:30 +02:00

policy_molmoact2_README.md

Add MolmoAct2 policy (#3604)

2026-05-27 18:58:37 +02:00

policy_multi_task_dit_README.md

Feature/add multitask diffusion transformer policy implementation (#2545)

2026-03-28 00:41:26 +01:00

policy_pi0_README.md

feat(dependencies): minimal default tag install (#3362)

2026-04-12 20:03:04 +02:00

policy_pi05_README.md

chore(docs): no policy readme in src code (#3286)

2026-04-05 19:25:38 +02:00

policy_rtc_README.md

chore(docs): no policy readme in src code (#3286)

2026-04-05 19:25:38 +02:00

policy_sarm_README.md

chore(docs): no policy readme in src code (#3286)

2026-04-05 19:25:38 +02:00

policy_smolvla_README.md

Update readme (#1570)

2025-08-01 17:39:39 +02:00

policy_tdmpc_README.md

Update readme (#1570)

2025-08-01 17:39:39 +02:00

policy_vla_jepa_README.md

feat(policies): add VLA-JEPA (#3568)

2026-06-04 19:22:51 +02:00

policy_vqbet_README.md

Update readme (#1570)

2025-08-01 17:39:39 +02:00

policy_walloss_README.md

fix a bug for kwargs in wallx (#2714)

2026-01-06 15:13:35 +01:00

porting_datasets_v3.mdx

docs: fix broken dataset script paths (datasets/v30 -> scripts) (#3695)

2026-06-03 14:48:19 +02:00

processors_robots_teleop.mdx

chore(docs): update code block syntax to specify python for clarity (#2770)

2026-01-08 14:45:07 +01:00

reachy2.mdx

feat(encoding parameters): adding support for user provided video encoding parameters (#3455)

2026-05-14 23:46:42 +02:00

rebot_b601.mdx

feat(robots): natively integrate Seeed Studio reBot B601-DM arm (#3624)

2026-05-18 19:49:21 +02:00

rename_map.mdx

feat(rollout): decouple policy deployment from data recording with new lerobot-rollout CLI (#3413)

2026-04-28 00:57:35 +02:00

robocasa.mdx

feat(envs): add RoboCasa365 benchmark integration (#3375)

2026-04-20 17:10:53 +02:00

robocerebra.mdx

feat(envs): add RoboCerebra long-horizon manipulation benchmark (#3314)

2026-04-20 19:12:15 +02:00

robometer.mdx

feat(rewards): add ROBOMETER reward model (#3627)

2026-05-29 21:45:39 +02:00

robomme.mdx

feat(envs): add RoboMME benchmark (#3311)

2026-04-20 20:21:27 +02:00

robotwin.mdx

feat(envs): add RoboTwin 2.0 benchmark (#3315)

2026-04-20 17:46:39 +02:00

rtc.mdx

feat(rollout): decouple policy deployment from data recording with new lerobot-rollout CLI (#3413)

2026-04-28 00:57:35 +02:00

sarm.mdx

Reward models refactor (#3142)

2026-04-28 17:56:24 +02:00

smolvla.mdx

fix examples (#3623)

2026-05-21 22:14:07 +02:00

so100.mdx

feat(robots): consolidate SO arms implementation (#2763)

2026-01-08 13:04:30 +01:00

so101.mdx

Fix SO-101 assembly instruction order to match videos (#3242)

2026-03-31 12:16:34 +02:00

streaming_video_encoding.mdx

feat(encoding parameters): adding support for user provided video encoding parameters (#3455)

2026-05-14 23:46:42 +02:00

tools.mdx

Add extensive language support (#3467)

2026-05-19 14:46:11 +02:00

topreward.mdx

feat(rewards): add TOPReward reward model (#3629)

2026-05-27 14:24:31 +02:00

torch_accelerators.mdx

Add a documentation page with a brief intro to hw backends (#2385)

2025-12-05 13:32:58 +01:00

unitree_g1.mdx

feat(rollout): decouple policy deployment from data recording with new lerobot-rollout CLI (#3413)

2026-04-28 00:57:35 +02:00

using_dataset_tools.mdx

feat(encoding parameters): adding support for user provided video encoding parameters (#3455)

2026-05-14 23:46:42 +02:00

video_encoding_parameters.mdx

feat(encoding parameters): adding support for user provided video encoding parameters (#3455)

2026-05-14 23:46:42 +02:00

vla_jepa.mdx

feat(policies): add VLA-JEPA (#3568)

2026-06-04 19:22:51 +02:00

vlabench.mdx

feat(sim): VLABench benchmark integration (#3396)

2026-04-21 17:54:11 +02:00

walloss.mdx

chore: remove usernames + use entrypoints in docs, comments & sample commands (#2988)

2026-02-18 22:46:12 +01:00

xvla.mdx

fix xvla docs (#3291)

2026-04-23 14:50:32 +02:00