fix(peft): allow fresh LoRA fine-tuning from a base-model checkpoint

This commit is contained in:
Maxime Ellerbach
2026-07-24 13:37:03 +00:00
parent 051b13573e
commit a3755f07c5
5 changed files with 211 additions and 6 deletions
+7
View File
@@ -62,3 +62,10 @@ to the `--peft.full_training_modules` parameter:
The learning rate and the scheduled target learning rate can usually be scaled by a factor of 10 compared to the
learning rate used for full fine-tuning (e.g., 1e-4 normal, so 1e-3 using LoRA).
## Other policies
The same `--peft.*` flags work for any pre-trained policy. Some policies (SmolVLA, π₀, π₀.₅) ship
built-in default `target_modules`, so `--peft.method_type=LORA` is enough. Others do not, and will
ask you to pass `--peft.target_modules` explicitly — for example LingBot-VA, whose recommended
targets are documented in its [dedicated guide](./lingbot_va#training--fine-tuning).