feat(policiy): fine tune vision encoder refactor completness

This commit is contained in:
Steven Palma
2026-07-29 18:35:24 +02:00
parent 9e30807eeb
commit 0371e99117
5 changed files with 66 additions and 19 deletions
+13
View File
@@ -70,6 +70,19 @@ cd lerobot && lerobot-train \
GPU allows it, as long as loading times remain short.
</Tip>
For tasks that require adapting visual features, such as distinguishing new colors or shapes, selectively
fine-tune the vision encoder and its connector:
```bash
lerobot-train ... \
--policy.path=lerobot/smolvla_base \
--policy.fine_tune_vision_encoder=true
```
This keeps the language model frozen with the default `train_expert_only=true` setting and trains the vision
path at `0.1` times the main learning rate by default. Fine-tuning the vision encoder increases memory use and
can reduce the model's general visual knowledge, so enable it only when the frozen encoder is insufficient.
Fine-tuning is an art. For a complete overview of the options for finetuning, run
```bash