mirror of
https://github.com/huggingface/lerobot.git
synced 2026-07-25 10:46:01 +00:00
fix: nanoVLM processing
This commit is contained in:
@@ -52,9 +52,14 @@ lerobot-train \
|
||||
--dataset.repo_id=<dataset_repo_id> \
|
||||
--output_dir=outputs/vf_nanovlm_probe \
|
||||
--steps=5000 \
|
||||
--batch_size=16
|
||||
--batch_size=1
|
||||
```
|
||||
|
||||
The released checkpoint's native preprocessing resizes the long image side to
|
||||
2048 and creates 512px global/split views. A 480x640 camera therefore produces
|
||||
13 vision inputs and roughly 832 image placeholders; use batch size 1 initially
|
||||
for a three-camera setup.
|
||||
|
||||
Then load the probe checkpoint and selectively fine-tune the projector/decoder
|
||||
at a lower learning rate.
|
||||
|
||||
@@ -107,5 +112,8 @@ uv run python scripts/overfit_vf_variant.py \
|
||||
--steps=500
|
||||
```
|
||||
|
||||
For `nanovlm_value_function`, start with `--num_samples=2` because all overfit
|
||||
samples are held in one batch and native image tiling is memory intensive.
|
||||
|
||||
Compare runs using held-out episode NLL/MAE, per-episode return rank correlation,
|
||||
terminal success/failure separation, and the matched-versus-shuffled image loss gap.
|
||||
|
||||
Reference in New Issue
Block a user