Comparing 93e29b0cfc..5753f8c18b - lerobot - Gitea: Git with a cup of tea

admin/lerobot

mirror of https://github.com/huggingface/lerobot.git synced 2026-06-16 15:57:03 +00:00

Author	SHA1	Message	Date
Steven Palma	5753f8c18b	fix(groot): GPU/tensor N1.7 image preprocessing + resize to trained resolution GR00T training was dataloader-bound (0->100->0 GPU-utilization sawtooth). GrootN17VLMEncodeStep ran the Qwen3-VL image processor per frame on PIL images on the single CPU main-loop thread, and that cost is timed inside dataloading_s (preprocessor(batch) runs in the main process, not the dataloader workers), so adding workers cannot hide it. - Feed the torchvision-backed Qwen3-VL processor (C,H,W) uint8 tensors instead of a per-frame Image.fromarray PIL roundtrip, and run resize/normalize/patchify on config.device (GPU) when available. Bit-identical on CPU when no resize is configured; with a resize only the PIL->torchvision bicubic backend differs (<2/255 per pixel). The use_albumentations path stays PIL/cv2; reload on a box without the saved device falls back to CPU. - Default image_target_size/crop to the N1.7 backbone's training geometry (256x256 / 230x230) when a checkpoint ships no image sizing (checkpoint_assets is None, e.g. finetuning nvidia/GR00T-N1.7-3B via repo-id with a new embodiment). Previously image_target_size=None disabled the resize, so full-resolution frames were patchified into ~4.7x more vision tokens than the model was trained on -- inflating dataloading_s (patchify) and update_s (VLM sequence) and skewing the input distribution. Checkpoints that pin their own sizing are honored; the default constants are shared with GR00T_N1_7_DEFAULTS. Net: preprocessing leaves the CPU critical path and the VLM sees the resolution it was trained on -- faster training/inference and a correct train/serve distribution. Affects inference too (shared preprocessor); existing checkpoints still load (backward compatible) but must be retrained to gain the benefits.	2026-06-15 18:20:49 +02:00
Kartik	97bd373d15	Merge pull request #15 from huggingface/fix/groot_n17_core fix(groot): N1.7 config defaults, N1.5 rejection, and processor/model runtime fixes	2026-06-13 23:05:51 +02:00
Kartik	10a73e3c95	Merge pull request #14 from huggingface/fix/groot_n17_backbone fix(groot): N1.7 backbone loading and DiT parameter-count logging	2026-06-13 21:47:35 +02:00
Kartik	27c9288b24	Merge pull request #13 from huggingface/fix/groot_n17_docs docs(groot): document the N1.5 removal and the N1.7 parity test	2026-06-13 21:47:05 +02:00

Diff Content Not Available