lerobot

mirror of https://github.com/huggingface/lerobot.git synced 2026-06-10 05:09:48 +00:00

Files

T

Pepijn 68fa5d80b0 feat(streaming): multinode example, dataloading benchmark, distributed smoke test

- examples/scaling/train_streaming_multinode.py: Accelerate-based distributed/
  resumable streaming training (no DistributedSampler; rank/world_size auto-resolved),
  checkpoints the dataset stream state, and supports a --dummy pure-dataloading path
  with throughput logging. SLURM launcher in slurm/train_streaming_robocasa.sh.
- benchmarks/streaming/benchmark_streaming.py: dummy-consumer dataloading benchmark
  (single / sarm frame modes) emitting frames/s/node, p50/p95/p99 sample latency,
  first-batch latency, and VideoDecoderCache reuse stats as JSON + CSV. SLURM launcher
  + README documenting the source/node/mode matrix and manual bucket prewarming.
- VideoDecoderCache: add hit/miss/eviction counters and a stats() method so the
  benchmark can surface decoder thrash (no new cache, no eviction-policy change).
- tests/datasets/test_streaming_distributed.py: accelerate-launch smoke test asserting
  per-rank disjointness; skips (does not false-pass) when <2 processes spawn.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-09 13:48:23 +02:00

backward_compatibility

feat(dependencies): minimal default tag install (#3362 )

2026-04-12 20:03:04 +02:00

dataset

refactor: support custom progress parquet overlays (#3640 )

2026-05-21 14:32:10 +02:00

lekiwi

feat(rollout): decouple policy deployment from data recording with new lerobot-rollout CLI (#3413 )

2026-04-28 00:57:35 +02:00

notebooks

docs(quickstart): adding rollout (#3598 )