lerobot

mirror of https://github.com/huggingface/lerobot.git synced 2026-07-12 12:32:02 +00:00

Files

T

CarolinePascal 7bee7fb9e3 feat(datasets): distribute stats recomputation across SLURM workers

Expose the shardable unit of work behind recompute_stats: compute_dataset_episode_stats
computes per-episode stats for an episode subset, and aggregate_episode_stats merges the
concatenated shards (count-weighted) and writes stats.json. recompute_stats now composes
these, so single-process behavior is unchanged.

Add examples/dataset/slurm_recompute_stats.py, a datatrove compute/aggregate driver that
shards episodes across workers and is read-only safe (reference-copies the source when
--new-root is given). Most useful for the expensive image/video stats path.

2026-07-10 14:47:18 +02:00

create_progress_videos.py

refactor: support custom progress parquet overlays (#3640 )

2026-05-21 14:32:10 +02:00

load_lerobot_dataset.py

feat(dependencies): minimal default tag install (#3362 )

2026-04-12 20:03:04 +02:00

slurm_compute_rabc.py

Reward models refactor (#3142 )

2026-04-28 17:56:24 +02:00

slurm_recompute_stats.py

feat(datasets): distribute stats recomputation across SLURM workers