Commit Graph

4 Commits

Author SHA1 Message Date
CarolinePascal a369f1d1ee migration: make per-rank resume prefetch resilient to Hub timeouts
Retry the destination list_repo_files with backoff (and fall back to an
empty resume set) so a single transient huggingface.co read timeout no
longer fails an entire SLURM rank and skips its dataset slice.
2026-07-22 18:33:54 +02:00
CarolinePascal a62c5495ff migration: add MolmoAct standalone-dataset migration path
Add migrate_molmoact.py to migrate the SO-100/101 datasets listed by
allenai/MolmoAct2-SO100_101-Dataset (repo ids derived from language_annotations
folder names), skipping any already in lerobot/community_dataset_v3. Generalize
run_migration.migrate_one/_write_dataset_card with a backward-compatible
standalone mode (download a whole standalone repo, provenance -> source repo)
and extend slurm_migrate.py with --source molmoact / --reference-repo.
2026-07-22 16:32:19 +02:00
CarolinePascal f82713cdb2 migration: scope subfolder download, keep normalized joints as-is
- download_subfolder: fetch only the target sub-dataset subtree instead of
  enumerating the whole community_dataset_v3 monorepo tree (fixes apparent hang)
- normalized SO gripper (RANGE_0_100) left in native 0..100 frame, matching
  degrees_new datasets, instead of remapping to +/-45deg
- uncalibrated normalized datasets: skip identity value rewrite, keep normalized
  units and flag them APPROXIMATE on the dataset card
- remove --allow-uncalibrated flag and its CANON_IS_CALIBRATED side effect
2026-07-16 17:30:10 +02:00
CarolinePascal f9dd1cf25f feat(slurm): adding support for slurm computing 2026-07-16 15:33:41 +02:00