A robot_type of so100/so101 is treated as wrong when the joint dim isn't a
multiple of 6, or (when names are present) the first 6 joints don't match the
canonical SO set. Such datasets are migrated structurally to v3.0 with joints
left untouched and robot_type relabeled 'unknown', instead of being skipped or
degrees-converted on a false assumption.
Only migrate datasets usable right away as a clean 6-DOF joint stack. SO datasets
whose action/observation.state dim isn't a multiple of 6 (extra bbox/EE columns
appended) are now skipped as out-of-scope, matching the end-effector skip, rather
than being relabeled '_nonstandard' and migrated structurally.
Preserve the original robot_type lineage (e.g. so100 -> so100_nonstandard) for
arms whose joint dim isn't a multiple of 6, rather than erasing it to 'unknown'.
Idempotent: won't re-append the suffix.
Some datasets store task-space end-effector pose (names like ee_x/ee_roll or
x/y/z) instead of joint angles; the degrees mapping is meaningless there. Detect
via feature names and skip them entirely (no conversion, no upload) rather than
migrating a mislabeled arm.
download_subfolder gains a repo= arg so it can source from any monorepo; the new
extract_dataset.py scoped-downloads one sub-dataset and re-uploads it at the root
of a new standalone dataset repo.
The manifest robot_type distribution has 'bi_so100_follower', which the trailing
underscore in 'bi_so_' missed. Drop it to 'bi_so' to catch all bimanual SO
variants (bi_so_follower, bi_so100_follower, ...) with no false positives.
Broaden SO_PREFIXES to (so100, so101, so_, bi_so_) so bimanual two-arm datasets
(robot_type 'bi_so_follower', 12-dim) are recognized as SO and get the degrees
conversion, instead of being skipped as non_so. so_ boundary avoids matching
stray names like 'sofa'.
Split classify() into reusable pieces: is_so_robot_type() for the robot_type
name test, and encoding_from_bounds() as the single source of truth for the
degrees_old/degrees_new/normalized/radians decision from per-joint min/max
(layout-agnostic, so v2.1 episodes_stats and v3.0 stats.json both feed it).
is_so is decided purely from the robot_type string, so datasets labeled
so100/so101 whose joint dim isn't a multiple of 6 carry a provably wrong label.
Rewrite meta/info.json robot_type to 'unknown' in that case so the v3.0 output
isn't misidentified as an SO arm.
Datasets whose action/state dim is not a multiple of 6 (e.g. 7-dim with an
appended EE pose, or 10-dim) are not a plain stack of SO arms, so the degrees
mapping doesn't apply. Migrate them structurally instead of aborting the whole
dataset with a ValueError.
- download_subfolder: fetch only the target sub-dataset subtree instead of
enumerating the whole community_dataset_v3 monorepo tree (fixes apparent hang)
- normalized SO gripper (RANGE_0_100) left in native 0..100 frame, matching
degrees_new datasets, instead of remapping to +/-45deg
- uncalibrated normalized datasets: skip identity value rewrite, keep normalized
units and flag them APPROXIMATE on the dataset card
- remove --allow-uncalibrated flag and its CANON_IS_CALIBRATED side effect
CarolinePascal