If the data files and any camera's video files (or two cameras) don't have the
same number of episodes, skip the dataset up front instead of letting the
converter raise 'All cams dont have same number of episodes' mid-run.
When the data files and video files agree on episode count N but the metadata
lists a different count, rewrite meta/episodes.jsonl, meta/episodes_stats.jsonl
and info.json to N before the v2.1->v3.0 converter runs (which otherwise raises
'Number of episodes is not the same'). Only the safe direction is handled:
trimming metadata that lists MORE episodes than exist. Non-contiguous data,
data/video disagreement, or metadata missing episodes are left untouched.
When the leading joint names match the canonical SO set, treat the dataset as a
genuine SO arm and convert those joints to degrees even if extra columns follow
(bbox, etc.), passing the trailing non-joint columns through untouched. Encoding
detection now looks only at the SO slice so appended columns can't skew it. Only
when no leading 6-DOF SO block can be substantiated is the dataset relabeled
'unknown' and migrated structurally.
A robot_type of so100/so101 is treated as wrong when the joint dim isn't a
multiple of 6, or (when names are present) the first 6 joints don't match the
canonical SO set. Such datasets are migrated structurally to v3.0 with joints
left untouched and robot_type relabeled 'unknown', instead of being skipped or
degrees-converted on a false assumption.
Only migrate datasets usable right away as a clean 6-DOF joint stack. SO datasets
whose action/observation.state dim isn't a multiple of 6 (extra bbox/EE columns
appended) are now skipped as out-of-scope, matching the end-effector skip, rather
than being relabeled '_nonstandard' and migrated structurally.
Preserve the original robot_type lineage (e.g. so100 -> so100_nonstandard) for
arms whose joint dim isn't a multiple of 6, rather than erasing it to 'unknown'.
Idempotent: won't re-append the suffix.
is_so is decided purely from the robot_type string, so datasets labeled
so100/so101 whose joint dim isn't a multiple of 6 carry a provably wrong label.
Rewrite meta/info.json robot_type to 'unknown' in that case so the v3.0 output
isn't misidentified as an SO arm.
Datasets whose action/state dim is not a multiple of 6 (e.g. 7-dim with an
appended EE pose, or 10-dim) are not a plain stack of SO arms, so the degrees
mapping doesn't apply. Migrate them structurally instead of aborting the whole
dataset with a ValueError.
- download_subfolder: fetch only the target sub-dataset subtree instead of
enumerating the whole community_dataset_v3 monorepo tree (fixes apparent hang)
- normalized SO gripper (RANGE_0_100) left in native 0..100 frame, matching
degrees_new datasets, instead of remapping to +/-45deg
- uncalibrated normalized datasets: skip identity value rewrite, keep normalized
units and flag them APPROXIMATE on the dataset card
- remove --allow-uncalibrated flag and its CANON_IS_CALIBRATED side effect
CarolinePascal