fix(edit-dataset): align root with LeRobotDataset v2 semantics

.github/workflows/nightly.yml

@@ -188,7 +188,7 @@ jobs:
       - name: Verify GPU availability
         run: |
           nvidia-smi
-          python -c "import torch; print(f'PyTorch version: {torch.__version__}'); print(f'PyTorch CUDA available: {torch.cuda.is_available()}'); print(f'Number of GPUs: {torch.cuda.device_count()}')"
+          python -c "import torch; print(f'PyTorch CUDA available: {torch.cuda.is_available()}'); print(f'Number of GPUs: {torch.cuda.device_count()}')"
 
       - name: Run multi-GPU training tests
       # TODO(Steven): Investigate why motors tests are failing in multi-GPU setup

docs/source/_toctree.yml

@@ -19,8 +19,6 @@
     title: Multi GPU training
   - local: peft_training
     title: Training with PEFT (e.g., LoRA)
-  - local: rename_map
-    title: Using Rename Map and Empty Cameras
   title: "Tutorials"
 - sections:
   - local: lerobot-dataset-v3

docs/source/rename_map.mdx

@@ -1,145 +0,0 @@
-# Understanding the Rename Map and Empty Cameras
-
-When you train or evaluate a robot policy, your **dataset** or **environment** hands you observations under one set of keys (e.g. `observation.images.front`, `observation.images.eagle`), while your **policy** was built to expect another (e.g. `observation.images.image`, `observation.images.image2`). The rename map is how you bridge that gap without changing the policy or the data source.
-
-This guide explains why it exists, how to use it in training and evaluation, and when to use **empty cameras** so you can fine-tune multi-camera policies on datasets that have fewer views.
-
----
-
-## Why observation keys don’t always match
-
-Policies have a fixed set of **input feature names** (often coming from a pretrained config). For example:
-
-- **XVLA-base** expects three image keys: `observation.images.image`, `observation.images.image2`, `observation.images.image3`.
-- **pi0-fast-libero** might expect `observation.images.base_0_rgb` and `observation.images.left_wrist_0_rgb`.
-
-Your dataset or sim might use completely different names: `observation.images.front`, `observation.images.eagle`, `observation.images.glove` (e.g. [svla_so100_sorting](https://huggingface.co/datasets/lerobot/svla_so100_sorting)). Or your eval env (e.g. LIBERO) might return `observation.images.image` and `observation.images.image2`.
-
-Rather than renaming columns in the dataset or editing the policy code, LeRobot lets you pass a **rename map**: a dictionary that says “when you see this key in the data, treat it as this key for the policy.” Renaming is applied in the preprocessing pipeline so the policy always receives the keys it expects.
-
----
-
-## How the rename map works
-
-The rename map is a dictionary:
-
-- **Keys** = observation keys as produced by your **dataset** (training) or **environment** (evaluation).
-- **Values** = the observation keys your **policy** expects.
-
-Only keys listed in the map are renamed; everything else is left as-is. Under the hood, the [RenameObservationsProcessorStep](https://github.com/huggingface/lerobot/blob/main/src/lerobot/processor/rename_processor.py) runs in the preprocessor and rewrites observation keys (and keeps normalization stats aligned) so the batch matches the policy’s `input_features`.
-
-You can use the same idea for **training** (dataset → policy) and **evaluation** (env → policy).
-
-<p align="center">
-  <img
-    src="https://huggingface.co/datasets/jadechoghari/images/resolve/main/rename-map.png"
-    alt="Rename map: mapping dataset or environment observation keys to policy input keys"
-    style="max-width: 100%; height: auto;"
-  />
-</p>
-
----
-
-## Option 1: Use a rename map (recommended)
-
-You pass the mapping on the command line so dataset/env keys are renamed to what the policy expects. No need to change the policy repo or the data.
-
-### Training example: XVLA on a dataset with different camera names
-
-Suppose you fine-tune [lerobot/xvla-base](https://huggingface.co/lerobot/xvla-base) on a dataset whose images are stored under `observation.images.front`, `observation.images.eagle`, and `observation.images.glove`. XVLA expects `observation.images.image`, `observation.images.image2`, and `observation.images.image3`. Map the dataset keys to the policy keys:
-
-```bash
-lerobot-train \
-  --dataset.repo_id=YOUR_DATASET \
-  --output_dir=./outputs/xvla_training \
-  --job_name=xvla_training \
-  --policy.path="lerobot/xvla-base" \
-  --policy.repo_id="HF_USER/xvla-your-robot" \
-  --policy.dtype=bfloat16 \
-  --policy.action_mode=auto \
-  --steps=20000 \
-  --policy.device=cuda \
-  --policy.freeze_vision_encoder=false \
-  --policy.freeze_language_encoder=false \
-  --policy.train_policy_transformer=true \
-  --policy.train_soft_prompts=true \
-  --rename_map='{"observation.images.front": "observation.images.image", "observation.images.eagle": "observation.images.image2", "observation.images.glove": "observation.images.image3"}'
-```
-
-Order of entries in the map doesn’t matter; each dataset key is renamed to the corresponding policy key.
-
-### Evaluation example: Policy trained on different camera names than the env
-
-You trained (or downloaded) a policy that expects `observation.images.base_0_rgb` and `observation.images.left_wrist_0_rgb` (e.g. [pi0fast-libero](https://huggingface.co/lerobot/pi0fast-libero)), but your evaluation environment (e.g. LIBERO) returns `observation.images.image` and `observation.images.image2`. Tell the eval script how to rename env keys to policy keys:
-
-```bash
-lerobot-eval \
-  --policy.path=lerobot/pi0fast-libero \
-  --env.type=libero \
-  ... \
-  --rename_map='{"observation.images.image": "observation.images.base_0_rgb", "observation.images.image2": "observation.images.left_wrist_0_rgb"}'
-```
-
-So: **key = what the env gives, value = what the policy expects.** Same convention as in training.
-
----
-
-## Option 2: Change the policy config (no rename map)
-
-If you prefer not to pass a rename map every time, you can **edit the policy’s `config.json`** so that its expected observation keys match your dataset or environment. For example, change the policy’s visual input keys to `observation.images.front`, `observation.images.eagle`, `observation.images.glove` to match your dataset, or to `observation.images.image` / `observation.images.image2` to match LIBERO.
-
-- **Training:** If the dataset’s camera keys match the (modified) policy config, you don’t need a rename map.
-- **Evaluation:** If the env’s keys match the (modified) policy config, you don’t need a rename map for eval either.
-
-The tradeoff: you’re changing the policy repo or your local checkpoint. That’s fine if you’re only ever using that one dataset or env; a rename map keeps the same policy usable across multiple data sources without touching the config.
-
----
-
-## When you have fewer cameras than the policy expects: empty cameras
-
-Some policies (e.g. XVLA) are built for a fixed number of image inputs (e.g. three). Your dataset might only have **two** cameras. You still want to fine-tune without changing the model architecture.
-
-LeRobot supports this with **empty cameras**: the config declares extra “slots” that the policy expects, but the dataset (or env) does not provide. Those slots are filled with placeholder keys and typically zero or masked inputs so the policy can run with fewer real views.
-
-<p align="center">
-  <img
-    src="https://huggingface.co/datasets/jadechoghari/images/resolve/main/empty_cam.png"
-    alt="Empty cameras: using placeholder slots when the dataset has fewer views than the policy expects"
-    style="max-width: 100%; height: auto;"
-  />
-</p>
-
-- In the policy config (e.g. [xvla-base config.json](https://huggingface.co/lerobot/xvla-base/blob/main/config.json)), `empty_cameras` is the number of these extra slots (default `0`).
-- For each slot, the config adds an observation key of the form:
-  `observation.images.empty_camera_0`, `observation.images.empty_camera_1`, …
-
-Example: XVLA-base has three visual inputs and `empty_cameras=0`. Your dataset has only two images. Set **`empty_cameras=1`**. Then:
-
-1. The config gains a third visual key: `observation.images.empty_camera_0`.
-2. You still use the rename map (or matching config keys) for the two real cameras.
-3. The third view is treated as “empty” (no corresponding dataset key); the policy ignores or masks it as needed.
-
-So you fine-tune on two observations only, and the third visual input is effectively unused. You do **not** need to add a fake third image to your dataset.
-
----
-
-## Where the rename map is used in the codebase
-
-- **Training** ([`lerobot_train.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/scripts/lerobot_train.py)): `rename_map` is passed into `make_policy(..., rename_map=cfg.rename_map)` and into the preprocessor as `rename_observations_processor: {"rename_map": cfg.rename_map}`. Batches from the dataset are renamed before being fed to the policy.
-- **Evaluation** ([`lerobot_eval.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/scripts/lerobot_eval.py)): Same idea—`rename_map` is passed to `make_policy` and to the preprocessor so env observations are renamed before the policy sees them.
-- **Processor** ([`rename_processor.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/processor/rename_processor.py)): `RenameObservationsProcessorStep` does the actual key renaming and updates feature metadata so normalization stats stay consistent with the renamed keys.
-
-If you see a feature mismatch error (“Missing features” / “Extra features”), the error message suggests using `--rename_map` with a mapping from your data’s keys to the policy’s expected keys.
-
----
-
-## Quick reference
-
-| Goal                                  | What to do                                                                                                 |
-| ------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
-| Dataset keys ≠ policy keys (training) | `--rename_map='{"dataset_key": "policy_key", ...}'`                                                        |
-| Env keys ≠ policy keys (eval)         | `--rename_map='{"env_key": "policy_key", ...}'`                                                            |
-| Fewer cameras than policy expects     | Set `empty_cameras` in the policy config (e.g. `1` when you have 2 real cameras and the policy expects 3). |
-| Avoid passing a rename map            | Edit the policy’s `config.json` so its observation keys match your dataset or env.                         |
-
-The rename map keeps your pipeline flexible: one policy, many data sources, no code changes—just a small dictionary on the command line or in your config.

pyproject.toml

@@ -76,7 +76,7 @@ dependencies = [
     "pyserial>=3.5,<4.0",
     "wandb>=0.24.0,<0.25.0",
 
-    "torch==2.10.0",
+    "torch>=2.2.1,<2.11.0", # TODO: Bump dependency
     "torchcodec>=0.2.1,<0.11.0; sys_platform != 'win32' and (sys_platform != 'linux' or (platform_machine != 'aarch64' and platform_machine != 'arm64' and platform_machine != 'armv7l')) and (sys_platform != 'darwin' or platform_machine != 'x86_64')", # TODO: Bump dependency
     "torchvision>=0.21.0,<0.26.0", # TODO: Bump dependency
 

src/lerobot/scripts/lerobot_edit_dataset.py

@@ -19,7 +19,9 @@ Edit LeRobot datasets using various transformation tools.
 
 This script allows you to delete episodes, split datasets, merge datasets,
 remove features, modify tasks, and convert image datasets to video format.
-When new_repo_id is specified, creates a new dataset.
+
+Path semantics (v2): --root and --new_root are exact dataset folders containing
+meta/, data/, videos/. When omitted, defaults to $HF_LEROBOT_HOME/{repo_id}.
 
 Usage Examples:
 
@@ -29,10 +31,18 @@ Delete episodes 0, 2, and 5 from a dataset:
         --operation.type delete_episodes \
         --operation.episode_indices "[0, 2, 5]"
 
-Delete episodes and save to a new dataset:
+Delete episodes from a local dataset at a specific path:
+    lerobot-edit-dataset \
+        --repo_id lerobot/pusht \
+        --root /data/lerobot/pusht \
+        --operation.type delete_episodes \
+        --operation.episode_indices "[0, 2, 5]"
+
+Delete episodes and save to a new dataset at a specific path:
     lerobot-edit-dataset \
         --repo_id lerobot/pusht \
         --new_repo_id lerobot/pusht_filtered \
+        --new_root /data/lerobot/pusht_filtered \
         --operation.type delete_episodes \
         --operation.episode_indices "[0, 2, 5]"
 
@@ -42,6 +52,13 @@ Split dataset by fractions:
         --operation.type split \
         --operation.splits '{"train": 0.8, "val": 0.2}'
 
+Split dataset by fractions with custom output base directory:
+    lerobot-edit-dataset \
+        --repo_id lerobot/pusht \
+        --new_root /data/splits \
+        --operation.type split \
+        --operation.splits '{"train": 0.8, "val": 0.2}'
+
 Split dataset by episode indices:
     lerobot-edit-dataset \
         --repo_id lerobot/pusht \
@@ -54,12 +71,27 @@ Split into more than two splits:
         --operation.type split \
         --operation.splits '{"train": 0.6, "val": 0.2, "test": 0.2}'
 
-Merge multiple datasets:
+Merge multiple datasets (from Hub or default cache):
     lerobot-edit-dataset \
         --repo_id lerobot/pusht_merged \
         --operation.type merge \
         --operation.repo_ids "['lerobot/pusht_train', 'lerobot/pusht_val']"
 
+Merge local datasets with explicit paths:
+    lerobot-edit-dataset \
+        --repo_id lerobot/pusht_merged \
+        --operation.type merge \
+        --operation.repo_ids "['lerobot/pusht_train', 'lerobot/pusht_val']" \
+        --operation.roots "['/data/pusht_train', '/data/pusht_val']"
+
+Merge local datasets with custom output path:
+    lerobot-edit-dataset \
+        --repo_id lerobot/pusht_merged \
+        --new_root /output/merged \
+        --operation.type merge \
+        --operation.repo_ids "['lerobot/pusht_train', 'lerobot/pusht_val']" \
+        --operation.roots "['/data/pusht_train', '/data/pusht_val']"
+
 Remove camera feature:
     lerobot-edit-dataset \
         --repo_id lerobot/pusht \
@@ -88,8 +120,8 @@ Modify tasks - set default task with overrides for specific episodes (WARNING: m
 Convert image dataset to video format and save locally:
     lerobot-edit-dataset \
         --repo_id lerobot/pusht_image \
-        --operation.type convert_image_to_video \
-        --operation.output_dir /path/to/output/pusht_video
+        --new_root /path/to/output/pusht_video \
+        --operation.type convert_image_to_video
 
 Convert image dataset to video format and save with new repo_id:
     lerobot-edit-dataset \
@@ -167,6 +199,8 @@ class SplitConfig(OperationConfig):
 @dataclass
 class MergeConfig(OperationConfig):
     repo_ids: list[str] | None = None
+    # Per-dataset exact folder paths. Must match repo_ids length when provided.
+    roots: list[str] | None = None
 
 
 @OperationConfig.register_subclass("remove_feature")
@@ -208,26 +242,35 @@ class InfoConfig(OperationConfig):
 class EditDatasetConfig:
     repo_id: str
     operation: OperationConfig
+    # Exact dataset folder (containing meta/, data/, videos/). If None, defaults to HF_LEROBOT_HOME/{repo_id}.
     root: str | None = None
     new_repo_id: str | None = None
+    # Exact output dataset folder. If None, derived from new_repo_id or in-place.
+    new_root: str | None = None
     push_to_hub: bool = False
 
 
-def get_output_path(repo_id: str, new_repo_id: str | None, root: Path | None) -> tuple[str, Path]:
-    if new_repo_id:
+def get_output_path(
+    input_root: Path,
+    new_repo_id: str | None,
+    new_root: Path | None,
+    default_repo_id: str,
+) -> tuple[str, Path]:
+    if new_root:
+        output_repo_id = new_repo_id or default_repo_id
+        output_dir = new_root
+    elif new_repo_id:
         output_repo_id = new_repo_id
-        output_dir = root / new_repo_id if root else HF_LEROBOT_HOME / new_repo_id
+        output_dir = HF_LEROBOT_HOME / new_repo_id
     else:
-        output_repo_id = repo_id
-        dataset_path = root / repo_id if root else HF_LEROBOT_HOME / repo_id
-        old_path = Path(str(dataset_path) + "_old")
+        output_repo_id = default_repo_id
+        output_dir = input_root
+        backup_path = input_root.with_name(input_root.name + "_old")
 
-        if dataset_path.exists():
-            if old_path.exists():
-                shutil.rmtree(old_path)
-            shutil.move(str(dataset_path), str(old_path))
-
-        output_dir = dataset_path
+        if input_root.exists():
+            if backup_path.exists():
+                shutil.rmtree(backup_path)
+            shutil.move(str(input_root), str(backup_path))
 
     return output_repo_id, output_dir
 
@@ -241,11 +284,14 @@ def handle_delete_episodes(cfg: EditDatasetConfig) -> None:
 
     dataset = LeRobotDataset(cfg.repo_id, root=cfg.root)
     output_repo_id, output_dir = get_output_path(
-        cfg.repo_id, cfg.new_repo_id, Path(cfg.root) if cfg.root else None
+        input_root=dataset.root,
+        new_repo_id=cfg.new_repo_id,
+        new_root=Path(cfg.new_root) if cfg.new_root else None,
+        default_repo_id=cfg.repo_id,
     )
 
-    if cfg.new_repo_id is None:
-        dataset.root = Path(str(dataset.root) + "_old")
+    if output_dir == dataset.root:
+        dataset.root = dataset.root.with_name(dataset.root.name + "_old")
 
     logging.info(f"Deleting episodes {cfg.operation.episode_indices} from {cfg.repo_id}")
     new_dataset = delete_episodes(
@@ -275,16 +321,20 @@ def handle_split(cfg: EditDatasetConfig) -> None:
     dataset = LeRobotDataset(cfg.repo_id, root=cfg.root)
 
     logging.info(f"Splitting dataset {cfg.repo_id} with splits: {cfg.operation.splits}")
-    split_datasets = split_dataset(dataset, splits=cfg.operation.splits)
+    split_output_dir = Path(cfg.new_root) if cfg.new_root else None
+    split_datasets = split_dataset(
+        dataset,
+        splits=cfg.operation.splits,
+        output_dir=split_output_dir,
+    )
 
     for split_name, split_ds in split_datasets.items():
-        split_repo_id = f"{cfg.repo_id}_{split_name}"
         logging.info(
             f"{split_name}: {split_ds.meta.total_episodes} episodes, {split_ds.meta.total_frames} frames"
         )
 
         if cfg.push_to_hub:
-            logging.info(f"Pushing {split_name} split to hub as {split_repo_id}")
+            logging.info(f"Pushing {split_name} split to hub as {split_ds.repo_id}")
             LeRobotDataset(split_ds.repo_id, root=split_ds.root).push_to_hub()
 
 
@@ -298,10 +348,19 @@ def handle_merge(cfg: EditDatasetConfig) -> None:
     if not cfg.repo_id:
         raise ValueError("repo_id must be specified as the output repository for merged dataset")
 
-    logging.info(f"Loading {len(cfg.operation.repo_ids)} datasets to merge")
-    datasets = [LeRobotDataset(repo_id, root=cfg.root) for repo_id in cfg.operation.repo_ids]
+    if cfg.operation.roots:
+        if len(cfg.operation.roots) != len(cfg.operation.repo_ids):
+            raise ValueError("operation.roots and operation.repo_ids must have the same length")
+        logging.info(f"Loading {len(cfg.operation.roots)} local datasets to merge")
+        datasets = [
+            LeRobotDataset(repo_id, root=root)
+            for repo_id, root in zip(cfg.operation.repo_ids, cfg.operation.roots, strict=True)
+        ]
+    else:
+        logging.info(f"Loading {len(cfg.operation.repo_ids)} datasets to merge")
+        datasets = [LeRobotDataset(repo_id) for repo_id in cfg.operation.repo_ids]
 
-    output_dir = Path(cfg.root) / cfg.repo_id if cfg.root else HF_LEROBOT_HOME / cfg.repo_id
+    output_dir = Path(cfg.new_root) if cfg.new_root else HF_LEROBOT_HOME / cfg.repo_id
 
     logging.info(f"Merging datasets into {cfg.repo_id}")
     merged_dataset = merge_datasets(
@@ -329,11 +388,14 @@ def handle_remove_feature(cfg: EditDatasetConfig) -> None:
 
     dataset = LeRobotDataset(cfg.repo_id, root=cfg.root)
     output_repo_id, output_dir = get_output_path(
-        cfg.repo_id, cfg.new_repo_id, Path(cfg.root) if cfg.root else None
+        input_root=dataset.root,
+        new_repo_id=cfg.new_repo_id,
+        new_root=Path(cfg.new_root) if cfg.new_root else None,
+        default_repo_id=cfg.repo_id,
     )
 
-    if cfg.new_repo_id is None:
-        dataset.root = Path(str(dataset.root) + "_old")
+    if output_dir == dataset.root:
+        dataset.root = dataset.root.with_name(dataset.root.name + "_old")
 
     logging.info(f"Removing features {cfg.operation.feature_names} from {cfg.repo_id}")
     new_dataset = remove_feature(
@@ -361,9 +423,10 @@ def handle_modify_tasks(cfg: EditDatasetConfig) -> None:
     if new_task is None and episode_tasks_raw is None:
         raise ValueError("Must specify at least one of new_task or episode_tasks for modify_tasks operation")
 
-    # Warn about in-place modification behavior
-    if cfg.new_repo_id is not None:
-        logging.warning("modify_tasks modifies datasets in-place. The --new_repo_id parameter is ignored.")
+    if cfg.new_repo_id is not None or cfg.new_root is not None:
+        logging.warning(
+            "modify_tasks modifies datasets in-place. The --new_repo_id and --new_root parameters are ignored."
+        )
 
     dataset = LeRobotDataset(cfg.repo_id, root=cfg.root)
     logging.warning(f"Modifying dataset in-place at {dataset.root}. Original data will be overwritten.")
@@ -399,31 +462,24 @@ def handle_convert_image_to_video(cfg: EditDatasetConfig) -> None:
     dataset = LeRobotDataset(cfg.repo_id, root=cfg.root)
 
     # Determine output directory and repo_id
-    # Priority: 1) new_repo_id, 2) operation.output_dir, 3) auto-generated name
+    # Priority: 1) new_root, 2) new_repo_id, 3) operation.output_dir, 4) auto-generated name
     output_dir_config = getattr(cfg.operation, "output_dir", None)
 
-    if cfg.new_repo_id:
-        # Use new_repo_id for both local storage and hub push
+    if cfg.new_root:
+        output_dir = Path(cfg.new_root)
+        output_repo_id = cfg.new_repo_id or f"{cfg.repo_id}_video"
+        logging.info(f"Saving to {output_dir} as {output_repo_id}")
+    elif cfg.new_repo_id:
         output_repo_id = cfg.new_repo_id
-        # Place new dataset as a sibling to the original dataset
-        # Get the parent of the actual dataset root (not cfg.root which might be the lerobot cache dir)
-        # Extract just the dataset name (after last slash) for the local directory
-        local_dir_name = cfg.new_repo_id.split("/")[-1]
-        output_dir = dataset.root.parent / local_dir_name
+        output_dir = HF_LEROBOT_HOME / cfg.new_repo_id
         logging.info(f"Saving to new dataset: {cfg.new_repo_id} at {output_dir}")
     elif output_dir_config:
-        # Use custom output directory for local-only storage
         output_dir = Path(output_dir_config)
-        # Extract repo name from output_dir for the dataset
         output_repo_id = output_dir.name
         logging.info(f"Saving to local directory: {output_dir}")
     else:
-        # Auto-generate name: append "_video" to original repo_id
         output_repo_id = f"{cfg.repo_id}_video"
-        # Place new dataset as a sibling to the original dataset
-        # Extract just the dataset name (after last slash) for the local directory
-        local_dir_name = output_repo_id.split("/")[-1]
-        output_dir = dataset.root.parent / local_dir_name
+        output_dir = HF_LEROBOT_HOME / output_repo_id
         logging.info(f"Saving to auto-generated location: {output_dir}")
 
     logging.info(f"Converting dataset {cfg.repo_id} to video format")