revert initialization to empty

refactor(buffer): use Gymnasium terminology (terminated/truncated)
- Rename 'done' to 'terminated' for true task completion - Use 'truncated' for time-limit termination - Change torch.empty to torch.zeros for storage initialization - Convert _lerobotdataset_to_transitions to generator for memory efficiency - Add proper docstrings to BatchTransition and ReplayBuffer - Update concatenate_batch_transitions to use new terminology - Update tests to use new field names This aligns ReplayBuffer with Gymnasium's termination semantics where: - terminated: Episode ended due to task success/failure - truncated: Episode ended due to time limit or external factors
2026-05-15 16:49:55 +00:00 · 2025-12-17 16:36:59 +01:00 · 2025-12-17 15:52:26 +01:00 · 2025-12-09 19:08:09 +01:00 · 2025-12-08 18:50:21 +01:00 · 2025-12-08 14:47:54 +01:00
16 changed files with 789 additions and 203 deletions
@@ -31,7 +31,8 @@ jobs:
    name: Upload Preview and Comment
    if: >
      github.event.workflow_run.event == 'pull_request' &&
-      github.event.workflow_run.conclusion == 'success'
+      github.event.workflow_run.conclusion == 'success' &&
+      github.repository == 'huggingface/lerobot'
    uses: huggingface/doc-builder/.github/workflows/upload_pr_documentation.yml@main
    with:
      package_name: lerobot
@@ -42,7 +42,9 @@ jobs:
  # This job builds and deploys the official documentation.
  build_main_docs:
    name: Build Main Docs
-    if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'
+    if: >
+      (github.event_name == 'push' || github.event_name == 'workflow_dispatch') &&
+      github.repository == 'huggingface/lerobot'
    permissions:
      contents: read
    uses: huggingface/doc-builder/.github/workflows/build_main_documentation.yml@main
@@ -58,7 +60,7 @@ jobs:
  # The result of this job triggers the 'Upload PR Documentation' workflow.
  build_pr_docs:
    name: Build PR Docs
-    if: github.event_name == 'pull_request'
+    if: github.event_name == 'pull_request' && github.repository == 'huggingface/lerobot'
    permissions:
      contents: read
      pull-requests: write
@@ -45,7 +45,6 @@ permissions:
 env:
  UV_VERSION: "0.8.0"
  PYTHON_VERSION: "3.10"
-  DOCKER_IMAGE_NAME: huggingface/lerobot-gpu

 # Ensures that only the latest commit for a PR or branch is built, canceling older runs.
 concurrency:
@@ -43,6 +43,7 @@ jobs:
    name: Build CPU Docker for Nightly
    runs-on:
      group: aws-general-8-plus
+    if: github.repository == 'huggingface/lerobot'
    outputs:
      image_tag: ${{ env.DOCKER_IMAGE_NAME_CPU }}
    steps:
@@ -77,6 +78,7 @@ jobs:
    name: Build GPU Docker for Nightly
    runs-on:
      group: aws-general-8-plus
+    if: github.repository == 'huggingface/lerobot'
    outputs:
      image_tag: ${{ env.DOCKER_IMAGE_NAME_GPU }}
    steps:
@@ -29,6 +29,7 @@ jobs:
  build-and-publish:
    name: Build and publish Python distributions
    runs-on: ubuntu-latest
+    if: github.repository == 'huggingface/lerobot'
    outputs:
      version: ${{ steps.extract_info.outputs.tag_version }}
    permissions:
@@ -45,6 +45,7 @@ jobs:
  stale:
    name: Close Stale Issues and PRs
    runs-on: ubuntu-latest
+    if: github.repository == 'huggingface/lerobot'
    permissions:
      actions: write
      contents: write # only for delete-branch option
@@ -43,6 +43,7 @@ jobs:
  full-tests:
    name: Full Unbound Tests
    runs-on: ubuntu-latest
+    if: github.repository == 'huggingface/lerobot'
    env:
      MUJOCO_GL: egl
      HF_HOME: /mnt/cache/.cache/huggingface
@@ -11,13 +11,14 @@ LeRobot provides several utilities for manipulating datasets:
 3. **Merge Datasets** - Combine multiple datasets into one. The datasets must have identical features, and episodes are concatenated in the order specified in `repo_ids`
 4. **Add Features** - Add new features to a dataset
 5. **Remove Features** - Remove features from a dataset
+6. **Convert to Video** - Convert image-based datasets to video format for efficient storage

 The core implementation is in `lerobot.datasets.dataset_tools`.
 An example script detailing how to use the tools API is available in `examples/dataset/use_dataset_tools.py`.

 ## Command-Line Tool: lerobot-edit-dataset

-`lerobot-edit-dataset` is a command-line script for editing datasets. It can be used to delete episodes, split datasets, merge datasets, add features, and remove features.
+`lerobot-edit-dataset` is a command-line script for editing datasets. It can be used to delete episodes, split datasets, merge datasets, add features, remove features, and convert image datasets to video format.

 Run `lerobot-edit-dataset --help` for more information on the configuration of each operation.

@@ -86,9 +87,71 @@ lerobot-edit-dataset \
    --operation.feature_names "['observation.images.top']"
 ```

+#### Convert to Video
+
+Convert an image-based dataset to video format, creating a new LeRobotDataset where images are stored as videos. This is useful for reducing storage requirements and improving data loading performance. The new dataset will have the exact same structure as the original, but with images encoded as MP4 videos in the proper LeRobot format.
+
+```bash
+# Local-only: Save to a custom output directory (no hub push)
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --operation.type convert_to_video \
+    --operation.output_dir /path/to/output/pusht_video
+
+# Save with new repo_id (local storage)
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --new_repo_id lerobot/pusht_video \
+    --operation.type convert_to_video
+
+# Convert and push to Hugging Face Hub
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --new_repo_id lerobot/pusht_video \
+    --operation.type convert_to_video \
+    --push_to_hub true
+
+# Convert with custom video codec and quality settings
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --operation.type convert_to_video \
+    --operation.output_dir outputs/pusht_video \
+    --operation.vcodec libsvtav1 \
+    --operation.pix_fmt yuv420p \
+    --operation.g 2 \
+    --operation.crf 30
+
+# Convert only specific episodes
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --operation.type convert_to_video \
+    --operation.output_dir outputs/pusht_video \
+    --operation.episode_indices "[0, 1, 2, 5, 10]"
+
+# Convert with multiple workers for parallel processing
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --operation.type convert_to_video \
+    --operation.output_dir outputs/pusht_video \
+    --operation.num_workers 8
+```
+
+**Parameters:**
+
+- `output_dir`: Custom output directory (optional - by default uses `new_repo_id` or `{repo_id}_video`)
+- `vcodec`: Video codec to use - options: `h264`, `hevc`, `libsvtav1` (default: `libsvtav1`)
+- `pix_fmt`: Pixel format - options: `yuv420p`, `yuv444p` (default: `yuv420p`)
+- `g`: Group of pictures (GOP) size - lower values give better quality but larger files (default: 2)
+- `crf`: Constant rate factor - lower values give better quality but larger files, 0 is lossless (default: 30)
+- `fast_decode`: Fast decode tuning option (default: 0)
+- `episode_indices`: List of specific episodes to convert (default: all episodes)
+- `num_workers`: Number of parallel workers for processing (default: 4)
+
+**Note:** The resulting dataset will be a proper LeRobotDataset with all cameras encoded as videos in the `videos/` directory, with parquet files containing only metadata (no raw image data). All episodes, stats, and tasks are preserved.
+
 ### Push to Hub

-Add the `--push_to_hub` flag to any command to automatically upload the resulting dataset to the Hugging Face Hub:
+Add the `--push_to_hub true` flag to any command to automatically upload the resulting dataset to the Hugging Face Hub:

 ```bash
 lerobot-edit-dataset \
@@ -96,7 +159,7 @@ lerobot-edit-dataset \
    --new_repo_id lerobot/pusht_after_deletion \
    --operation.type delete_episodes \
    --operation.episode_indices "[0, 2, 5]" \
-    --push_to_hub
+    --push_to_hub true
 ```

 There is also a tool for adding features to a dataset that is not yet covered in `lerobot-edit-dataset`.
@@ -24,7 +24,7 @@ Built from pure Transformer encoders, X-VLA scales naturally with model size and
  <img
    src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot/xvla-architecture2.png"
    alt="XVLA Architecture 2"
-    style="width: 32%; max-width: 450px; height: auto;"
+    style="width: 60%; height: auto;"
  />
 </p>

@@ -120,7 +120,7 @@ Adapted for Google Robot platforms.

 ### Recommended Training Configuration

-When fine-tuning X-VLA for a new embodiment or task, we recommend the following freezing strategy:
+When fine-tuning X-VLA for a new embodiment or task, we recommend not freezing the VLM, and also setting the `policy.dtype=bfloat16` to not hit OOM errors.

 ```bash
 lerobot-train \
@@ -129,25 +129,26 @@ lerobot-train \
  --job_name=xvla_training \
  --policy.path="lerobot/xvla-base" \
  --policy.repo_id="HF_USER/xvla-your-robot" \
+  --policy.dtype=bfloat16 \
  --steps=3000 \
  --policy.device=cuda \
-  --policy.freeze_vision_encoder=True \
-  --policy.freeze_language_encoder=True \
-  --policy.train_policy_transformer=True \
-  --policy.train_soft_prompts=True \
+  --policy.freeze_vision_encoder=false \
+  --policy.freeze_language_encoder=false \
+  --policy.train_policy_transformer=true \
+  --policy.train_soft_prompts=true \
  --policy.action_mode=YOUR_ACTION_MODE
 ```

 ### Training Parameters Explained

-| Parameter                  | Default | Description                              |
-| -------------------------- | ------- | ---------------------------------------- |
-| `freeze_vision_encoder`    | `True`  | Freeze the VLM vision encoder weights    |
-| `freeze_language_encoder`  | `True`  | Freeze the VLM language encoder weights  |
-| `train_policy_transformer` | `True`  | Allow policy transformer layers to train |
-| `train_soft_prompts`       | `True`  | Allow soft prompts to train              |
+| Parameter                  | Default | Description                                    |
+| -------------------------- | ------- | ---------------------------------------------- |
+| `freeze_vision_encoder`    | `false` | Do not freeze the VLM vision encoder weights   |
+| `freeze_language_encoder`  | `false` | Do not freeze the VLM language encoder weights |
+| `train_policy_transformer` | `true`  | Allow policy transformer layers to train       |
+| `train_soft_prompts`       | `true`  | Allow soft prompts to train                    |

-**💡 Best Practice**: For Phase II adaptation to new embodiments, freeze the VLM encoders and only train the policy transformer and soft prompts. This provides excellent sample efficiency with minimal compute.
+**💡 Best Practice**: For Phase II adaptation to new embodiments, do not freeze the VLM encoders and also train the policy transformer and soft prompts.

 ### Example: Training on Bimanual Robot

@@ -157,14 +158,15 @@ lerobot-train \
  --output_dir=./outputs/xvla_bimanual \
  --job_name=xvla_so101_training \
  --policy.path="lerobot/xvla-base" \
+  --policy.dtype=bfloat16 \
  --policy.repo_id="YOUR_USERNAME/xvla-biso101" \
  --steps=3000 \
  --policy.device=cuda \
  --policy.action_mode=so101_bimanual \
-  --policy.freeze_vision_encoder=True \
-  --policy.freeze_language_encoder=True \
-  --policy.train_policy_transformer=True \
-  --policy.train_soft_prompts=True
+  --policy.freeze_vision_encoder=false \
+  --policy.freeze_language_encoder=false \
+  --policy.train_policy_transformer=true \
+  --policy.train_soft_prompts=true
 ```

 💡 **Best Performance:** If you have sufficient computational resources and want to achieve best X-VLA finetuning performance, you should follow the official finetuning strategy:
@@ -172,71 +174,7 @@ lerobot-train \
 **🔥 Full-finetune all components with a custom learning-rate scheme**

 To ensure stable optimization, the Vision-Language Model (VLM) must be trained with only 1/10 of the base learning rate, while all other components use the full LR.
-This LR ratio is crucial for achieving strong and stable finetuning performance.
-To enable this behavior, you must:
-
-1. Implement a custom optimizer and register it in your training config
-
-```
-from dataclasses import dataclass, asdict
-from lerobot.optim.optimizers import OptimizerConfig
-import torch
-
-@OptimizerConfig.register_subclass("xvla-adamw")
-@dataclass
-class XVLAAdamW(OptimizerConfig):
-    lr: float = 1e-4
-    betas: tuple[float, float] = (0.9, 0.99)
-    eps: float = 1e-8
-    weight_decay: float = 0.0
-    grad_clip_norm: float = 10.0
-
-    def build(self, params: dict) -> torch.optim.Optimizer:
-        """
-        Expect `named_parameters()` as input.
-        Apply lr = lr / 10 for all VLM-related parameters.
-        """
-        assert isinstance(params, dict), \
-            "Custom LR optimizer requires `named_parameters()` as inputs."
-        kwargs = asdict(self)
-        kwargs.pop("grad_clip_norm")
-        vlm_group, other_group = [], []
-        for name, p in params.items():
-            if not p.requires_grad:
-                continue
-            if "vlm" in name.lower():
-                vlm_group.append(p)
-            else:
-                other_group.append(p)
-
-        param_groups = [
-            {"params": vlm_group, "lr": self.lr * 0.1, "weight_decay": self.weight_decay * 0.1},
-            {"params": other_group, "lr": self.lr, "weight_decay": self.weight_decay},
-        ]
-
-        return torch.optim.AdamW(param_groups, **kwargs)
-```
-
-2. Modify X-VLA’s get_optim_params to return named parameters
-
-Replace:
-
-```
-def get_optim_params(self) -> dict:
-    """Return only trainable parameters for optimization."""
-    return filter(lambda p: p.requires_grad, self.parameters())
-```
-
-with:
-
-```
-def get_optim_params(self):
-    """Return trainable named parameters."""
-    return filter(lambda kv: kv[1].requires_grad, self.named_parameters())
-```
-
-This ensures the optimizer receives a dict of named parameters, allowing it to correctly detect VLM modules and apply the 1/10 LR rule.
-
+This LR ratio is crucial for achieving strong and stable finetuning performance. This is already done for you by default.
 ❕Note

 Completely matching the official reported performance may require an additional warm-up LR schedule for soft-prompts, which can bring minor improvements.
@@ -326,6 +264,26 @@ domain_id = 3

 The domain_id is automatically added to observations by the `XVLAAddDomainIdProcessorStep` in the preprocessing pipeline.

+The `lerobot/xvla-base` model has been trained on the following domain IDs. It is recommended to choose one that most resembles your robot/configuration:
+
+#### Fine-tuning Datasets
+
+| Dataset Name     | Domain ID |
+| ---------------- | --------- |
+| Bridge           | 0         |
+| RT1              | 1         |
+| Calvin           | 2         |
+| libero           | 3         |
+| widowx-air       | 4         |
+| AIR-AGILEX-HQ    | 5         |
+| robotwin2_abs_ee | 6         |
+| robotwin2_clean  | 6         |
+| robocasa-human   | 7         |
+| VLABench         | 8         |
+| AGIBOT-challenge | 9         |
+| AIR-AGILEX       | 10        |
+| AIRBOT           | 18        |
+
 ### 3. Processor Steps

 X-VLA requires specific preprocessing and postprocessing steps for proper operation.
@@ -812,16 +812,13 @@ class PI0Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        )

        dt = -1.0 / num_steps
-        dt = torch.tensor(dt, dtype=torch.float32, device=device)

        x_t = noise
-        time = torch.tensor(1.0, dtype=torch.float32, device=device)
-        while time >= -dt / 2:
-            expanded_time = time.expand(bsize)
+        for step in range(num_steps):
+            time = 1.0 + step * dt
+            time_tensor = torch.tensor(time, dtype=torch.float32, device=device).expand(bsize)

-            # Define a closure function to properly capture expanded_time
-            # This avoids the lambda expression (E731) and loop variable binding (B023) issues
-            def denoise_step_partial_call(input_x_t, current_timestep=expanded_time):
+            def denoise_step_partial_call(input_x_t, current_timestep=time_tensor):
                return self.denoise_step(
                    state=state,
                    prefix_pad_masks=prefix_pad_masks,
@@ -846,15 +843,11 @@ class PI0Pytorch(nn.Module):  # see openpi `PI0Pytorch`
            else:
                v_t = denoise_step_partial_call(x_t)

-            # Euler step
-            x_t += dt * v_t
+            x_t = x_t + dt * v_t

-            # Record x_t and v_t after Euler step
            if self.rtc_processor is not None and self.rtc_processor.is_debug_enabled():
                self.rtc_processor.track(time=time, x_t=x_t, v_t=v_t)

-            time += dt
-
        return x_t

    def denoise_step(
@@ -787,16 +787,13 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        )

        dt = -1.0 / num_steps
-        dt = torch.tensor(dt, dtype=torch.float32, device=device)

        x_t = noise
-        time = torch.tensor(1.0, dtype=torch.float32, device=device)
-        while time >= -dt / 2:
-            expanded_time = time.expand(bsize)
+        for step in range(num_steps):
+            time = 1.0 + step * dt
+            time_tensor = torch.tensor(time, dtype=torch.float32, device=device).expand(bsize)

-            # Define a closure function to properly capture expanded_time
-            # This avoids the lambda expression (E731) and loop variable binding (B023) issues
-            def denoise_step_partial_call(input_x_t, current_timestep=expanded_time):
+            def denoise_step_partial_call(input_x_t, current_timestep=time_tensor):
                return self.denoise_step(
                    prefix_pad_masks=prefix_pad_masks,
                    past_key_values=past_key_values,
@@ -820,15 +817,11 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
            else:
                v_t = denoise_step_partial_call(x_t)

-            # Euler step
-            x_t += dt * v_t
+            x_t = x_t + dt * v_t

-            # Record x_t and v_t after Euler step
            if self.rtc_processor is not None and self.rtc_processor.is_debug_enabled():
                self.rtc_processor.track(time=time, x_t=x_t, v_t=v_t)

-            time += dt
-
        return x_t

    def denoise_step(
@@ -783,18 +783,15 @@ class VLAFlowMatching(nn.Module):
            use_cache=self.config.use_cache,
            fill_kv_cache=True,
        )
-        dt = -1.0 / self.config.num_steps
-        dt = torch.tensor(dt, dtype=torch.float32, device=device)
+        num_steps = self.config.num_steps
+        dt = -1.0 / num_steps

        x_t = noise
-        time = torch.tensor(1.0, dtype=torch.float32, device=device)
+        for step in range(num_steps):
+            time = 1.0 + step * dt
+            time_tensor = torch.tensor(time, dtype=torch.float32, device=device).expand(bsize)

-        while time >= -dt / 2:
-            expanded_time = time.expand(bsize)
-
-            # Define a closure function to properly capture expanded_time
-            # This avoids the lambda expression (E731) and loop variable binding (B023) issues
-            def denoise_step_partial_call(input_x_t, current_timestep=expanded_time):
+            def denoise_step_partial_call(input_x_t, current_timestep=time_tensor):
                return self.denoise_step(
                    x_t=input_x_t,
                    prefix_pad_masks=prefix_pad_masks,
@@ -818,15 +815,11 @@ class VLAFlowMatching(nn.Module):
            else:
                v_t = denoise_step_partial_call(x_t)

-            # Euler step
-            x_t += dt * v_t
+            x_t = x_t + dt * v_t

-            # Record x_t and v_t after Euler step (other params are recorded in rtc_processor.denoise_step)
            if self.rtc_processor is not None and self.rtc_processor.is_debug_enabled():
                self.rtc_processor.track(time=time, x_t=x_t, v_t=v_t)

-            time += dt
-
        return x_t

    def denoise_step(
@@ -15,7 +15,8 @@
 # limitations under the License.

 import functools
-from collections.abc import Callable, Sequence
+import itertools
+from collections.abc import Callable, Generator, Sequence
 from contextlib import suppress
 from typing import TypedDict

@@ -29,13 +30,20 @@ from lerobot.utils.transition import Transition


 class BatchTransition(TypedDict):
+    """Batch transition for single-step RL algorithms.
+
+    Uses Gymnasium terminology:
+    - terminated: True termination due to task success/failure
+    - truncated: Termination due to time limit or other external factors
+    """
+
    state: dict[str, torch.Tensor]
    action: torch.Tensor
    reward: torch.Tensor
    next_state: dict[str, torch.Tensor]
-    done: torch.Tensor
-    truncated: torch.Tensor
-    complementary_info: dict[str, torch.Tensor | float | int] | None = None
+    terminated: torch.Tensor  # True termination due to task success/failure
+    truncated: torch.Tensor  # Termination due to time limit
+    complementary_info: dict[str, torch.Tensor] | None


 def random_crop_vectorized(images: torch.Tensor, output_size: tuple) -> torch.Tensor:
@@ -78,6 +86,8 @@ def random_shift(images: torch.Tensor, pad: int = 4):


 class ReplayBuffer:
+    """Replay buffer for storing transitions used in RL training (e.g., SAC)."""
+
    def __init__(
        self,
        capacity: int,
@@ -133,7 +143,7 @@ class ReplayBuffer:
        self,
        state: dict[str, torch.Tensor],
        action: torch.Tensor,
-        complementary_info: dict[str, torch.Tensor] | None = None,
+        complementary_info: dict[str, torch.Tensor | float | int] | None = None,
    ):
        """Initialize the storage tensors based on the first transition."""
        # Determine shapes from the first transition
@@ -159,8 +169,8 @@ class ReplayBuffer:
            # Just create a reference to states for consistent API
            self.next_states = self.states  # Just a reference for API consistency

-        self.dones = torch.empty((self.capacity,), dtype=torch.bool, device=self.storage_device)
-        self.truncateds = torch.empty((self.capacity,), dtype=torch.bool, device=self.storage_device)
+        self.terminated = torch.empty((self.capacity,), dtype=torch.bool, device=self.storage_device)
+        self.truncated = torch.empty((self.capacity,), dtype=torch.bool, device=self.storage_device)

        # Initialize storage for complementary_info
        self.has_complementary_info = complementary_info is not None
@@ -195,7 +205,7 @@ class ReplayBuffer:
        next_state: dict[str, torch.Tensor],
        done: bool,
        truncated: bool,
-        complementary_info: dict[str, torch.Tensor] | None = None,
+        complementary_info: dict[str, torch.Tensor | float | int] | None = None,
    ):
        """Saves a transition, ensuring tensors are stored on the designated storage device."""
        # Initialize storage if this is the first transition
@@ -212,8 +222,8 @@ class ReplayBuffer:

        self.actions[self.position].copy_(action.squeeze(dim=0))
        self.rewards[self.position] = reward
-        self.dones[self.position] = done
-        self.truncateds[self.position] = truncated
+        self.terminated[self.position] = done
+        self.truncated[self.position] = truncated

        # Handle complementary_info if provided and storage is initialized
        if complementary_info is not None and self.has_complementary_info:
@@ -283,8 +293,8 @@ class ReplayBuffer:
        # Sample other tensors
        batch_actions = self.actions[idx].to(self.device)
        batch_rewards = self.rewards[idx].to(self.device)
-        batch_dones = self.dones[idx].to(self.device).float()
-        batch_truncateds = self.truncateds[idx].to(self.device).float()
+        batch_terminated = self.terminated[idx].to(self.device).float()
+        batch_truncated = self.truncated[idx].to(self.device).float()

        # Sample complementary_info if available
        batch_complementary_info = None
@@ -298,8 +308,8 @@ class ReplayBuffer:
            action=batch_actions,
            reward=batch_rewards,
            next_state=batch_next_state,
-            done=batch_dones,
-            truncated=batch_truncateds,
+            terminated=batch_terminated,
+            truncated=batch_truncated,
            complementary_info=batch_complementary_info,
        )

@@ -431,7 +441,6 @@ class ReplayBuffer:
            device (str): The device for sampling tensors. Defaults to "cuda:0".
            state_keys (Sequence[str] | None): The list of keys that appear in `state` and `next_state`.
            capacity (int | None): Buffer capacity. If None, uses dataset length.
-            action_mask (Sequence[int] | None): Indices of action dimensions to keep.
            image_augmentation_function (Callable | None): Function for image augmentation.
                If None, uses default random shift with pad=4.
            use_drq (bool): Whether to use DrQ image augmentation when sampling.
@@ -460,12 +469,16 @@ class ReplayBuffer:
            optimize_memory=optimize_memory,
        )

-        # Convert dataset to transitions
-        list_transition = cls._lerobotdataset_to_transitions(dataset=lerobot_dataset, state_keys=state_keys)
+        # Convert dataset to transitions generator
+        transitions_generator = cls._lerobotdataset_to_transitions(
+            dataset=lerobot_dataset, state_keys=state_keys
+        )
+
+        # Get first transition to initialize storage
+        first_transition = next(transitions_generator, None)

        # Initialize the buffer with the first transition to set up storage tensors
-        if list_transition:
-            first_transition = list_transition[0]
+        if first_transition is not None:
            first_state = {k: v.to(device) for k, v in first_transition["state"].items()}
            first_action = first_transition[ACTION].to(device)

@@ -483,26 +496,28 @@ class ReplayBuffer:
                state=first_state, action=first_action, complementary_info=first_complementary_info
            )

-        # Fill the buffer with all transitions
-        for data in list_transition:
-            for k, v in data.items():
-                if isinstance(v, dict):
-                    for key, tensor in v.items():
-                        v[key] = tensor.to(storage_device)
-                elif isinstance(v, torch.Tensor):
-                    data[k] = v.to(storage_device)
+        # Fill the buffer with all transitions (first + remaining)
+        if first_transition is not None:
+            for data in itertools.chain([first_transition], transitions_generator):
+                for k, v in data.items():
+                    if isinstance(v, dict):
+                        for key, tensor in v.items():
+                            if isinstance(tensor, torch.Tensor):
+                                v[key] = tensor.to(storage_device)
+                    elif isinstance(v, torch.Tensor):
+                        data[k] = v.to(storage_device)

-            action = data[ACTION]
+                action = data[ACTION]

-            replay_buffer.add(
-                state=data["state"],
-                action=action,
-                reward=data["reward"],
-                next_state=data["next_state"],
-                done=data["done"],
-                truncated=False,  # NOTE: Truncation are not supported yet in lerobot dataset
-                complementary_info=data.get("complementary_info", None),
-            )
+                replay_buffer.add(
+                    state=data["state"],
+                    action=action,
+                    reward=data["reward"],
+                    next_state=data["next_state"],
+                    done=data["done"],
+                    truncated=data["truncated"],
+                    complementary_info=data.get("complementary_info"),
+                )

        return replay_buffer

@@ -576,10 +591,12 @@ class ReplayBuffer:
            for key in self.states:
                frame_dict[key] = self.states[key][actual_idx].cpu()

-            # Fill action, reward, done
+            # Fill action, reward, done (done = terminated or truncated)
            frame_dict[ACTION] = self.actions[actual_idx].cpu()
            frame_dict[REWARD] = torch.tensor([self.rewards[actual_idx]], dtype=torch.float32).cpu()
-            frame_dict[DONE] = torch.tensor([self.dones[actual_idx]], dtype=torch.bool).cpu()
+            frame_dict[DONE] = torch.tensor(
+                [self.terminated[actual_idx] or self.truncated[actual_idx]], dtype=torch.bool
+            ).cpu()
            frame_dict["task"] = task_name

            # Add complementary_info if available
@@ -599,7 +616,7 @@ class ReplayBuffer:
            lerobot_dataset.add_frame(frame_dict)

            # If we reached an episode boundary, call save_episode, reset counters
-            if self.dones[actual_idx] or self.truncateds[actual_idx]:
+            if self.terminated[actual_idx] or self.truncated[actual_idx]:
                lerobot_dataset.save_episode()

        # Save any remaining frames in the buffer
@@ -615,9 +632,11 @@ class ReplayBuffer:
    def _lerobotdataset_to_transitions(
        dataset: LeRobotDataset,
        state_keys: Sequence[str] | None = None,
-    ) -> list[Transition]:
+    ) -> Generator[Transition, None, None]:
        """
-        Convert a LeRobotDataset into a list of RL (s, a, r, s', done) transitions.
+        Convert a LeRobotDataset into a generator of RL (s, a, r, s', done) transitions.
+
+        Using a generator instead of a list is more memory efficient for large datasets.

        Args:
            dataset (LeRobotDataset):
@@ -637,14 +656,12 @@ class ReplayBuffer:
                ["observation.state", "observation.environment_state"].
                If None, you must handle or define default keys.

-        Returns:
-            transitions (List[Transition]):
-                A list of Transition dictionaries with the same length as `dataset`.
+        Yields:
+            Transition: A transition dictionary.
        """
        if state_keys is None:
            raise ValueError("State keys must be provided when converting LeRobotDataset to Transitions.")

-        transitions = []
        num_frames = len(dataset)

        # Check if the dataset has "next.done" key
@@ -687,8 +704,17 @@ class ReplayBuffer:
                    if next_sample["episode_index"] != current_sample["episode_index"]:
                        done = True

-            # TODO: (azouitine) Handle truncation (using the same value as done for now)
-            truncated = done
+            # Handle truncation separately from done
+            # This is important if the dataset has truncations (e.g., time limits)
+            truncated = False
+            if not done:
+                # If this is the last frame or if next frame is in a different episode, mark as truncated
+                if i == num_frames - 1:
+                    truncated = True
+                elif i < num_frames - 1:
+                    next_sample = dataset[i + 1]
+                    if next_sample["episode_index"] != current_sample["episode_index"]:
+                        truncated = True

            # ----- 4) Next state -----
            # If not done and the next sample is in the same episode, we pull the next sample's state.
@@ -716,7 +742,6 @@ class ReplayBuffer:
                    if isinstance(val, torch.Tensor):
                        complementary_info[clean_key] = val.unsqueeze(0)  # Add batch dimension
                    else:
-                        # TODO: (azouitine) Check if it's necessary to convert to tensor
                        # For non-tensor values, use directly
                        complementary_info[clean_key] = val

@@ -730,12 +755,10 @@ class ReplayBuffer:
                truncated=truncated,
                complementary_info=complementary_info,
            )
-            transitions.append(transition)

-        return transitions
+            yield transition


-# Utility function to guess shapes/dtypes from a tensor
 def guess_feature_info(t, name: str):
    """
    Return a dictionary with the 'dtype' and 'shape' for a given tensor or scalar value.
@@ -805,9 +828,9 @@ def concatenate_batch_transitions(
        for key in left_batch_transitions["next_state"]
    }

-    # Concatenate done and truncated fields
-    left_batch_transitions["done"] = torch.cat(
-        [left_batch_transitions["done"], right_batch_transition["done"]], dim=0
+    # Concatenate terminated and truncated fields
+    left_batch_transitions["terminated"] = torch.cat(
+        [left_batch_transitions["terminated"], right_batch_transition["terminated"]], dim=0
    )
    left_batch_transitions["truncated"] = torch.cat(
        [left_batch_transitions["truncated"], right_batch_transition["truncated"]],
@@ -18,7 +18,8 @@
 Edit LeRobot datasets using various transformation tools.

 This script allows you to delete episodes, split datasets, merge datasets,
-and remove features. When new_repo_id is specified, creates a new dataset.
+remove features, and convert image datasets to video format.
+When new_repo_id is specified, creates a new dataset.

 Usage Examples:

@@ -65,6 +66,25 @@ Remove camera feature:
        --operation.type remove_feature \
        --operation.feature_names "['observation.images.top']"

+Convert image dataset to video format (saves locally):
+    python -m lerobot.scripts.lerobot_edit_dataset \
+        --repo_id lerobot/pusht_image \
+        --operation.type convert_to_video \
+        --operation.output_dir /path/to/output/pusht_video
+
+Convert image dataset and save with new repo_id:
+    python -m lerobot.scripts.lerobot_edit_dataset \
+        --repo_id lerobot/pusht_image \
+        --new_repo_id lerobot/pusht_video \
+        --operation.type convert_to_video
+
+Convert and push to hub:
+    python -m lerobot.scripts.lerobot_edit_dataset \
+        --repo_id lerobot/pusht_image \
+        --new_repo_id lerobot/pusht_video \
+        --operation.type convert_to_video \
+        --push_to_hub true
+
 Using JSON config file:
    python -m lerobot.scripts.lerobot_edit_dataset \
        --config_path path/to/edit_config.json
@@ -72,9 +92,13 @@ Using JSON config file:

 import logging
 import shutil
+from concurrent.futures import ThreadPoolExecutor, as_completed
 from dataclasses import dataclass
 from pathlib import Path

+import pandas as pd
+from tqdm import tqdm
+
 from lerobot.configs import parser
 from lerobot.datasets.dataset_tools import (
    delete_episodes,
@@ -82,8 +106,10 @@ from lerobot.datasets.dataset_tools import (
    remove_feature,
    split_dataset,
 )
-from lerobot.datasets.lerobot_dataset import LeRobotDataset
-from lerobot.utils.constants import HF_LEROBOT_HOME
+from lerobot.datasets.lerobot_dataset import LeRobotDataset, LeRobotDatasetMetadata
+from lerobot.datasets.utils import write_stats, write_tasks
+from lerobot.datasets.video_utils import encode_video_frames, get_video_info
+from lerobot.utils.constants import HF_LEROBOT_HOME, OBS_IMAGE
 from lerobot.utils.utils import init_logging


@@ -111,10 +137,23 @@ class RemoveFeatureConfig:
    feature_names: list[str] | None = None


+@dataclass
+class ConvertToVideoConfig:
+    type: str = "convert_to_video"
+    output_dir: str | None = None
+    vcodec: str = "libsvtav1"
+    pix_fmt: str = "yuv420p"
+    g: int = 2
+    crf: int = 30
+    fast_decode: int = 0
+    episode_indices: list[int] | None = None
+    num_workers: int = 4
+
+
@dataclass
 class EditDatasetConfig:
    repo_id: str
-    operation: DeleteEpisodesConfig | SplitConfig | MergeConfig | RemoveFeatureConfig
+    operation: DeleteEpisodesConfig | SplitConfig | MergeConfig | RemoveFeatureConfig | ConvertToVideoConfig
    root: str | None = None
    new_repo_id: str | None = None
    push_to_hub: bool = False
@@ -258,6 +297,415 @@ def handle_remove_feature(cfg: EditDatasetConfig) -> None:
        LeRobotDataset(output_repo_id, root=output_dir).push_to_hub()


+def save_episode_images_for_video(
+    dataset: LeRobotDataset,
+    imgs_dir: Path,
+    img_key: str,
+    episode_index: int,
+    num_workers: int = 4,
+) -> None:
+    """Save images from a specific episode and camera to disk for video encoding.
+
+    Args:
+        dataset: The LeRobot dataset to extract images from
+        imgs_dir: Directory to save images to
+        img_key: The image key (camera) to extract
+        episode_index: Index of the episode to save
+        num_workers: Number of threads for parallel image saving
+    """
+    # Create directory
+    imgs_dir.mkdir(parents=True, exist_ok=True)
+
+    # Get dataset without torch format for PIL image access
+    hf_dataset = dataset.hf_dataset.with_format(None)
+
+    # Select only this camera's images
+    imgs_dataset = hf_dataset.select_columns(img_key)
+
+    # Get episode start and end indices
+    from_idx = dataset.meta.episodes["dataset_from_index"][episode_index]
+    to_idx = dataset.meta.episodes["dataset_to_index"][episode_index]
+
+    # Get all items for this episode
+    episode_dataset = imgs_dataset.select(range(from_idx, to_idx))
+
+    # Define function to save a single image
+    def save_single_image(i_item_tuple):
+        i, item = i_item_tuple
+        img = item[img_key]
+        # Use frame-XXXXXX.png format to match encode_video_frames expectations
+        img.save(str(imgs_dir / f"frame-{i:06d}.png"), quality=100)
+        return i
+
+    # Save images with proper naming convention for encode_video_frames (frame-XXXXXX.png)
+    items = list(enumerate(episode_dataset))
+
+    with ThreadPoolExecutor(max_workers=num_workers) as executor:
+        futures = [executor.submit(save_single_image, item) for item in items]
+        for future in as_completed(futures):
+            future.result()  # This will raise any exceptions that occurred
+
+
+def encode_episode_videos(
+    dataset: LeRobotDataset,
+    new_meta: LeRobotDatasetMetadata,
+    episode_index: int,
+    vcodec: str,
+    pix_fmt: str,
+    g: int,
+    crf: int,
+    fast_decode: int,
+    temp_dir: Path,
+    num_image_workers: int = 4,
+) -> dict[str, dict]:
+    """Encode videos for a single episode and return video metadata.
+
+    Args:
+        dataset: Source dataset with images
+        new_meta: Metadata object for the new video dataset
+        episode_index: Episode index to process
+        vcodec: Video codec
+        pix_fmt: Pixel format
+        g: Group of pictures size
+        crf: Constant rate factor
+        fast_decode: Fast decode tuning
+        temp_dir: Temporary directory for images
+        num_image_workers: Number of workers for saving images
+
+    Returns:
+        Dictionary mapping video keys to their metadata (chunk_index, file_index, timestamps)
+    """
+    hf_dataset = dataset.hf_dataset.with_format(None)
+    img_keys = [key for key in hf_dataset.features if key.startswith(OBS_IMAGE)]
+
+    video_metadata = {}
+    fps = int(dataset.fps)  # Convert to int for PyAV compatibility
+    episode_length = dataset.meta.episodes["length"][episode_index]
+    episode_duration = episode_length / dataset.fps  # Use original fps for duration calculation
+
+    for img_key in img_keys:
+        # Save images temporarily
+        imgs_dir = temp_dir / f"episode_{episode_index:06d}" / img_key
+        save_episode_images_for_video(dataset, imgs_dir, img_key, episode_index, num_image_workers)
+
+        # Determine chunk and file indices
+        # For simplicity, we'll put each episode in its own file
+        chunk_idx = episode_index // new_meta.chunks_size
+        file_idx = episode_index % new_meta.chunks_size
+
+        # Create video path in the new dataset structure
+        video_path = new_meta.root / new_meta.video_path.format(
+            video_key=img_key, chunk_index=chunk_idx, file_index=file_idx
+        )
+        video_path.parent.mkdir(parents=True, exist_ok=True)
+
+        # Encode video
+        encode_video_frames(
+            imgs_dir=imgs_dir,
+            video_path=video_path,
+            fps=fps,
+            vcodec=vcodec,
+            pix_fmt=pix_fmt,
+            g=g,
+            crf=crf,
+            fast_decode=fast_decode,
+            overwrite=True,
+        )
+
+        # Clean up temporary images
+        shutil.rmtree(imgs_dir)
+
+        # Store video metadata
+        video_metadata[img_key] = {
+            f"videos/{img_key}/chunk_index": chunk_idx,
+            f"videos/{img_key}/file_index": file_idx,
+            f"videos/{img_key}/from_timestamp": 0.0,
+            f"videos/{img_key}/to_timestamp": episode_duration,
+        }
+
+    return video_metadata
+
+
+def convert_dataset_to_videos(
+    dataset: LeRobotDataset,
+    output_dir: Path,
+    repo_id: str | None = None,
+    vcodec: str = "libsvtav1",
+    pix_fmt: str = "yuv420p",
+    g: int = 2,
+    crf: int = 30,
+    fast_decode: int = 0,
+    episode_indices: list[int] | None = None,
+    num_workers: int = 4,
+) -> LeRobotDataset:
+    """Convert image-based dataset to video-based dataset.
+
+    Creates a new LeRobotDataset with videos instead of images, following the proper
+    LeRobot dataset structure with videos stored in chunked MP4 files.
+
+    Args:
+        dataset: The source LeRobot dataset with images
+        output_dir: Directory to save the new video dataset
+        repo_id: Repository ID for the new dataset (default: original_id + "_video")
+        vcodec: Video codec (default: libsvtav1)
+        pix_fmt: Pixel format (default: yuv420p)
+        g: Group of pictures size (default: 2)
+        crf: Constant rate factor (default: 30)
+        fast_decode: Fast decode tuning (default: 0)
+        episode_indices: List of episode indices to convert (None = all episodes)
+        num_workers: Number of threads for parallel processing (default: 4)
+
+    Returns:
+        New LeRobotDataset with videos
+    """
+    # Check that it's an image dataset
+    if len(dataset.meta.video_keys) > 0:
+        raise ValueError(
+            f"This operation is for image datasets only. Video dataset provided: {dataset.repo_id}"
+        )
+
+    # Get all image keys
+    hf_dataset = dataset.hf_dataset.with_format(None)
+    img_keys = [key for key in hf_dataset.features if key.startswith(OBS_IMAGE)]
+
+    if len(img_keys) == 0:
+        raise ValueError(f"No image keys found in dataset {dataset.repo_id}")
+
+    # Determine which episodes to process
+    if episode_indices is None:
+        episode_indices = list(range(dataset.meta.total_episodes))
+
+    if repo_id is None:
+        repo_id = f"{dataset.repo_id}_video"
+
+    logging.info(
+        f"Converting {len(episode_indices)} episodes with {len(img_keys)} cameras from {dataset.repo_id}"
+    )
+    logging.info(f"Video codec: {vcodec}, pixel format: {pix_fmt}, GOP: {g}, CRF: {crf}")
+
+    # Create new features dict, converting image features to video features
+    new_features = {}
+    for key, value in dataset.meta.features.items():
+        if key not in img_keys:
+            new_features[key] = value
+        else:
+            # Convert image key to video format
+            new_features[key] = value.copy()
+            new_features[key]["dtype"] = "video"  # Change dtype from "image" to "video"
+            # Video info will be updated after episodes are encoded
+
+    # Create new metadata for video dataset
+    new_meta = LeRobotDatasetMetadata.create(
+        repo_id=repo_id,
+        fps=dataset.meta.fps,
+        features=new_features,
+        robot_type=dataset.meta.robot_type,
+        root=output_dir,
+        use_videos=True,
+        chunks_size=dataset.meta.chunks_size,
+        data_files_size_in_mb=dataset.meta.data_files_size_in_mb,
+        video_files_size_in_mb=dataset.meta.video_files_size_in_mb,
+    )
+
+    # Create temporary directory for image extraction
+    temp_dir = output_dir / "temp_images"
+    temp_dir.mkdir(parents=True, exist_ok=True)
+
+    # Process each episode
+    all_episode_metadata = []
+
+    try:
+        for ep_idx in tqdm(episode_indices, desc="Converting episodes to videos"):
+            # Get episode metadata from source
+            src_episode = dataset.meta.episodes[ep_idx]
+
+            # Encode videos for this episode
+            video_metadata = encode_episode_videos(
+                dataset=dataset,
+                new_meta=new_meta,
+                episode_index=ep_idx,
+                vcodec=vcodec,
+                pix_fmt=pix_fmt,
+                g=g,
+                crf=crf,
+                fast_decode=fast_decode,
+                temp_dir=temp_dir,
+                num_image_workers=num_workers,
+            )
+
+            # Build episode metadata
+            episode_meta = {
+                "episode_index": ep_idx,
+                "length": src_episode["length"],
+                "dataset_from_index": ep_idx * src_episode["length"],
+                "dataset_to_index": (ep_idx + 1) * src_episode["length"],
+            }
+
+            # Add video metadata
+            for img_key in img_keys:
+                episode_meta.update(video_metadata[img_key])
+
+            # Add data chunk/file info (using same structure as source)
+            if "data/chunk_index" in src_episode:
+                episode_meta["data/chunk_index"] = src_episode["data/chunk_index"]
+                episode_meta["data/file_index"] = src_episode["data/file_index"]
+
+            all_episode_metadata.append(episode_meta)
+
+        # Copy and transform data files (removing image columns)
+        _copy_data_without_images(dataset, new_meta, episode_indices, img_keys)
+
+        # Save episode metadata
+        episodes_df = pd.DataFrame(all_episode_metadata)
+        episodes_path = new_meta.root / "meta" / "episodes" / "chunk-000" / "file-000.parquet"
+        episodes_path.parent.mkdir(parents=True, exist_ok=True)
+        episodes_df.to_parquet(episodes_path, index=False)
+
+        # Update metadata info
+        new_meta.info["total_episodes"] = len(episode_indices)
+        new_meta.info["total_frames"] = sum(ep["length"] for ep in all_episode_metadata)
+        new_meta.info["total_tasks"] = dataset.meta.total_tasks
+        new_meta.info["splits"] = {"train": f"0:{len(episode_indices)}"}
+
+        # Update video info for all image keys (now videos)
+        # We need to manually set video info since update_video_info() checks video_keys first
+        for img_key in img_keys:
+            if not new_meta.features[img_key].get("info", None):
+                video_path = new_meta.root / new_meta.video_path.format(
+                    video_key=img_key, chunk_index=0, file_index=0
+                )
+                new_meta.info["features"][img_key]["info"] = get_video_info(video_path)
+
+        from lerobot.datasets.utils import write_info
+
+        write_info(new_meta.info, new_meta.root)
+
+        # Copy stats and tasks
+        if dataset.meta.stats is not None:
+            # Remove image stats
+            new_stats = {k: v for k, v in dataset.meta.stats.items() if k not in img_keys}
+            write_stats(new_stats, new_meta.root)
+
+        if dataset.meta.tasks is not None:
+            write_tasks(dataset.meta.tasks, new_meta.root)
+
+    finally:
+        # Clean up temporary directory
+        if temp_dir.exists():
+            shutil.rmtree(temp_dir)
+
+    logging.info(f"✓ Completed converting {dataset.repo_id} to video format")
+    logging.info(f"New dataset saved to: {output_dir}")
+
+    # Return new dataset
+    return LeRobotDataset(repo_id=repo_id, root=output_dir)
+
+
+def _copy_data_without_images(
+    src_dataset: LeRobotDataset,
+    dst_meta: LeRobotDatasetMetadata,
+    episode_indices: list[int],
+    img_keys: list[str],
+) -> None:
+    """Copy data files without image columns.
+
+    Args:
+        src_dataset: Source dataset
+        dst_meta: Destination metadata
+        episode_indices: Episodes to include
+        img_keys: Image keys to remove
+    """
+    from lerobot.datasets.utils import DATA_DIR
+
+    data_dir = src_dataset.root / DATA_DIR
+    parquet_files = sorted(data_dir.glob("*/*.parquet"))
+
+    if not parquet_files:
+        raise ValueError(f"No parquet files found in {data_dir}")
+
+    episode_set = set(episode_indices)
+
+    for src_path in tqdm(parquet_files, desc="Processing data files"):
+        df = pd.read_parquet(src_path).reset_index(drop=True)
+
+        # Filter to only include selected episodes
+        df = df[df["episode_index"].isin(episode_set)].copy()
+
+        if len(df) == 0:
+            continue
+
+        # Remove image columns
+        columns_to_drop = [col for col in img_keys if col in df.columns]
+        if columns_to_drop:
+            df = df.drop(columns=columns_to_drop)
+
+        # Get chunk and file indices from path
+        relative_path = src_path.relative_to(src_dataset.root)
+        chunk_dir = relative_path.parts[1]
+        file_name = relative_path.parts[2]
+        chunk_idx = int(chunk_dir.split("-")[1])
+        file_idx = int(file_name.split("-")[1].split(".")[0])
+
+        # Write to destination without pandas index
+        dst_path = dst_meta.root / f"data/chunk-{chunk_idx:03d}/file-{file_idx:03d}.parquet"
+        dst_path.parent.mkdir(parents=True, exist_ok=True)
+        df.to_parquet(dst_path, index=False)
+
+
+def handle_convert_to_video(cfg: EditDatasetConfig) -> None:
+    # Note: Parser may create any config type with the right fields, so we access fields directly
+    # instead of checking isinstance()
+    dataset = LeRobotDataset(cfg.repo_id, root=cfg.root)
+
+    # Determine output directory and repo_id
+    # Priority: 1) new_repo_id, 2) operation.output_dir, 3) auto-generated name
+    output_dir_config = getattr(cfg.operation, "output_dir", None)
+
+    if cfg.new_repo_id:
+        # Use new_repo_id for both local storage and hub push
+        output_repo_id = cfg.new_repo_id
+        output_dir = Path(cfg.root) / cfg.new_repo_id if cfg.root else HF_LEROBOT_HOME / cfg.new_repo_id
+        logging.info(f"Saving to new dataset: {cfg.new_repo_id}")
+    elif output_dir_config:
+        # Use custom output directory for local-only storage
+        output_dir = Path(output_dir_config)
+        # Extract repo name from output_dir for the dataset
+        output_repo_id = output_dir.name
+        logging.info(f"Saving to local directory: {output_dir}")
+    else:
+        # Auto-generate name: append "_video" to original repo_id
+        output_repo_id = f"{cfg.repo_id}_video"
+        output_dir = Path(cfg.root) / output_repo_id if cfg.root else HF_LEROBOT_HOME / output_repo_id
+        logging.info(f"Saving to auto-generated location: {output_dir}")
+
+    logging.info(f"Converting dataset {cfg.repo_id} to video format")
+
+    new_dataset = convert_dataset_to_videos(
+        dataset=dataset,
+        output_dir=output_dir,
+        repo_id=output_repo_id,
+        vcodec=getattr(cfg.operation, "vcodec", "libsvtav1"),
+        pix_fmt=getattr(cfg.operation, "pix_fmt", "yuv420p"),
+        g=getattr(cfg.operation, "g", 2),
+        crf=getattr(cfg.operation, "crf", 30),
+        fast_decode=getattr(cfg.operation, "fast_decode", 0),
+        episode_indices=getattr(cfg.operation, "episode_indices", None),
+        num_workers=getattr(cfg.operation, "num_workers", 4),
+    )
+
+    logging.info("Video dataset created successfully!")
+    logging.info(f"Location: {output_dir}")
+    logging.info(f"Episodes: {new_dataset.meta.total_episodes}")
+    logging.info(f"Frames: {new_dataset.meta.total_frames}")
+
+    if cfg.push_to_hub:
+        logging.info(f"Pushing to hub as {output_repo_id}...")
+        new_dataset.push_to_hub()
+        logging.info("✓ Successfully pushed to hub!")
+    else:
+        logging.info("Dataset saved locally (not pushed to hub)")
+
+
@parser.wrap()
 def edit_dataset(cfg: EditDatasetConfig) -> None:
    operation_type = cfg.operation.type
@@ -270,10 +718,12 @@ def edit_dataset(cfg: EditDatasetConfig) -> None:
        handle_merge(cfg)
    elif operation_type == "remove_feature":
        handle_remove_feature(cfg)
+    elif operation_type == "convert_to_video":
+        handle_convert_to_video(cfg)
    else:
        raise ValueError(
            f"Unknown operation type: {operation_type}\n"
-            f"Available operations: delete_episodes, split, merge, remove_feature"
+            f"Available operations: delete_episodes, split, merge, remove_feature, convert_to_video"
        )


@@ -29,6 +29,7 @@ from lerobot.datasets.dataset_tools import (
    remove_feature,
    split_dataset,
 )
+from lerobot.scripts.lerobot_edit_dataset import convert_dataset_to_videos


@pytest.fixture
@@ -1047,3 +1048,107 @@ def test_modify_features_preserves_file_structure(sample_dataset, tmp_path):
        assert new_chunk_indices == original_chunk_indices, "Chunk indices should be preserved"
        assert new_file_indices == original_file_indices, "File indices should be preserved"
        assert "reward" in modified_dataset.meta.features
+
+
+def test_convert_dataset_to_videos(tmp_path):
+    """Test converting lerobot/pusht_image dataset to video format."""
+    from lerobot.datasets.lerobot_dataset import LeRobotDataset
+
+    # Load the actual lerobot/pusht_image dataset (only first 2 episodes for speed)
+    source_dataset = LeRobotDataset("lerobot/pusht_image", episodes=[0, 1])
+
+    output_dir = tmp_path / "pusht_video"
+
+    with (
+        patch("lerobot.datasets.lerobot_dataset.get_safe_version") as mock_get_safe_version,
+        patch("lerobot.datasets.lerobot_dataset.snapshot_download") as mock_snapshot_download,
+    ):
+        mock_get_safe_version.return_value = "v3.0"
+        mock_snapshot_download.return_value = str(output_dir)
+
+        # Verify source dataset has images, not videos
+        assert len(source_dataset.meta.video_keys) == 0
+        assert "observation.image" in source_dataset.meta.features
+
+        # Convert to video dataset (only first 2 episodes for speed)
+        video_dataset = convert_dataset_to_videos(
+            dataset=source_dataset,
+            output_dir=output_dir,
+            repo_id="lerobot/pusht_video",
+            vcodec="libsvtav1",
+            pix_fmt="yuv420p",
+            g=2,
+            crf=30,
+            episode_indices=[0, 1],
+            num_workers=2,
+        )
+
+        # Verify new dataset has videos
+        assert len(video_dataset.meta.video_keys) > 0
+        assert "observation.image" in video_dataset.meta.video_keys
+
+        # Verify correct number of episodes and frames (2 episodes)
+        assert video_dataset.meta.total_episodes == 2
+        # Compare against the actual number of frames in the loaded episodes, not metadata total
+        assert len(video_dataset) == len(source_dataset)
+
+        # Verify video files exist
+        for ep_idx in range(video_dataset.meta.total_episodes):
+            for video_key in video_dataset.meta.video_keys:
+                video_path = video_dataset.root / video_dataset.meta.get_video_file_path(ep_idx, video_key)
+                assert video_path.exists(), f"Video file should exist: {video_path}"
+
+        # Verify we can load the dataset and access it
+        assert len(video_dataset) == video_dataset.meta.total_frames
+
+        # Test that we can actually get an item from the video dataset
+        item = video_dataset[0]
+        assert "observation.image" in item
+        assert "action" in item
+
+        # Cleanup
+        import shutil
+
+        if output_dir.exists():
+            shutil.rmtree(output_dir)
+
+
+def test_convert_dataset_to_videos_subset_episodes(tmp_path):
+    """Test converting only specific episodes from lerobot/pusht_image to video format."""
+    from lerobot.datasets.lerobot_dataset import LeRobotDataset
+
+    # Load the actual lerobot/pusht_image dataset (only first 3 episodes)
+    source_dataset = LeRobotDataset("lerobot/pusht_image", episodes=[0, 1, 2])
+
+    output_dir = tmp_path / "pusht_video_subset"
+
+    with (
+        patch("lerobot.datasets.lerobot_dataset.get_safe_version") as mock_get_safe_version,
+        patch("lerobot.datasets.lerobot_dataset.snapshot_download") as mock_snapshot_download,
+    ):
+        mock_get_safe_version.return_value = "v3.0"
+        mock_snapshot_download.return_value = str(output_dir)
+
+        # Convert only episode 0 to video (subset of loaded episodes)
+        episode_indices = [0]
+
+        video_dataset = convert_dataset_to_videos(
+            dataset=source_dataset,
+            output_dir=output_dir,
+            repo_id="lerobot/pusht_video_subset",
+            episode_indices=episode_indices,
+            num_workers=2,
+        )
+
+        # Verify correct number of episodes
+        assert video_dataset.meta.total_episodes == len(episode_indices)
+
+        # Verify video files exist for selected episodes
+        assert len(video_dataset.meta.video_keys) > 0
+        assert "observation.image" in video_dataset.meta.video_keys
+
+        # Cleanup
+        import shutil
+
+        if output_dir.exists():
+            shutil.rmtree(output_dir)
@@ -68,7 +68,7 @@ def create_dummy_transition() -> dict:
        OBS_STATE: torch.randn(
            10,
        ),
-        "done": torch.tensor(False),
+        "terminated": torch.tensor(False),
        "truncated": torch.tensor(False),
        "complementary_info": {},
    }
@@ -191,8 +191,8 @@ def test_add_transition(replay_buffer, dummy_state, dummy_action):
        "Action should be equal to the first transition."
    )
    assert replay_buffer.rewards[0] == 1.0, "Reward should be equal to the first transition."
-    assert not replay_buffer.dones[0], "Done should be False for the first transition."
-    assert not replay_buffer.truncateds[0], "Truncated should be False for the first transition."
+    assert not replay_buffer.terminated[0], "Terminated should be False for the first transition."
+    assert not replay_buffer.truncated[0], "Truncated should be False for the first transition."

    for dim in state_dims():
        assert torch.equal(replay_buffer.states[dim][0], dummy_state[dim]), (
@@ -232,8 +232,8 @@ def test_add_over_capacity():
        "Action should be equal to the last transition."
    )
    assert replay_buffer.rewards[0] == 1.0, "Reward should be equal to the last transition."
-    assert replay_buffer.dones[0], "Done should be True for the first transition."
-    assert replay_buffer.truncateds[0], "Truncated should be True for the first transition."
+    assert replay_buffer.terminated[0], "Terminated should be True for the first transition."
+    assert replay_buffer.truncated[0], "Truncated should be True for the first transition."


 def test_sample_from_empty_buffer(replay_buffer):
@@ -250,7 +250,7 @@ def test_sample_with_1_transition(replay_buffer, dummy_state, next_dummy_state,
        action=dummy_action.clone(),
        reward=1.0,
        next_state=clone_state(next_dummy_state),
-        done=False,
+        terminated=False,
        truncated=False,
    )

@@ -289,7 +289,7 @@ def test_sample_with_batch_bigger_than_buffer_size(
        action=dummy_action,
        reward=1.0,
        next_state=next_dummy_state,
-        done=False,
+        terminated=False,
        truncated=False,
    )

@@ -383,7 +383,8 @@ def test_to_lerobot_dataset(tmp_path):
            elif feature == REWARD:
                assert torch.equal(value, buffer.rewards[i])
            elif feature == DONE:
-                assert torch.equal(value, buffer.dones[i])
+                # DONE in dataset is terminated OR truncated
+                assert torch.equal(value, buffer.terminated[i] | buffer.truncated[i])
            elif feature == OBS_IMAGE:
                # Tensor -> numpy is not precise, so we have some diff there
                # TODO: Check and fix it
@@ -427,12 +428,12 @@ def test_from_lerobot_dataset(tmp_path):
        reconverted_buffer.rewards[: len(replay_buffer)], replay_buffer.rewards[: len(replay_buffer)]
    ), "Rewards from converted buffer should be equal to the original replay buffer."
    assert torch.equal(
-        reconverted_buffer.dones[: len(replay_buffer)], replay_buffer.dones[: len(replay_buffer)]
-    ), "Dones from converted buffer should be equal to the original replay buffer."
+        reconverted_buffer.terminated[: len(replay_buffer)], replay_buffer.terminated[: len(replay_buffer)]
+    ), "Terminated flags from converted buffer should be equal to the original replay buffer."

-    # Lerobot DS haven't supported truncateds yet
-    expected_truncateds = torch.zeros(len(replay_buffer)).bool()
-    assert torch.equal(reconverted_buffer.truncateds[: len(replay_buffer)], expected_truncateds), (
+    # LeRobot DS hasn't supported truncated yet
+    expected_truncated = torch.zeros(len(replay_buffer)).bool()
+    assert torch.equal(reconverted_buffer.truncated[: len(replay_buffer)], expected_truncated), (
        "Truncateds from converted buffer should be equal False"
    )

@@ -498,7 +499,7 @@ def test_buffer_sample_alignment():
        action_val = batch[ACTION][i].item()
        reward_val = batch["reward"][i].item()
        next_state_sig = batch["next_state"]["state_value"][i].item()
-        is_done = batch["done"][i].item() > 0.5
+        is_terminated = batch["terminated"][i].item() > 0.5

        # Verify relationships
        assert abs(action_val - 2.0 * state_sig) < 1e-4, (
@@ -509,9 +510,9 @@ def test_buffer_sample_alignment():
            f"Reward {reward_val} should be 3x state signature {state_sig}"
        )

-        if is_done:
+        if is_terminated:
            assert abs(next_state_sig - state_sig) < 1e-4, (
-                f"For done states, next_state {next_state_sig} should equal state {state_sig}"
+                f"For terminated states, next_state {next_state_sig} should equal state {state_sig}"
            )
        else:
            # Either it's the next sequential state (+0.01) or same state (for episode boundaries)
Author	SHA1	Message	Date
Michel Aractingi	e3539cb78e	revert initialization to empty	2025-12-17 16:36:59 +01:00
Michel Aractingi	9014f9a7c5	refactor(buffer): use Gymnasium terminology (terminated/truncated) - Rename 'done' to 'terminated' for true task completion - Use 'truncated' for time-limit termination - Change torch.empty to torch.zeros for storage initialization - Convert _lerobotdataset_to_transitions to generator for memory efficiency - Add proper docstrings to BatchTransition and ReplayBuffer - Update concatenate_batch_transitions to use new terminology - Update tests to use new field names This aligns ReplayBuffer with Gymnasium's termination semantics where: - terminated: Episode ended due to task success/failure - truncated: Episode ended due to time limit or external factors	2025-12-17 15:52:26 +01:00
Jade Choghari	cb920235c4	docs: update X-VLA training strategies/commands (#2611 )	2025-12-09 19:08:09 +01:00
Jade Choghari	7f40b3bf82	feat(dataset): add tool to convert images to video datasets (#2560 ) * add video encoding tool * style * make it work * more fixes	2025-12-08 18:50:21 +01:00
Michel Aractingi	2e9c9fd832	Replay while loop in sample actions with for loops (#2600 )	2025-12-08 14:47:54 +01:00
Steven Palma	f9cb5e659c	chore(ci): skip workflows if not lerobot repository (#2601 ) Co-authored-by: Alex Tyshka <atyshka15@gmail.com>	2025-12-08 12:44:36 +01:00