run precommit and fix issues

Merge branch 'main' into feature/basic-peft-support
feat: support wallx model (#2593 )
2026-05-12 15:19:43 +00:00 · 2025-12-22 14:50:58 +01:00 · 2025-12-22 14:29:51 +01:00 · 2025-12-22 10:12:39 +01:00 · 2025-12-22 10:04:33 +01:00 · 2025-12-20 19:20:12 +01:00
154 changed files with 14593 additions and 12158 deletions
@@ -12,57 +12,83 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

-name: "\U0001F41B Bug Report"
-description: Submit a bug report to help us improve LeRobot
+name: "🚀 Issue / Bug / Request"
+description: Report a bug, suggest an improvement, or ask a technical question.
 body:
  - type: markdown
    attributes:
      value: |
-        Thanks for taking the time to submit a bug report! 🐛
-        If this is not a bug related to the LeRobot library directly, but instead a general question about your code or the library specifically please use our [discord](https://discord.gg/s3KuuzsPFb).
+        ### Thanks for contributing to LeRobot! 🙌
+        Please choose the most relevant sections below. If this is a general "how-to" question, consider our [Discord](https://discord.gg/s3KuuzsPFb) for faster community support.
+
+  - type: dropdown
+    id: issue-type
+    attributes:
+      label: Ticket Type
+      description: What kind of ticket are you opening?
+      options:
+        - "🐛 Bug Report (Something isn't working)"
+        - "💡 Feature Request / Improvement"
+        - "❓ Technical Question"
+        - "🧹 Maintenance / Documentation"
+    validations:
+      required: true

  - type: textarea
    id: system-info
    attributes:
-      label: System Info
-      description: Please share your LeRobot configuration by running `lerobot-info` (if installed) or `python -m lerobot.scripts.display_sys_info` (if not installed) and pasting the output below.
+      label: Environment & System Info
+      description: |
+        For bugs or technical questions, please run `lerobot-info` and paste the output.
+        (Optional for feature requests).
      render: Shell
-      placeholder: lerobot version, OS, python version, numpy version, torch version, and lerobot's configuration
+      placeholder: lerobot version, OS, python version, etc.
+
+  - type: textarea
+    id: description
    validations:
      required: true
+    attributes:
+      label: Description
+      description: |
+        Provide a clear summary of the issue or your proposal.
+        - **Bugs:** What is happening?
+        - **Features:** What is the goal/use case?
+        - **Questions:** What are you trying to achieve?
+      placeholder: |
+        A clear and concise description of the issue or suggestion.
+
+  - type: textarea
+    id: context-repro
+    attributes:
+      label: Context & Reproduction
+      description: |
+        Provide a code snippet, steps to reproduce a bug, or technical details about your proposal.
+        Please use code blocks for scripts and CLI commands.
+      placeholder: |
+        Steps to reproduce / Usage example:
+        1.
+        2.
+        3.
+
+  - type: textarea
+    id: logs
+    attributes:
+      label: Relevant logs or stack trace
+      description: If applicable, paste relevant error logs here.
+      render: Shell

  - type: checkboxes
-    id: information-scripts-examples
+    id: extras
    attributes:
-      label: Information
-      description: 'The problem arises when using:'
+      label: Checklist
      options:
-        - label: "One of the scripts in the examples/ folder of LeRobot"
-        - label: "My own task or dataset (give details below)"
+        - label: I have searched existing tickets to ensure this isn't a duplicate.
+        - label: I am using the latest version of the `main` branch.
+        - label: I have verified this is not an environment-specific problem.

  - type: textarea
-    id: reproduction
-    validations:
-      required: true
+    id: workaround
    attributes:
-      label: Reproduction
-      description: |
-        If needed, provide a simple code sample that reproduces the problem you ran into. It can be a Colab link or just a code snippet.
-        Sharing error messages or stack traces could be useful as well!
-        Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
-        Try to avoid screenshots, as they are hard to read and don't allow copy-and-pasting.
-
-      placeholder: |
-        Steps to reproduce the behavior:
-
-          1.
-          2.
-          3.
-
-  - type: textarea
-    id: expected-behavior
-    validations:
-      required: true
-    attributes:
-      label: Expected behavior
-      description: "A clear and concise description of what you would expect to happen."
+      label: Additional Info / Workarounds
+      description: Anything else we should know? If you have a workaround, please share it!
@@ -1,41 +1,54 @@
-## What this does
+## Title

-Explain what this PR does. Feel free to tag your PR with the appropriate label(s).
+Short, imperative summary (e.g., "fix(robots): handle None in sensor parser"). See [CONTRIBUTING.md](../CONTRIBUTING.md) for PR conventions.

-Examples:
-| Title | Label |
-|----------------------|-----------------|
-| Fixes #[issue] | (🐛 Bug) |
-| Adds new dataset | (🗃️ Dataset) |
-| Optimizes something | (⚡️ Performance) |
+## Type / Scope

-## How it was tested
+- **Type**: (Bug | Feature | Docs | Performance | Test | CI | Chore)
+- **Scope**: (optional — name of module or package affected)

-Explain/show how you tested your changes.
+## Summary / Motivation

-Examples:
+- One-paragraph description of what changes and why.
+- Why this change is needed and any trade-offs or design notes.

- Added `test_something` in `tests/test_stuff.py`.
- Added `new_feature` and checked that training converges with policy X on dataset/environment Y.
- Optimized `some_function`, it now runs X times faster than previously.
+## Related issues

-## How to checkout & try? (for the reviewer)
+- Fixes / Closes: # (if any)
+- Related: # (if any)

-Provide a simple way for the reviewer to try out your changes.
+## What changed

-Examples:
+- Short, concrete bullets of the modifications (files/behaviour).
+- Short note if this introduces breaking changes and migration steps.

-```bash
-pytest -sx tests/test_stuff.py::test_something
-```
+## How was this tested

-```bash
-lerobot-train --some.option=true
-```
+- Tests added: list new tests or test files.
+- Manual checks / dataset runs performed.

-## SECTION TO REMOVE BEFORE SUBMITTING YOUR PR
+## How to run locally (reviewer)

-**Note**: Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
-members/contributors who may be interested in your PR. Try to avoid tagging more than 3 people.
+- Run the relevant tests:

-**Note**: Before submitting this PR, please read the [contributor guideline](https://github.com/huggingface/lerobot/blob/main/CONTRIBUTING.md#submitting-a-pull-request-pr).
+  ```bash
+  pytest -q tests/ -k <keyword>
+  ```
+
+- Run a quick example or CLI (if applicable):
+
+  ```bash
+  lerobot-train --some.option=true
+  ```
+
+## Checklist (required before merge)
+
+- [ ] Linting/formatting run (`pre-commit run -a`)
+- [ ] All tests pass locally (`pytest`)
+- [ ] Documentation updated
+- [ ] CI is green
+
+## Reviewer notes
+
+- Anything the reviewer should focus on (performance, edge-cases, specific files) or general notes.
+- Anyone in the community is free to review the PR.
@@ -0,0 +1,69 @@
+# Copyright 2025 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+CI:
+  - changed-files:
+      - any-glob-to-any-file:
+        - '.github/**'
+        - 'docker/**'
+
+github_actions:
+  - changed-files:
+      - any-glob-to-any-file: '.github/**'
+
+documentation:
+  - changed-files:
+      - any-glob-to-any-file:
+          - '**/*.md'
+          - '**/*.mdx'
+          - 'docs/**'
+
+examples:
+  - changed-files:
+      - any-glob-to-any-file: 'examples/**'
+
+tests:
+  - changed-files:
+      - any-glob-to-any-file: 'tests/**'
+
+sensors:
+  - changed-files:
+      - any-glob-to-any-file: 'src/lerobot/cameras/**'
+
+configuration:
+  - changed-files:
+      - any-glob-to-any-file: 'src/lerobot/configs/**'
+
+dataset:
+  - changed-files:
+      - any-glob-to-any-file: 'src/lerobot/datasets/**'
+
+evaluation:
+  - changed-files:
+      - any-glob-to-any-file: 'src/lerobot/envs/**'
+
+robots:
+  - changed-files:
+      - any-glob-to-any-file:
+          - 'src/lerobot/teleoperators/**'
+          - 'src/lerobot/robots/**'
+          - 'src/lerobot/motors/**'
+
+policies:
+  - changed-files:
+      - any-glob-to-any-file: 'src/lerobot/policies/**'
+
+processor:
+  - changed-files:
+      - any-glob-to-any-file: 'src/lerobot/processor/**'
@@ -31,7 +31,8 @@ jobs:
    name: Upload Preview and Comment
    if: >
      github.event.workflow_run.event == 'pull_request' &&
-      github.event.workflow_run.conclusion == 'success'
+      github.event.workflow_run.conclusion == 'success' &&
+      github.repository == 'huggingface/lerobot'
    uses: huggingface/doc-builder/.github/workflows/upload_pr_documentation.yml@main
    with:
      package_name: lerobot
@@ -42,7 +42,9 @@ jobs:
  # This job builds and deploys the official documentation.
  build_main_docs:
    name: Build Main Docs
-    if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'
+    if: >
+      (github.event_name == 'push' || github.event_name == 'workflow_dispatch') &&
+      github.repository == 'huggingface/lerobot'
    permissions:
      contents: read
    uses: huggingface/doc-builder/.github/workflows/build_main_documentation.yml@main
@@ -58,7 +60,7 @@ jobs:
  # The result of this job triggers the 'Upload PR Documentation' workflow.
  build_pr_docs:
    name: Build PR Docs
-    if: github.event_name == 'pull_request'
+    if: github.event_name == 'pull_request' && github.repository == 'huggingface/lerobot'
    permissions:
      contents: read
      pull-requests: write
@@ -45,7 +45,6 @@ permissions:
 env:
  UV_VERSION: "0.8.0"
  PYTHON_VERSION: "3.10"
-  DOCKER_IMAGE_NAME: huggingface/lerobot-gpu

 # Ensures that only the latest commit for a PR or branch is built, canceling older runs.
 concurrency:
@@ -85,7 +85,7 @@ jobs:
          python-version: ${{ env.PYTHON_VERSION }}

      - name: Install lerobot with all extras
-        run: uv sync --all-extras --no-extra groot # TODO(Steven): Make flash-attn optional
+        run: uv sync --all-extras --no-extra groot --no-extra wallx # TODO(Steven): Make flash-attn optional

      - name: Run pytest (all extras)
        run: uv run pytest tests -vv --maxfail=10
@@ -0,0 +1,89 @@
+# Copyright 2025 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This workflow automatically labels issues based on their content.
+name: Issue Labeler
+on:
+  # Trigger on new issues and edits to existing issues
+  issues:
+    types: [opened, edited]
+
+permissions:
+  contents: read
+  issues: write
+
+jobs:
+  label-issue:
+    name: Auto Label Issue
+    runs-on: ubuntu-latest
+    if: github.repository == 'huggingface/lerobot'
+    steps:
+      - uses: actions/github-script@v8
+        with:
+          script: |
+            // Setup Input Text
+            const body = (context.payload.issue.body || '');
+            const title = (context.payload.issue.title || '');
+            const cleanBody = body.replace(/```[\s\S]*?```/g, '');
+            const text = `${title}\n${cleanBody}`.toLowerCase();
+            const labelsToAdd = new Set();
+            const matches = (re) => re.test(text);
+
+            // Keyword Heuristics
+
+            // Domain Specific
+            if (matches(/\b(bug|error|issue|fault|crash|exception)\b/i)) labelsToAdd.add('bug');
+            if (matches(/\b(feature|enhancement|improvement|support|implement|proposal)\b/i)) labelsToAdd.add('enhancement');
+            if (matches(/\b(question|help|how to||clarify|explain|unclear)\b/i)) labelsToAdd.add('question');
+            if (matches(/\b(maintenance|documentation|docs|readme|tutorial|guide|wiki)\b/i)) labelsToAdd.add('documentation');
+            if (matches(/\b(example|script|sample|demo|notebook)s?\b/i)) labelsToAdd.add('examples');
+            if (matches(/\b(datasets?|data loader|data augmentation|data preprocessing)\b/i)) labelsToAdd.add('dataset');
+            if (matches(/\b(mujoco|isaac|simulation|sim)\b/i)) labelsToAdd.add('simulation');
+            if (matches(/\b(train|training|loss|optimizer|backward|gradient|wandb|sac)\b/i)) labelsToAdd.add('training');
+            if (matches(/\b(rerun|plot|video|render|visualiz|gif)/i)) labelsToAdd.add('visualization');
+            if (matches(/\b(camera|realsense|lidar|depth|sensor|imu|microphone|rgbd)\b/i)) labelsToAdd.add('sensors');
+            if (matches(/\b(aloha|koch|so-100|so100|mobile|teleop|manipulator|robots?)\b/i)) labelsToAdd.add('robots');
+            if (matches(/\b(teleop|teleoperator|controller|leader|follower|joystick|gamepad)\b/i)) labelsToAdd.add('teleoperators');
+            if (matches(/\b(policy|policies|p0licy)\b/i)) labelsToAdd.add('policies');
+            if (matches(/\b(processors?|pipeline)\b/i)) labelsToAdd.add('processor');
+            if (matches(/\b(eval|evaluate|evaluation|metrics?|score|benchmark)\b/i)) labelsToAdd.add('evaluation');
+
+            // Infrastructure & Code Quality
+            if (matches(/\b(tests?|pytest|unittest|failing test)\b/i)) labelsToAdd.add('tests');
+            if (matches(/\b(ci|github actions|workflow|gha|actions?|pipeline)\b/i)) {
+              labelsToAdd.add('CI');
+              labelsToAdd.add('github_actions');
+            }
+            if (matches(/\b(perf|latency|throughput|fps|speed|performance)\b/i)) labelsToAdd.add('performance');
+            if (matches(/\b(dependency|requirements|pip|conda|install error|importerror|package not found)\b/i)) labelsToAdd.add('dependencies');
+            if (matches(/\b(python|pyproject|requirements(\.txt)?|pip install|typing error)\b/i)) labelsToAdd.add('python');
+
+            // Documentation & Meta
+            if (matches(/\b(doc|documentation|docs|readme|typo|how to)\b/i)) labelsToAdd.add('documentation');
+            if (matches(/\b(refactor|cleanup|restructure|rename|modernize code)\b/i)) labelsToAdd.add('refactor');
+            if (matches(/\b(release|changelog|version bump|cut a release|tag v)\b/i)) labelsToAdd.add('release');
+            if (matches(/\b(breaking change|major change)\b/i)) labelsToAdd.add('breaking change');
+
+            // Apply Labels
+            const labels = Array.from(labelsToAdd).filter(Boolean);
+
+            if (labels.length > 0) {
+              console.log(`Adding labels: ${labels.join(', ')}`);
+              await github.rest.issues.addLabels({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                issue_number: context.issue.number,
+                labels,
+              });
+            }
@@ -43,6 +43,7 @@ jobs:
    name: Build CPU Docker for Nightly
    runs-on:
      group: aws-general-8-plus
+    if: github.repository == 'huggingface/lerobot'
    outputs:
      image_tag: ${{ env.DOCKER_IMAGE_NAME_CPU }}
    steps:
@@ -77,6 +78,7 @@ jobs:
    name: Build GPU Docker for Nightly
    runs-on:
      group: aws-general-8-plus
+    if: github.repository == 'huggingface/lerobot'
    outputs:
      image_tag: ${{ env.DOCKER_IMAGE_NAME_GPU }}
    steps:
@@ -0,0 +1,39 @@
+# Copyright 2025 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This workflow labels pull requests based on the files that were changed.
+name: Pull Request Labeler
+
+on:
+  # Allows labeling pull requests when they are opened or updated
+  # zizmor: ignore[dangerous-triggers] Needed to label PRs from forks
+  pull_request_target:
+    branches:
+      - main
+    types: [opened, synchronize, reopened, ready_for_review]
+
+permissions:
+  contents: read
+  pull-requests: write
+
+jobs:
+  triage:
+    name: Label PR
+    runs-on: ubuntu-latest
+    if: github.repository == 'huggingface/lerobot' && !github.event.pull_request.draft
+    steps:
+      - uses: actions/labeler@v6
+        with:
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          sync-labels: true # Removes labels if files are removed from the PR
@@ -29,6 +29,7 @@ jobs:
  build-and-publish:
    name: Build and publish Python distributions
    runs-on: ubuntu-latest
+    if: github.repository == 'huggingface/lerobot'
    outputs:
      version: ${{ steps.extract_info.outputs.tag_version }}
    permissions:
@@ -45,6 +45,7 @@ jobs:
  stale:
    name: Close Stale Issues and PRs
    runs-on: ubuntu-latest
+    if: github.repository == 'huggingface/lerobot'
    permissions:
      actions: write
      contents: write # only for delete-branch option
@@ -43,6 +43,7 @@ jobs:
  full-tests:
    name: Full Unbound Tests
    runs-on: ubuntu-latest
+    if: github.repository == 'huggingface/lerobot'
    env:
      MUJOCO_GL: egl
      HF_HOME: /mnt/cache/.cache/huggingface
@@ -77,7 +78,7 @@ jobs:
          echo "Dependencies unbound:" && cat pyproject.toml

      - name: Install lerobot with all extras
-        run: uv sync --all-extras --no-extra groot # TODO(Steven): Make flash-attn optional
+        run: uv sync --all-extras --no-extra groot --no-extra wallx # TODO(Steven): Make flash-attn optional

      - name: Run pytest (all extras)
        run: uv run pytest tests -vv
@@ -87,7 +87,7 @@ repos:
  # TODO(Steven): Uncomment when ready to use
  ##### Static Analysis & Typing #####
  - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v1.18.2
+    rev: v1.19.1
    hooks:
      - id: mypy
        args: [--config-file=pyproject.toml]
@@ -52,7 +52,7 @@ decisions when appropriate.

 This Code of Conduct applies within all community spaces, and also applies when
 an individual is officially representing the community in public spaces.
-Examples of representing our community include using an official email address,
+Examples of representing our community include using an official e-mail address,
 posting via an official social media account, or acting as an appointed
 representative at an online or offline event.

@@ -60,7 +60,7 @@ representative at an online or offline event.

 Instances of abusive, harassing, or otherwise unacceptable behavior may be
 reported to the community leaders responsible for enforcement at
-[feedback@huggingface.co](mailto:feedback@huggingface.co).
+feedback@huggingface.co.
 All complaints will be reviewed and investigated promptly and fairly.

 All community leaders are obligated to respect the privacy and security of the
@@ -1,7 +1,5 @@
 <p align="center">
-  <img alt="LeRobot, Hugging Face Robotics Library" src="https://raw.githubusercontent.com/huggingface/lerobot/main/media/lerobot-logo-thumbnail.png" width="100%">
-  <br/>
-  <br/>
+  <img alt="LeRobot, Hugging Face Robotics Library" src="./media/readme/lerobot-logo-thumbnail.png" width="100%">
 </p>

 <div align="center">
@@ -12,323 +10,130 @@
 [![Status](https://img.shields.io/pypi/status/lerobot)](https://pypi.org/project/lerobot/)
 [![Version](https://img.shields.io/pypi/v/lerobot)](https://pypi.org/project/lerobot/)
 [![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.1-ff69b4.svg)](https://github.com/huggingface/lerobot/blob/main/CODE_OF_CONDUCT.md)
-[![Discord](https://dcbadge.vercel.app/api/server/C5P34WJ68S?style=flat)](https://discord.gg/s3KuuzsPFb)
-
-<!-- [![Coverage](https://codecov.io/gh/huggingface/lerobot/branch/main/graph/badge.svg?token=TODO)](https://codecov.io/gh/huggingface/lerobot) -->

 </div>

-<h2 align="center">
-    <p><a href="https://huggingface.co/docs/lerobot/hope_jr">
-        Build Your Own HopeJR Robot!</a></p>
-</h2>
+**LeRobot** aims to provide models, datasets, and tools for real-world robotics in PyTorch. The goal is to lower the barrier to entry so that everyone can contribute to and benefit from shared datasets and pretrained models.

-<div align="center">
-  <img
-    src="https://raw.githubusercontent.com/huggingface/lerobot/main/media/hope_jr/hopejr.png"
-    alt="HopeJR robot"
-    title="HopeJR robot"
-    width="60%"
-  />
+🤗 A hardware-agnostic, Python-native interface that standardizes control across diverse platforms, from low-cost arms (SO-100) to humanoids.

-  <p><strong>Meet HopeJR – A humanoid robot arm and hand for dexterous manipulation!</strong></p>
-  <p>Control it with exoskeletons and gloves for precise hand movements.</p>
-  <p>Perfect for advanced manipulation tasks! 🤖</p>
+🤗 A standardized, scalable LeRobotDataset format (Parquet + MP4 or images) hosted on the Hugging Face Hub, enabling efficient storage, streaming and visualization of massive robotic datasets.

-  <p><a href="https://huggingface.co/docs/lerobot/hope_jr">
-      See the full HopeJR tutorial here.</a></p>
-</div>
+🤗 State-of-the-art policies that have been shown to transfer to the real-world ready for training and deployment.

-<br/>
+🤗 Comprehensive support for the open-source ecosystem to democratize physical AI.

-<h2 align="center">
-    <p><a href="https://huggingface.co/docs/lerobot/so101">
-        Build Your Own SO-101 Robot!</a></p>
-</h2>
+## Quick Start

-<div align="center">
-  <table>
-    <tr>
-      <td align="center"><img src="https://raw.githubusercontent.com/huggingface/lerobot/main/media/so101/so101.webp" alt="SO-101 follower arm" title="SO-101 follower arm" width="90%"/></td>
-      <td align="center"><img src="https://raw.githubusercontent.com/huggingface/lerobot/main/media/so101/so101-leader.webp" alt="SO-101 leader arm" title="SO-101 leader arm" width="90%"/></td>
-    </tr>
-  </table>
-
-  <p><strong>Meet the updated SO100, the SO-101 – Just €114 per arm!</strong></p>
-  <p>Train it in minutes with a few simple moves on your laptop.</p>
-  <p>Then sit back and watch your creation act autonomously! 🤯</p>
-
-  <p><a href="https://huggingface.co/docs/lerobot/so101">
-      See the full SO-101 tutorial here.</a></p>
-
-  <p>Want to take it to the next level? Make your SO-101 mobile by building LeKiwi!</p>
-  <p>Check out the <a href="https://huggingface.co/docs/lerobot/lekiwi">LeKiwi tutorial</a> and bring your robot to life on wheels.</p>
-
-  <img src="https://raw.githubusercontent.com/huggingface/lerobot/main/media/lekiwi/kiwi.webp" alt="LeKiwi mobile robot" title="LeKiwi mobile robot" width="50%">
-</div>
-
-<br/>
-
-<h3 align="center">
-    <p>LeRobot: State-of-the-art AI for real-world robotics</p>
-</h3>
-
---
-
-🤗 LeRobot aims to provide models, datasets, and tools for real-world robotics in PyTorch. The goal is to lower the barrier to entry to robotics so that everyone can contribute and benefit from sharing datasets and pretrained models.
-
-🤗 LeRobot contains state-of-the-art approaches that have been shown to transfer to the real-world with a focus on imitation learning and reinforcement learning.
-
-🤗 LeRobot already provides a set of pretrained models, datasets with human collected demonstrations, and simulation environments to get started without assembling a robot. In the coming weeks, the plan is to add more and more support for real-world robotics on the most affordable and capable robots out there.
-
-🤗 LeRobot hosts pretrained models and datasets on this Hugging Face community page: [huggingface.co/lerobot](https://huggingface.co/lerobot)
-
-#### Examples of pretrained models on simulation environments
-
-<table>
-  <tr>
-    <td><img src="https://raw.githubusercontent.com/huggingface/lerobot/main/media/gym/aloha_act.gif" width="100%" alt="ACT policy on ALOHA env"/></td>
-    <td><img src="https://raw.githubusercontent.com/huggingface/lerobot/main/media/gym/simxarm_tdmpc.gif" width="100%" alt="TDMPC policy on SimXArm env"/></td>
-    <td><img src="https://raw.githubusercontent.com/huggingface/lerobot/main/media/gym/pusht_diffusion.gif" width="100%" alt="Diffusion policy on PushT env"/></td>
-  </tr>
-  <tr>
-    <td align="center">ACT policy on ALOHA env</td>
-    <td align="center">TDMPC policy on SimXArm env</td>
-    <td align="center">Diffusion policy on PushT env</td>
-  </tr>
-</table>
-
-## Installation
-
-LeRobot works with Python 3.10+ and PyTorch 2.2+.
-
-### Environment Setup
-
-Create a virtual environment with Python 3.10 and activate it, e.g. with [`miniforge`](https://conda-forge.org/download/):
-
-```bash
-conda create -y -n lerobot python=3.10
-conda activate lerobot
-```
-
-When using `conda`, install `ffmpeg` in your environment:
-
-```bash
-conda install ffmpeg -c conda-forge
-```
-
-> **NOTE:** This usually installs `ffmpeg 7.X` for your platform compiled with the `libsvtav1` encoder. If `libsvtav1` is not supported (check supported encoders with `ffmpeg -encoders`), you can:
->
-> - _[On any platform]_ Explicitly install `ffmpeg 7.X` using:
->
-> ```bash
-> conda install ffmpeg=7.1.1 -c conda-forge
-> ```
->
-> - _[On Linux only]_ Install [ffmpeg build dependencies](https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu#GettheDependencies) and [compile ffmpeg from source with libsvtav1](https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu#libsvtav1), and make sure you use the corresponding ffmpeg binary to your install with `which ffmpeg`.
-
-### Install LeRobot 🤗
-
-#### From Source
-
-First, clone the repository and navigate into the directory:
-
-```bash
-git clone https://github.com/huggingface/lerobot.git
-cd lerobot
-```
-
-Then, install the library in editable mode. This is useful if you plan to contribute to the code.
-
-```bash
-pip install -e .
-```
-
-> **NOTE:** If you encounter build errors, you may need to install additional dependencies (`cmake`, `build-essential`, and `ffmpeg libs`). On Linux, run:
-> `sudo apt-get install cmake build-essential python3-dev pkg-config libavformat-dev libavcodec-dev libavdevice-dev libavutil-dev libswscale-dev libswresample-dev libavfilter-dev`. For other systems, see: [Compiling PyAV](https://pyav.org/docs/develop/overview/installation.html#bring-your-own-ffmpeg)
-
-For simulations, 🤗 LeRobot comes with gymnasium environments that can be installed as extras:
-
- [aloha](https://github.com/huggingface/gym-aloha)
- [xarm](https://github.com/huggingface/gym-xarm)
- [pusht](https://github.com/huggingface/gym-pusht)
-
-For instance, to install 🤗 LeRobot with aloha and pusht, use:
-
-```bash
-pip install -e ".[aloha, pusht]"
-```
-
-### Installation from PyPI
-
-**Core Library:**
-Install the base package with:
+LeRobot can be installed directly from PyPI.

 ```bash
 pip install lerobot
+lerobot-info
 ```

-_This installs only the default dependencies._
+> [!IMPORTANT]
+> For detailed installation guide, please see the [Installation Documentation](https://huggingface.co/docs/lerobot/installation).

-**Extra Features:**
-To install additional functionality, use one of the following:
+## Robots & Control
+
+<div align="center">
+  <img src="./media/readme/robots_control_video.webp" width="640px" alt="Reachy 2 Demo">
+</div>
+
+LeRobot provides a unified `Robot` class interface that decouples control logic from hardware specifics. It supports a wide range of robots and teleoperation devices.
+
+```python
+from lerobot.robots.myrobot import MyRobot
+
+# Connect to a robot
+robot = MyRobot(config=...)
+robot.connect()
+
+# Read observation and send action
+obs = robot.get_observation()
+action = model.select_action(obs)
+robot.send_action(action)
+```
+
+**Supported Hardware:** SO100, LeKiwi, Koch, HopeJR, OMX, EarthRover, Reachy2, Gamepads, Keyboards, Phones, OpenARM, Unitree G1.
+
+While these devices are natively integrated into the LeRobot codebase, the library is designed to be extensible. You can easily implement the Robot interface to utilize LeRobot's data collection, training, and visualization tools for your own custom robot.
+
+For detailed hardware setup guides, see the [Hardware Documentation](https://huggingface.co/docs/lerobot/integrate_hardware).
+
+## LeRobot Dataset
+
+To solve the data fragmentation problem in robotics, we utilize the **LeRobotDataset** format.
+
+- **Structure:** Synchronized MP4 videos (or images) for vision and Parquet files for state/action data.
+- **HF Hub Integration:** Explore thousands of robotics datasets on the [Hugging Face Hub](https://huggingface.co/lerobot).
+- **Tools:** Seamlessly delete episodes, split by indices/fractions, add/remove features, and merge multiple datasets.
+
+```python
+from lerobot.datasets.lerobot_dataset import LeRobotDataset
+
+# Load a dataset from the Hub
+dataset = LeRobotDataset("lerobot/aloha_mobile_cabinet")
+
+# Access data (automatically handles video decoding)
+episode_index=0
+print(f"{dataset[episode_index]['action'].shape=}\n")
+```
+
+Learn more about it in the [LeRobotDataset Documentation](https://huggingface.co/docs/lerobot/lerobot-dataset-v3)
+
+## SoTA Models
+
+LeRobot implements state-of-the-art policies in pure PyTorch, covering Imitation Learning, Reinforcement Learning, and Vision-Language-Action (VLA) models, with more coming soon. It also provides you with the tools to instrument and inspect your training process.
+
+<p align="center">
+  <img alt="Gr00t Architecture" src="./media/readme/VLA_architecture.jpg" width="640px">
+</p>
+
+Training a policy is as simple as running a script configuration:

 ```bash
-pip install 'lerobot[all]'          # All available features
-pip install 'lerobot[aloha,pusht]'  # Specific features (Aloha & Pusht)
-pip install 'lerobot[feetech]'      # Feetech motor support
+lerobot-train \
+  --policy=act \
+  --dataset.repo_id=lerobot/aloha_mobile_cabinet
 ```

-_Replace `[...]` with your desired features._
+| Category                   | Models                                                                                                                                                                 |
+| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Imitation Learning**     | [ACT](./docs/source/policy_act_README.md), [Diffusion](./docs/source/policy_diffusion_README.md), [VQ-BeT](./docs/source/policy_vqbet_README.md)                       |
+| **Reinforcement Learning** | [HIL-SERL](./docs/source/hilserl.mdx), [TDMPC](./docs/source/policy_tdmpc_README.md) & QC-FQL (coming soon)                                                            |
+| **VLAs Models**            | [Pi0.5](./docs/source/pi05.mdx), [GR00T N1.5](./docs/source/policy_groot_README.md), [SmolVLA](./docs/source/policy_smolvla_README.md), [XVLA](./docs/source/xvla.mdx) |

-**Available Tags:**
-For a full list of optional dependencies, see:
-https://pypi.org/project/lerobot/
+Similarly to the hardware, you can easily implement your own policy & leverage LeRobot's data collection, training, and visualization tools, and share your model to the HF Hub

-> [!NOTE]
-> For lerobot 0.4.0, if you want to install pi tags, you will have to do: `pip install "lerobot[pi]@git+https://github.com/huggingface/lerobot.git"`.
->
-> This will be solved in the next patch release
+For detailed policy setup guides, see the [Policy Documentation](https://huggingface.co/docs/lerobot/bring_your_own_policies).

-### Weights & Biases
+## Inference & Evaluation

-To use [Weights and Biases](https://docs.wandb.ai/quickstart) for experiment tracking, log in with
+Evaluate your policies in simulation or on real hardware using the unified evaluation script. LeRobot supports standard benchmarks like **LIBERO**, **MetaWorld** and more to come.

 ```bash
-wandb login
+# Evaluate a policy on the LIBERO benchmark
+lerobot-eval \
+  --policy.path=lerobot/pi0_libero_finetuned \
+  --env.type=libero \
+  --env.task=libero_object \
+  --eval.n_episodes=10
 ```

-(note: you will also need to enable WandB in the configuration. See below.)
+Learn how to implement your own simulation environment or benchmark and distribute it from the HF Hub by following the [EnvHub Documentation](https://huggingface.co/docs/lerobot/envhub)

-### Visualize datasets
+## Resources

-Check out [example 1](https://github.com/huggingface/lerobot/blob/main/examples/dataset/load_lerobot_dataset.py) that illustrates how to use our dataset class which automatically downloads data from the Hugging Face hub.
-
-You can also locally visualize episodes from a dataset on the hub by executing our script from the command line:
-
-```bash
-lerobot-dataset-viz \
-    --repo-id lerobot/pusht \
-    --episode-index 0
-```
-
-or from a dataset in a local folder with the `root` option and the `--mode local` (in the following case the dataset will be searched for in `./my_local_data_dir/lerobot/pusht`)
-
-```bash
-lerobot-dataset-viz \
-    --repo-id lerobot/pusht \
-    --root ./my_local_data_dir \
-    --mode local \
-    --episode-index 0
-```
-
-It will open `rerun.io` and display the camera streams, robot states and actions, like this:
-
-https://github-production-user-asset-6210df.s3.amazonaws.com/4681518/328035972-fd46b787-b532-47e2-bb6f-fd536a55a7ed.mov?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240505%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240505T172924Z&X-Amz-Expires=300&X-Amz-Signature=d680b26c532eeaf80740f08af3320d22ad0b8a4e4da1bcc4f33142c15b509eda&X-Amz-SignedHeaders=host&actor_id=24889239&key_id=0&repo_id=748713144
-
-Our script can also visualize datasets stored on a distant server. See `lerobot-dataset-viz --help` for more instructions.
-
-### The `LeRobotDataset` format
-
-A dataset in `LeRobotDataset` format is very simple to use. It can be loaded from a repository on the Hugging Face hub or a local folder simply with e.g. `dataset = LeRobotDataset("lerobot/aloha_static_coffee")` and can be indexed into like any Hugging Face and PyTorch dataset. For instance `dataset[0]` will retrieve a single temporal frame from the dataset containing observation(s) and an action as PyTorch tensors ready to be fed to a model.
-
-A specificity of `LeRobotDataset` is that, rather than retrieving a single frame by its index, we can retrieve several frames based on their temporal relationship with the indexed frame, by setting `delta_timestamps` to a list of relative times with respect to the indexed frame. For example, with `delta_timestamps = {"observation.image": [-1, -0.5, -0.2, 0]}` one can retrieve, for a given index, 4 frames: 3 "previous" frames 1 second, 0.5 seconds, and 0.2 seconds before the indexed frame, and the indexed frame itself (corresponding to the 0 entry). See example [1_load_lerobot_dataset.py](https://github.com/huggingface/lerobot/blob/main/examples/dataset/load_lerobot_dataset.py) for more details on `delta_timestamps`.
-
-Under the hood, the `LeRobotDataset` format makes use of several ways to serialize data which can be useful to understand if you plan to work more closely with this format. We tried to make a flexible yet simple dataset format that would cover most type of features and specificities present in reinforcement learning and robotics, in simulation and in real-world, with a focus on cameras and robot states but easily extended to other types of sensory inputs as long as they can be represented by a tensor.
-
-Here are the important details and internal structure organization of a typical `LeRobotDataset` instantiated with `dataset = LeRobotDataset("lerobot/aloha_static_coffee")`. The exact features will change from dataset to dataset but not the main aspects:
-
-```
-dataset attributes:
-  ├ hf_dataset: a Hugging Face dataset (backed by Arrow/parquet). Typical features example:
-  │  ├ observation.images.cam_high (VideoFrame):
-  │  │   VideoFrame = {'path': path to a mp4 video, 'timestamp' (float32): timestamp in the video}
-  │  ├ observation.state (list of float32): position of an arm joints (for instance)
-  │  ... (more observations)
-  │  ├ action (list of float32): goal position of an arm joints (for instance)
-  │  ├ episode_index (int64): index of the episode for this sample
-  │  ├ frame_index (int64): index of the frame for this sample in the episode ; starts at 0 for each episode
-  │  ├ timestamp (float32): timestamp in the episode
-  │  ├ next.done (bool): indicates the end of an episode ; True for the last frame in each episode
-  │  └ index (int64): general index in the whole dataset
-  ├ meta: a LeRobotDatasetMetadata object containing:
-  │  ├ info: a dictionary of metadata on the dataset
-  │  │  ├ codebase_version (str): this is to keep track of the codebase version the dataset was created with
-  │  │  ├ fps (int): frame per second the dataset is recorded/synchronized to
-  │  │  ├ features (dict): all features contained in the dataset with their shapes and types
-  │  │  ├ total_episodes (int): total number of episodes in the dataset
-  │  │  ├ total_frames (int): total number of frames in the dataset
-  │  │  ├ robot_type (str): robot type used for recording
-  │  │  ├ data_path (str): formattable string for the parquet files
-  │  │  └ video_path (str): formattable string for the video files (if using videos)
-  │  ├ episodes: a DataFrame containing episode metadata with columns:
-  │  │  ├ episode_index (int): index of the episode
-  │  │  ├ tasks (list): list of tasks for this episode
-  │  │  ├ length (int): number of frames in this episode
-  │  │  ├ dataset_from_index (int): start index of this episode in the dataset
-  │  │  └ dataset_to_index (int): end index of this episode in the dataset
-  │  ├ stats: a dictionary of statistics (max, mean, min, std) for each feature in the dataset, for instance
-  │  │  ├ observation.images.front_cam: {'max': tensor with same number of dimensions (e.g. `(c, 1, 1)` for images, `(c,)` for states), etc.}
-  │  │  └ ...
-  │  └ tasks: a DataFrame containing task information with task names as index and task_index as values
-  ├ root (Path): local directory where the dataset is stored
-  ├ image_transforms (Callable): optional image transformations to apply to visual modalities
-  └ delta_timestamps (dict): optional delta timestamps for temporal queries
-```
-
-A `LeRobotDataset` is serialised using several widespread file formats for each of its parts, namely:
-
- hf_dataset stored using Hugging Face datasets library serialization to parquet
- videos are stored in mp4 format to save space
- metadata are stored in plain json/jsonl files
-
-Dataset can be uploaded/downloaded from the HuggingFace hub seamlessly. To work on a local dataset, you can specify its location with the `root` argument if it's not in the default `~/.cache/huggingface/lerobot` location.
-
-#### Reproduce state-of-the-art (SOTA)
-
-We provide some pretrained policies on our [hub page](https://huggingface.co/lerobot) that can achieve state-of-the-art performances.
-You can reproduce their training by loading the config from their run. Simply running:
-
-```bash
-lerobot-train --config_path=lerobot/diffusion_pusht
-```
-
-reproduces SOTA results for Diffusion Policy on the PushT task.
-
-## Contribute
-
-If you would like to contribute to 🤗 LeRobot, please check out our [contribution guide](https://github.com/huggingface/lerobot/blob/main/CONTRIBUTING.md).
-
-### Add a pretrained policy
-
-Once you have trained a policy you may upload it to the Hugging Face hub using a hub id that looks like `${hf_user}/${repo_name}` (e.g. [lerobot/diffusion_pusht](https://huggingface.co/lerobot/diffusion_pusht)).
-
-You first need to find the checkpoint folder located inside your experiment directory (e.g. `outputs/train/2024-05-05/20-21-12_aloha_act_default/checkpoints/002500`). Within that there is a `pretrained_model` directory which should contain:
-
- `config.json`: A serialized version of the policy configuration (following the policy's dataclass config).
- `model.safetensors`: A set of `torch.nn.Module` parameters, saved in [Hugging Face Safetensors](https://huggingface.co/docs/safetensors/index) format.
- `train_config.json`: A consolidated configuration containing all parameters used for training. The policy configuration should match `config.json` exactly. This is useful for anyone who wants to evaluate your policy or for reproducibility.
-
-To upload these to the hub, run the following:
-
-```bash
-huggingface-cli upload ${hf_user}/${repo_name} path/to/pretrained_model
-```
-
-See [lerobot_eval.py](https://github.com/huggingface/lerobot/blob/main/src/lerobot/scripts/lerobot_eval.py) for an example of how other people may use your policy.
-
-### Acknowledgment
-
- The LeRobot team 🤗 for building SmolVLA [Paper](https://arxiv.org/abs/2506.01844), [Blog](https://huggingface.co/blog/smolvla).
- Thanks to Tony Zhao, Zipeng Fu and colleagues for open sourcing ACT policy, ALOHA environments and datasets. Ours are adapted from [ALOHA](https://tonyzhaozh.github.io/aloha) and [Mobile ALOHA](https://mobile-aloha.github.io).
- Thanks to Cheng Chi, Zhenjia Xu and colleagues for open sourcing Diffusion policy, Pusht environment and datasets, as well as UMI datasets. Ours are adapted from [Diffusion Policy](https://diffusion-policy.cs.columbia.edu) and [UMI Gripper](https://umi-gripper.github.io).
- Thanks to Nicklas Hansen, Yunhai Feng and colleagues for open sourcing TDMPC policy, Simxarm environments and datasets. Ours are adapted from [TDMPC](https://github.com/nicklashansen/tdmpc) and [FOWM](https://www.yunhaifeng.com/FOWM).
- Thanks to Antonio Loquercio and Ashish Kumar for their early support.
- Thanks to [Seungjae (Jay) Lee](https://sjlee.cc/), [Mahi Shafiullah](https://mahis.life/) and colleagues for open sourcing [VQ-BeT](https://sjlee.cc/vq-bet/) policy and helping us adapt the codebase to our repository. The policy is adapted from [VQ-BeT repo](https://github.com/jayLEE0301/vq_bet_official).
+- **[Documentation](https://huggingface.co/docs/lerobot/index):** The complete guide to tutorials & API.
+- **[Discord](https://discord.gg/3gxM6Avj):** Join the `LeRobot` server to discuss with the community.
+- **[X](https://x.com/LeRobotHF):** Follow us on X to stay up-to-date with the latest developments.
+- **[Robot Learning Tutorial](https://huggingface.co/spaces/lerobot/robot-learning-tutorial):** A free, hands-on course to learn robot learning using LeRobot.

 ## Citation

-If you want, you can cite this work with:
+If you use LeRobot in your research, please cite:

 ```bibtex
@misc{cadene2024lerobot,
@@ -339,6 +144,14 @@ If you want, you can cite this work with:
 }
 ```

-## Star History
+## Contribute

-[![Star History Chart](https://api.star-history.com/svg?repos=huggingface/lerobot&type=Timeline)](https://star-history.com/#huggingface/lerobot&Timeline)
+We welcome contributions from everyone in the community! To get started, please read our [CONTRIBUTING.md](./CONTRIBUTING.md) guide. Whether you're adding a new feature, improving documentation, or fixing a bug, your help and feedback are invaluable. We're incredibly excited about the future of open-source robotics and can't wait to work with you on what's next—thank you for your support!
+
+<p align="center">
+  <img alt="SO101 Video" src="./media/readme/so100_video.webp" width="640px">
+</p>
+
+<div align="center">
+<sub>Built by the <a href="https://huggingface.co/lerobot">LeRobot</a> team at <a href="https://huggingface.co">Hugging Face</a> with ❤️</sub>
+</div>
@@ -19,6 +19,8 @@
    title: Train RL in Simulation
  - local: multi_gpu_training
    title: Multi GPU training
+  - local: peft_training
+    title: Training with PEFT (e.g., LoRA)
  title: "Tutorials"
 - sections:
  - local: lerobot-dataset-v3
@@ -42,6 +44,10 @@
  - local: xvla
    title: X-VLA
  title: "Policies"
+- sections:
+  - local: sarm
+    title: SARM
+  title: "Reward Models"
 - sections:
  - local: async
    title: Use Async Inference
@@ -201,7 +201,8 @@ from lerobot.teleoperators.so100_leader.so100_leader import SO100Leader
 from lerobot.utils.control_utils import init_keyboard_listener
 from lerobot.utils.utils import log_say
 from lerobot.utils.visualization_utils import init_rerun
-from lerobot.record import record_loop
+from lerobot.scripts.lerobot_record import record_loop
+from lerobot.processor import make_default_processors

 NUM_EPISODES = 5
 FPS = 30
@@ -209,12 +210,19 @@ EPISODE_TIME_SEC = 60
 RESET_TIME_SEC = 10
 TASK_DESCRIPTION = "My task description"

-# Create the robot and teleoperator configurations
-camera_config = {"front": OpenCVCameraConfig(index_or_path=0, width=640, height=480, fps=FPS)}
+# Create robot configuration
 robot_config = SO100FollowerConfig(
-    port="/dev/tty.usbmodem58760434471", id="my_awesome_follower_arm", cameras=camera_config
+    id="my_awesome_follower_arm",
+    cameras={
+        "front": OpenCVCameraConfig(index_or_path=0, width=640, height=480, fps=FPS) # Optional: fourcc="MJPG" for troubleshooting OpenCV async error.
+    },
+    port="/dev/tty.usbmodem58760434471",
+)
+
+teleop_config = SO100LeaderConfig(
+    id="my_awesome_leader_arm",
+    port="/dev/tty.usbmodem585A0077581",
 )
-teleop_config = SO100LeaderConfig(port="/dev/tty.usbmodem585A0077581", id="my_awesome_leader_arm")

 # Initialize the robot and teleoperator
 robot = SO100Follower(robot_config)
@@ -243,6 +251,9 @@ init_rerun(session_name="recording")
 robot.connect()
 teleop.connect()

+# Create the required processors
+teleop_action_processor, robot_action_processor, robot_observation_processor = make_default_processors()
+
 episode_idx = 0
 while episode_idx < NUM_EPISODES and not events["stop_recording"]:
    log_say(f"Recording episode {episode_idx + 1} of {NUM_EPISODES}")
@@ -251,6 +262,9 @@ while episode_idx < NUM_EPISODES and not events["stop_recording"]:
        robot=robot,
        events=events,
        fps=FPS,
+        teleop_action_processor=teleop_action_processor,
+        robot_action_processor=robot_action_processor,
+        robot_observation_processor=robot_observation_processor,
        teleop=teleop,
        dataset=dataset,
        control_time_s=EPISODE_TIME_SEC,
@@ -265,6 +279,9 @@ while episode_idx < NUM_EPISODES and not events["stop_recording"]:
            robot=robot,
            events=events,
            fps=FPS,
+            teleop_action_processor=teleop_action_processor,
+            robot_action_processor=robot_action_processor,
+            robot_observation_processor=robot_observation_processor,
            teleop=teleop,
            control_time_s=RESET_TIME_SEC,
            single_task=TASK_DESCRIPTION,
@@ -0,0 +1,62 @@
+# Parameter efficient fine-tuning with 🤗 PEFT
+
+[🤗 PEFT](https://github.com/huggingface/peft) (Parameter-Efficient Fine-Tuning) is a library for efficiently adapting
+large pretrained models such as pre-trained policies (e.g., SmolVLA, π₀, ...) to new tasks without training all
+of the model's parameters while yielding comparable performance.
+
+Install the `lerobot[peft]` optional package to enable PEFT support.
+
+To read about all the possible methods of adaption, please refer to the [🤗 PEFT docs](https://huggingface.co/docs/peft/index).
+
+## Training SmolVLA
+
+In this section we'll show you how to train a pre-trained SmolVLA policy with PEFT on the libero dataset.
+For brevity we're only training on the `libero_spatial` subset. We will use `lerobot/smolvla_base` as the model
+to parameter efficiently fine-tune:
+
+```
+lerobot-train \
+ --policy.path=lerobot/smolvla_base \
+ --policy.repo_id=your_hub_name/my_libero_smolvla \
+ --dataset.repo_id=HuggingFaceVLA/libero \
+ --policy.output_features=null \
+ --policy.input_features=null \
+ --policy.optimizer_lr=1e-3 \
+ --policy.scheduler_decay_lr=1e-4 \
+ --env.type=libero \
+ --env.task=libero_spatial \
+ --steps=100000 \
+ --batch_size=32 \
+ --peft.method_type=LORA \
+ --peft.r=64
+```
+
+Note the `--peft.method_type` parameter that let's you select which PEFT method to use. Here we use
+[LoRA](https://huggingface.co/docs/peft/main/en/package_reference/lora) (Low-Rank Adapter) which is probably the most
+popular fine-tuning method to date. Low-rank adaption means that we only fine-tune a matrix with comparably low rank
+instead of the full weight matrix. This rank can be specified using the `--peft.r` parameter. The higher the rank
+the closer you get to full fine-tuning
+
+There are more complex methods that have more parameters. These are not yet supported, feel free to raise an issue
+if you want to see a specific PEFT method supported.
+
+By default, PEFT will target the `q_proj` and `v_proj` layers of the LM expert in SmolVLA. It will also target the
+state and action projection matrices as they are most likely task-dependent. If you need to target different layers
+you can use `--peft.target_modules` to specify which layers to target. You can refer to the respective PEFT method's
+documentation to see what inputs are supported, (e.g., [LoRA's target_modules documentation](https://huggingface.co/docs/peft/main/en/package_reference/lora#peft.LoraConfig.target_modules)).
+Usually a list of suffixes or a regex are supported. For example, to target the MLPs of the `lm_expert` instead of
+the `q` and `v` projections, use:
+
+```
+--peft.target_modules='(model\.vlm_with_expert\.lm_expert\..*\.(down|gate|up)_proj|.*\.(state_proj|action_in_proj|action_out_proj|action_time_mlp_in|action_time_mlp_out))'
+```
+
+In case you need to fully fine-tune a layer instead of just adapting it, you can supply a list of layer suffixes
+to the `--peft.full_training_modules` parameter:
+
+```
+--peft.full_training_modules=["state_proj"]
+```
+
+The learning rate and the scheduled target learning rate can usually be scaled by a factor of 10 compared to the
+learning rate used for full fine-tuning (e.g., 1e-4 normal, so 1e-3 using LoRA).
@@ -0,0 +1,586 @@
+# SARM: Stage-Aware Reward Modeling
+
+SARM (Stage-Aware Reward Modeling) is a video-based reward modeling framework for long-horizon robot manipulation tasks. This guide covers how to train SARM reward models and optionally use them with Reward-Aligned Behavior Cloning (RA-BC).
+
+**Paper**: [SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulation](https://arxiv.org/abs/2509.25358)
+
+## Why Reward Models?
+
+Standard behavior cloning treats all demonstration frames equally, but real-world robot datasets are messy. They contain hesitations, corrections, and variable-quality trajectories. Reward models solve this by learning a generalizable notion of **task progress** from demonstrations: given video frames and a task description, they predict how close the robot is to completing the task (0→1). This learned "progress signal" can be used in multiple ways, two promising applications are: (1) **weighted imitation learning** (RA-BC), where high-progress frames receive more weight during policy training, and (2) **reinforcement learning**, where the reward model provides dense rewards for online or offline policy improvement.
+
+## Overview
+
+SARM has following features:
+
+1. **Stage-aware architecture**: Jointly predicts the high-level task stage and fine-grained progress within each stage
+2. **Subtask annotations**: Uses natural language subtask annotations to derive consistent progress labels
+3. **Temporal proportions**: Computes dataset-level priors (α̅\_k) for each subtask to normalize progress across variable-length demonstrations
+
+SARM trains on a compact **stage+tau** target for each frame:
+
+- **stage**: integer stage index `k ∈ {0, ..., K-1}`
+- **τ (tau)**: within-stage progress `τ ∈ [0, 1]`
+- **target encoding**: `y = k + τ` (this is what the dataset processor produces)
+
+At inference time (and in downstream RA-BC), SARM converts the raw `k + τ` value into a **normalized progress** in `[0, 1]` using dataset-level **temporal proportions** `α̅_k` (stored in `meta/temporal_proportions_*.json`).
+
+This matches **Formula (2)** from the paper:
+
+```
+progress_t = P_{k-1} + α̅_k × τ_t
+```
+
+Where:
+
+- `τ_t = (t - s_k) / (e_k - s_k)` is within-subtask normalized time
+- `P_{k-1}` is cumulative prior (sum of previous subtask proportions)
+- `α̅_k` is the temporal proportion for subtask k
+
+This ensures identical task states map to consistent progress values, even across demonstrations of different lengths.
+
+## Inputs and Targets (What the new code expects)
+
+SARM is trained through its processor (`src/lerobot/policies/sarm/processor_sarm.py`), which:
+
+- **Encodes** images and task text with CLIP (ViT-B/32) into `video_features` and `text_features`
+- **Pads/truncates** robot state into `state_features` (up to `max_state_dim`)
+- **Builds targets** as `sparse_targets` (and `dense_targets` in `dense_only`/`dual`) using the stage+tau encoding `y = k + τ`
+- **Masks rewind frames** using a per-sample `lengths` tensor (rewind is a training-time augmentation)
+
+At minimum, each training sample needs:
+
+- `task` (string): task description
+- `policy.image_key` images and `policy.state_key` states from the dataset
+
+---
+
+## Annotation Modes
+
+You can choose from **3 annotation modes** that determine how progress labels are computed:
+
+| Mode           | Annotations Required | Heads                        | Use Case                                                     |
+| -------------- | -------------------- | ---------------------------- | ------------------------------------------------------------ |
+| `single_stage` | None                 | Sparse only                  | Simple tasks, quick experiments, no VLM needed               |
+| `dense_only`   | Dense (VLM)          | Dual (sparse auto-generated) | Detailed subtask tracking without defining high-level stages |
+| `dual`         | Sparse + Dense (VLM) | Dual                         | Full SARM paper setup with both granularities                |
+
+### Mode Details
+
+<hfoptions id="mode_explanation">
+<hfoption id="single_stage">
+
+**No annotations required.** The entire episode is treated as a single stage called `"task"`, and progress is linear from 0 to 1 over the episode duration.
+
+- **Sparse head**: 1 stage ("task"), linear progress
+- **Dense head**: Not used
+- **Best for**: Simple tasks, quick experiments, or when VLM annotation is not available
+
+## Set Up Your Environment
+
+1. Install LeRobot by following our [Installation Guide](./installation).
+2. Install SARM dependencies by running:
+
+```bash
+pip install -e ".[sarm]"
+```
+
+Workflow:
+
+```
+1. Train SARM → 2. Visualize predictions → 3. (Optional) Train policy with RA-BC
+```
+
+</hfoption>
+<hfoption id="dense_only">
+
+**Only dense (fine-grained) annotations from a VLM.** The sparse head automatically uses a single `"task"` stage covering the full episode, while the dense head learns detailed subtask progression.
+
+- **Sparse head**: 1 stage ("task"), linear progress (auto-generated)
+- **Dense head**: Multiple fine-grained stages from VLM annotations
+- **Best for**: When you want detailed subtask tracking but don't need to define high-level stages
+
+Workflow:
+
+```
+1. Annotate (dense) → 2. Verify → 3. Train SARM → 4. Visualize → 5. (Optional) Train policy with RA-BC
+```
+
+</hfoption>
+<hfoption id="dual">
+
+**Both sparse and dense annotations from VLM.** Full dual-head mode as described in the SARM paper, with both high-level (sparse) and fine-grained (dense) stage predictions.
+
+- **Sparse head**: High-level stages from VLM annotations
+- **Dense head**: Fine-grained stages from VLM annotations
+- **Best for**: Complex multi-stage tasks where both granularities are useful
+
+Workflow:
+
+```
+1. Annotate (sparse+dense) → 2. Verify → 3. Train SARM → 4. Visualize → 5. (Optional) Train policy with RA-BC
+```
+
+</hfoption>
+</hfoptions>
+
+---
+
+## Step 1: Subtask Annotation
+
+<hfoptions id="annotation_mode">
+<hfoption id="single_stage">
+
+**No annotation required!** Skip this step entirely. The model will use the episode's task description and compute linear progress automatically.
+
+</hfoption>
+<hfoption id="dense_only">
+
+Generate **dense (fine-grained) annotations only** using a VLM. The sparse stage will be auto-generated.
+
+```bash
+python src/lerobot/data_processing/sarm_annotations/subtask_annotation.py \
+  --repo-id your-username/your-dataset \
+  --dense-only \
+  --dense-subtasks "Bring robot arms up from starting position,Grab near side and do 1st fold,Grab side and do 2nd fold,Grab side and do 3rd fold to finish folding" \
+  --video-key observation.images.base \
+  --num-workers 4 \
+  --push-to-hub
+```
+
+**What gets saved:**
+
+- `meta/temporal_proportions_sparse.json` - Auto-generated sparse proportions (`{"task": 1.0}`)
+- `meta/temporal_proportions_dense.json` - Dense temporal proportions
+- Per-episode columns in `episodes/*.parquet`:
+  - `dense_subtask_names`, `dense_subtask_start_frames`, `dense_subtask_end_frames`
+  - (also time-based columns: `dense_subtask_start_times`, `dense_subtask_end_times`)
+
+</hfoption>
+<hfoption id="dual">
+
+Generate **both sparse (high-level) and dense (fine-grained) annotations** using a VLM.
+
+```bash
+python src/lerobot/data_processing/sarm_annotations/subtask_annotation.py \
+  --repo-id your-username/your-dataset \
+  --sparse-subtasks "Bring arms up from starting position,Fold the towel (3 folds in total)" \
+  --dense-subtasks "Bring robot arms up from starting position,Grab near side and do 1st fold,Grab side and do 2nd fold,Grab side and do 3rd fold to finish folding" \
+  --video-key observation.images.base \
+  --num-workers 4 \
+  --push-to-hub
+```
+
+**What gets saved:**
+
+- `meta/temporal_proportions_sparse.json` - Sparse temporal proportions
+- `meta/temporal_proportions_dense.json` - Dense temporal proportions
+- Per-episode columns in `episodes/*.parquet`:
+  - `sparse_subtask_names`, `sparse_subtask_start_frames`, `sparse_subtask_end_frames`
+  - `dense_subtask_names`, `dense_subtask_start_frames`, `dense_subtask_end_frames`
+  - (also time-based columns: `*_subtask_start_times`, `*_subtask_end_times`)
+
+</hfoption>
+</hfoptions>
+
+### Annotation Arguments
+
+| Argument               | Description                                                                     |
+| ---------------------- | ------------------------------------------------------------------------------- |
+| `--repo-id`            | HuggingFace dataset repository ID                                               |
+| `--sparse-subtasks`    | Comma-separated list of high-level subtask names                                |
+| `--dense-subtasks`     | Comma-separated list of fine-grained subtask names                              |
+| `--dense-only`         | Generate only dense annotations (auto-creates sparse "task" stage)              |
+| `--video-key`          | Camera/video key to use (e.g., `observation.images.top`)                        |
+| `--num-workers`        | Number of parallel GPU workers (default: 1)                                     |
+| `--episodes`           | Specific episode indices to annotate (default: all)                             |
+| `--skip-existing`      | Skip episodes that already have annotations                                     |
+| `--model`              | VLM model (default: `Qwen/Qwen3-VL-30B-A3B-Instruct`)                           |
+| `--num-visualizations` | Number of episodes to visualize after annotation (default: 5, set to 0 to skip) |
+
+> **Note**: After annotation completes, 5 episodes are automatically visualized by default. Use `--num-visualizations 0` to skip this step.
+
+---
+
+## Step 2: Verify Annotations
+
+<hfoptions id="verify_mode">
+<hfoption id="single_stage">
+
+**No verification needed!** Skip this step.
+
+</hfoption>
+<hfoption id="dense_only">
+
+Visualize annotations using the `--visualize-only` flag:
+
+```bash
+python src/lerobot/data_processing/sarm_annotations/subtask_annotation.py \
+  --repo-id your-username/your-dataset \
+  --visualize-only \
+  --visualize-type dense \
+  --num-visualizations 5 \
+  --video-key observation.images.base \
+  --output-dir ./subtask_viz
+```
+
+</hfoption>
+<hfoption id="dual">
+
+Visualize annotations using the `--visualize-only` flag:
+
+```bash
+python src/lerobot/data_processing/sarm_annotations/subtask_annotation.py \
+  --repo-id your-username/your-dataset \
+  --visualize-only \
+  --visualize-type both \
+  --num-visualizations 5 \
+  --video-key observation.images.base \
+  --output-dir ./subtask_viz
+```
+
+</hfoption>
+</hfoptions>
+
+This generates visualizations showing video frames with subtask boundaries overlaid and timeline of subtasks.
+
+### Visualization Arguments
+
+| Argument               | Description                                                    |
+| ---------------------- | -------------------------------------------------------------- |
+| `--visualize-only`     | Only visualize existing annotations (no generation)            |
+| `--num-visualizations` | Number of episodes to visualize (default: 5)                   |
+| `--visualize-type`     | Type of annotations to visualize: `sparse`, `dense`, or `both` |
+
+**Tip**: If annotations are inaccurate, adjust your subtask descriptions to be more specific and re-run.
+
+---
+
+## Step 3: Train SARM
+
+<hfoptions id="train_mode">
+<hfoption id="single_stage">
+
+Train with **no annotations** - uses linear progress from 0 to 1:
+
+```bash
+python src/lerobot/scripts/lerobot_train.py \
+  --dataset.repo_id=your-username/your-dataset \
+  --policy.type=sarm \
+  --policy.annotation_mode=single_stage \
+  --policy.image_key=observation.images.base \
+  --output_dir=outputs/train/sarm_single \
+  --batch_size=32 \
+  --steps=5000 \
+  --wandb.enable=true \
+  --wandb.project=sarm \
+  --policy.repo_id=your-username/your-model-name
+```
+
+</hfoption>
+<hfoption id="dense_only">
+
+Train with **dense annotations only** (sparse auto-generated):
+
+```bash
+python src/lerobot/scripts/lerobot_train.py \
+  --dataset.repo_id=your-username/your-dataset \
+  --policy.type=sarm \
+  --policy.annotation_mode=dense_only \
+  --policy.image_key=observation.images.base \
+  --output_dir=outputs/train/sarm_dense \
+  --batch_size=32 \
+  --steps=5000 \
+  --wandb.enable=true \
+  --wandb.project=sarm \
+  --policy.repo_id=your-username/your-model-name
+```
+
+</hfoption>
+<hfoption id="dual">
+
+Train with **both sparse and dense annotations**:
+
+```bash
+python src/lerobot/scripts/lerobot_train.py \
+  --dataset.repo_id=your-username/your-dataset \
+  --policy.type=sarm \
+  --policy.annotation_mode=dual \
+  --policy.image_key=observation.images.base \
+  --output_dir=outputs/train/sarm_dual \
+  --batch_size=32 \
+  --steps=5000 \
+  --wandb.enable=true \
+  --wandb.project=sarm \
+  --policy.repo_id=your-username/your-model-name
+```
+
+</hfoption>
+</hfoptions>
+
+### Multi-GPU Training
+
+Add `accelerate launch --multi_gpu --num_processes=4` to use multiple GPUs for training.
+
+### Training Arguments
+
+| Argument                   | Description                                                       | Default                  |
+| -------------------------- | ----------------------------------------------------------------- | ------------------------ |
+| `--policy.annotation_mode` | `single_stage`, `dense_only`, or `dual`                           | `single_stage`           |
+| `--policy.image_key`       | Camera key for images                                             | `observation.images.top` |
+| `--policy.state_key`       | Key for joint states                                              | `observation.state`      |
+| `--policy.n_obs_steps`     | Observation history steps (total obs frames = `n_obs_steps + 1`)  | `8`                      |
+| `--policy.frame_gap`       | Gap (in frames) between sampled observations (at 30 fps: 30 ≈ 1s) | `30`                     |
+
+---
+
+## Step 4: Visualize Predictions
+
+Use `compute_rabc_weights.py` with `--visualize-only` to visualize model predictions (and, if available, annotation-derived targets) without writing a parquet file.
+
+<hfoptions id="viz_mode">
+<hfoption id="single_stage">
+
+```bash
+python src/lerobot/policies/sarm/compute_rabc_weights.py \
+  --dataset-repo-id your-username/your-dataset \
+  --reward-model-path your-username/sarm-model \
+  --visualize-only \
+  --num-visualizations 5 \
+  --head-mode sparse \
+  --output-dir ./sarm_viz
+```
+
+</hfoption>
+<hfoption id="dense_only">
+
+```bash
+python src/lerobot/policies/sarm/compute_rabc_weights.py \
+  --dataset-repo-id your-username/your-dataset \
+  --reward-model-path your-username/sarm-model \
+  --visualize-only \
+  --num-visualizations 5 \
+  --head-mode dense \
+  --output-dir ./sarm_viz
+```
+
+</hfoption>
+<hfoption id="dual">
+
+```bash
+python src/lerobot/policies/sarm/compute_rabc_weights.py \
+  --dataset-repo-id your-username/your-dataset \
+  --reward-model-path your-username/sarm-model \
+  --visualize-only \
+  --num-visualizations 5 \
+  --head-mode both \
+  --output-dir ./sarm_viz
+```
+
+</hfoption>
+</hfoptions>
+
+The visualization shows:
+
+- **Progress plot**: Predicted progress (and optional annotation-derived “GT” when available and `--stride 1`)
+- **Stage probabilities**: Stacked area plot of predicted stage probabilities
+- **Sample frames**: Key frames from the episode with progress/stage labels
+
+### Visualization Arguments
+
+| Argument               | Description                                               |
+| ---------------------- | --------------------------------------------------------- |
+| `--visualize-only`     | Only visualize predictions (no RABC computation)          |
+| `--num-visualizations` | Number of episodes to visualize (default: 5)              |
+| `--head-mode`          | SARM head to use: `sparse`, `dense`, or `both`            |
+| `--stride`             | Compute every N frames, interpolate the rest (default: 1) |
+
+---
+
+## Step 5 (Optional): Train Policy with RA-BC
+
+Reward-Aligned Behavior Cloning (RA-BC) uses the trained SARM model to weight training samples based on predicted progress improvement. This requires two steps:
+
+1. **Precompute progress values** for all frames using the trained SARM model
+2. **Train policy** with RA-BC weighting using the precomputed values
+
+### How RA-BC Works
+
+For each training sample, RA-BC computes the progress delta:
+
+```
+r_i = φ(o_{t+Δ}) - φ(o_t)
+```
+
+Where `φ` is the SARM progress prediction and `Δ` is the policy's `chunk_size`. Samples with positive progress (good demonstrations) get higher weights, while samples with negative or zero progress get down-weighted.
+
+The weighting follows **Equations 8-9** from the paper:
+
+- **Soft weight**: `w̃_i = clip((r_i − (μ − 2σ)) / (4σ + ε), 0, 1)`
+- **Final weight**: `w_i = 𝟙{r_i > κ} + 𝟙{0 ≤ r_i ≤ κ} × w̃_i`
+
+### Step 5a: Compute SARM Progress Values
+
+First, run the SARM model on all frames in your dataset to compute progress values:
+
+```bash
+python src/lerobot/policies/sarm/compute_rabc_weights.py \
+  --dataset-repo-id your-username/your-dataset \
+  --reward-model-path your-username/sarm-model \
+  --head-mode sparse \
+  --num-visualizations 5 \
+  --push-to-hub
+```
+
+This script:
+
+- Processes all frames and computes progress values
+- Saves progress values to a parquet file next to the dataset on disk (defaults to `<dataset_root>/sarm_progress.parquet`)
+- Generates visualizations of the first N episodes (default: 5)
+
+**Arguments:**
+
+| Argument               | Description                                                    | Default    |
+| ---------------------- | -------------------------------------------------------------- | ---------- |
+| `--reward-model-path`  | Path to trained SARM model                                     | (required) |
+| `--head-mode`          | SARM head to use: `sparse`, `dense`, or `both`                 | `sparse`   |
+| `--device`             | Device for inference                                           | `cuda`     |
+| `--visualize-only`     | Only visualize predictions (no RA-BC computation)              | `false`    |
+| `--num-visualizations` | Number of episodes to visualize (default: 5, set to 0 to skip) | `5`        |
+
+**Output format** (`sarm_progress.parquet`):
+
+| Column            | Description                                    |
+| ----------------- | ---------------------------------------------- |
+| `index`           | Global frame index in dataset                  |
+| `episode_index`   | Episode number                                 |
+| `frame_index`     | Local frame index within episode               |
+| `progress_sparse` | Sparse head progress value [0, 1]              |
+| `progress_dense`  | Dense head progress value [0, 1] (if computed) |
+
+### Step 5b: Train Policy with RA-BC
+
+Once you have the progress file, train your policy with RA-BC weighting. The progress file is auto-detected from the dataset path (`sarm_progress.parquet`). Currently PI0, PI0.5 and SmolVLA are supported with RA-BC:
+
+```bash
+python src/lerobot/scripts/lerobot_train.py \
+  --dataset.repo_id=your-username/your-dataset \
+  --policy.type=pi0 \
+  --use_rabc=true \
+  --rabc_head_mode=sparse \
+  --rabc_kappa=0.01 \
+  --output_dir=outputs/train/policy_rabc \
+  --batch_size=32 \
+  --steps=40000
+```
+
+The training script automatically:
+
+- Loads the precomputed progress values from the parquet file
+- Uses the policy's `chunk_size` to compute progress deltas (Δ)
+- Computes sample weights based on progress improvement
+- Applies weighted loss during training
+
+**RA-BC Arguments:**
+
+| Argument               | Description                                                | Default                            |
+| ---------------------- | ---------------------------------------------------------- | ---------------------------------- |
+| `--use_rabc`           | Enable RA-BC sample weighting                              | `false`                            |
+| `--rabc_progress_path` | Path to progress parquet file (auto-detected from dataset) | `sarm_progress.parquet` in dataset |
+| `--rabc_head_mode`     | Which SARM head's progress to use: `sparse` or `dense`     | `sparse`                           |
+| `--rabc_kappa`         | Threshold κ for high-quality samples                       | `0.01`                             |
+
+### Tuning RA-BC Kappa
+
+The `kappa` parameter is the threshold that determines which samples get full weight (w=1). Understanding how to tune it is critical for RA-BC to work effectively.
+
+**How the weighting works:**
+
+| Condition           | Weight                  |
+| ------------------- | ----------------------- |
+| `delta > kappa`     | 1.0 (hard threshold)    |
+| `0 ≤ delta ≤ kappa` | Soft weight from Eq. 8  |
+| `delta < 0`         | 0.0 (negative progress) |
+
+**Diagnosing kappa issues:**
+
+Monitor these WandB metrics during training:
+
+| Metric             | Healthy Range | Problem Indicator         |
+| ------------------ | ------------- | ------------------------- |
+| `rabc_mean_weight` | 0.3 - 0.8     | ≈ 1.0 means kappa too low |
+| `rabc_delta_mean`  | > 0           | Should be positive        |
+| `rabc_delta_std`   | > 0           | Variance in data quality  |
+
+**If `rabc_mean_weight ≈ 1.0`:** Your kappa is too low. Most samples have `delta > kappa` and bypass the soft-weighting entirely. RA-BC becomes equivalent to vanilla BC.
+
+**Setting kappa based on your data:**
+
+The default `kappa=0.01` was tuned for the paper's T-shirt folding task (~90s episodes at 30fps). For your dataset, check the logged `rabc_delta_mean` and `rabc_delta_std`:
+
+```
+# If delta_mean ≈ 0.03 and delta_std ≈ 0.02:
+# Most deltas fall in range [0.01, 0.05]
+
+# Option 1: Set kappa = delta_mean (medium selectivity)
+--rabc_kappa=0.03
+
+# Option 2: Set kappa = delta_mean + delta_std (high selectivity)
+--rabc_kappa=0.05
+
+# Option 3: Set kappa = delta_mean + 2*delta_std (very selective)
+--rabc_kappa=0.07
+```
+
+**When RA-BC may not help:**
+
+If your dataset is already high quality (consistent progress across all demonstrations), RA-BC won't provide much benefit since there's nothing to filter.
+
+### Multi-GPU Training with RA-BC
+
+```bash
+accelerate launch \
+  --multi_gpu \
+  --num_processes=4 \
+  src/lerobot/scripts/lerobot_train.py \
+  --dataset.repo_id=your-username/your-dataset \
+  --policy.type=pi0 \
+  --use_rabc=true \
+  --rabc_kappa=0.01 \
+  --output_dir=outputs/train/policy_rabc \
+  --batch_size=32 \
+  --steps=40000
+```
+
+---
+
+## Tips & Best Practices
+
+### Choosing a Mode
+
+- **Start with `single_stage`** for quick experiments - no annotation overhead
+- Use **`dense_only`** when you want detailed progress tracking but tasks don't have clear high-level stages
+- Use **`dual`** for complex tasks where both coarse and fine-grained progress is meaningful
+
+### Annotation Quality
+
+1. **Be specific with subtask names**: Instead of "fold", use "grab near side and fold toward center"
+2. **Verify with visualization**: Always check a few episodes before training
+3. **Consistent naming**: Use the same subtask names across all episodes
+
+### RA-BC
+
+1. **Train SARM first**: RA-BC quality depends entirely on SARM quality
+2. **Monitor `rabc_mean_weight`**: If it's ≈ 1.0, increase kappa (see [Tuning RA-BC Kappa](#tuning-ra-bc-kappa))
+
+---
+
+## Citation
+
+```bibtex
+@article{chen2025sarm,
+  title={SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulation},
+  author={Chen, Qianzhong and Yu, Justin and Schwager, Mac and Abbeel, Pieter and Shentu, Yide and Wu, Philipp},
+  journal={arXiv preprint arXiv:2509.25358},
+  year={2025}
+}
+```
@@ -4,11 +4,12 @@ This guide covers the complete setup process for the Unitree G1 humanoid, from i

 ## About the Unitree G1

-We offer support for both 29 and 23 DOF G1. In this first PR we introduce:
+We offer support for both 29 and 23 DOF G1. We introduce:

 - **`unitree g1` robot class, handling low level communication with the humanoid**
 - **ZMQ socket bridge** for remote communication over WiFi, allowing one to deploy policies remotely instead of over ethernet or directly on the Orin
 - **GR00T locomotion policy** for bipedal walking and balance
+- **MuJoCo simulation mode** for testing policies without the physical robot

 ---

@@ -191,6 +192,10 @@ Press `Ctrl+C` to stop the policy.

 ---

+## Extra: Running in Simulation Mode (MuJoCo)
+
+You can now test and develop policies without a physical robot using MuJoCo. to do so set `is_simulation=True` in config.
+
 ## Additional Resources

 - [Unitree SDK Documentation](https://github.com/unitreerobotics/unitree_sdk2_python)
@@ -11,13 +11,14 @@ LeRobot provides several utilities for manipulating datasets:
 3. **Merge Datasets** - Combine multiple datasets into one. The datasets must have identical features, and episodes are concatenated in the order specified in `repo_ids`
 4. **Add Features** - Add new features to a dataset
 5. **Remove Features** - Remove features from a dataset
+6. **Convert to Video** - Convert image-based datasets to video format for efficient storage

 The core implementation is in `lerobot.datasets.dataset_tools`.
 An example script detailing how to use the tools API is available in `examples/dataset/use_dataset_tools.py`.

 ## Command-Line Tool: lerobot-edit-dataset

-`lerobot-edit-dataset` is a command-line script for editing datasets. It can be used to delete episodes, split datasets, merge datasets, add features, and remove features.
+`lerobot-edit-dataset` is a command-line script for editing datasets. It can be used to delete episodes, split datasets, merge datasets, add features, remove features, and convert image datasets to video format.

 Run `lerobot-edit-dataset --help` for more information on the configuration of each operation.

@@ -86,9 +87,71 @@ lerobot-edit-dataset \
    --operation.feature_names "['observation.images.top']"
 ```

+#### Convert to Video
+
+Convert an image-based dataset to video format, creating a new LeRobotDataset where images are stored as videos. This is useful for reducing storage requirements and improving data loading performance. The new dataset will have the exact same structure as the original, but with images encoded as MP4 videos in the proper LeRobot format.
+
+```bash
+# Local-only: Save to a custom output directory (no hub push)
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --operation.type convert_to_video \
+    --operation.output_dir /path/to/output/pusht_video
+
+# Save with new repo_id (local storage)
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --new_repo_id lerobot/pusht_video \
+    --operation.type convert_to_video
+
+# Convert and push to Hugging Face Hub
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --new_repo_id lerobot/pusht_video \
+    --operation.type convert_to_video \
+    --push_to_hub true
+
+# Convert with custom video codec and quality settings
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --operation.type convert_to_video \
+    --operation.output_dir outputs/pusht_video \
+    --operation.vcodec libsvtav1 \
+    --operation.pix_fmt yuv420p \
+    --operation.g 2 \
+    --operation.crf 30
+
+# Convert only specific episodes
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --operation.type convert_to_video \
+    --operation.output_dir outputs/pusht_video \
+    --operation.episode_indices "[0, 1, 2, 5, 10]"
+
+# Convert with multiple workers for parallel processing
+lerobot-edit-dataset \
+    --repo_id lerobot/pusht_image \
+    --operation.type convert_to_video \
+    --operation.output_dir outputs/pusht_video \
+    --operation.num_workers 8
+```
+
+**Parameters:**
+
+- `output_dir`: Custom output directory (optional - by default uses `new_repo_id` or `{repo_id}_video`)
+- `vcodec`: Video codec to use - options: `h264`, `hevc`, `libsvtav1` (default: `libsvtav1`)
+- `pix_fmt`: Pixel format - options: `yuv420p`, `yuv444p` (default: `yuv420p`)
+- `g`: Group of pictures (GOP) size - lower values give better quality but larger files (default: 2)
+- `crf`: Constant rate factor - lower values give better quality but larger files, 0 is lossless (default: 30)
+- `fast_decode`: Fast decode tuning option (default: 0)
+- `episode_indices`: List of specific episodes to convert (default: all episodes)
+- `num_workers`: Number of parallel workers for processing (default: 4)
+
+**Note:** The resulting dataset will be a proper LeRobotDataset with all cameras encoded as videos in the `videos/` directory, with parquet files containing only metadata (no raw image data). All episodes, stats, and tasks are preserved.
+
 ### Push to Hub

-Add the `--push_to_hub` flag to any command to automatically upload the resulting dataset to the Hugging Face Hub:
+Add the `--push_to_hub true` flag to any command to automatically upload the resulting dataset to the Hugging Face Hub:

 ```bash
 lerobot-edit-dataset \
@@ -96,7 +159,45 @@ lerobot-edit-dataset \
    --new_repo_id lerobot/pusht_after_deletion \
    --operation.type delete_episodes \
    --operation.episode_indices "[0, 2, 5]" \
-    --push_to_hub
+    --push_to_hub true
 ```

 There is also a tool for adding features to a dataset that is not yet covered in `lerobot-edit-dataset`.
+
+# Dataset Visualization
+
+## Online Visualization
+
+When you record a dataset using `lerobot`, it automatically uploads to the Hugging Face Hub unless you specify otherwise. To view the dataset online, use our **LeRobot Dataset Visualizer**, available at:
+https://huggingface.co/spaces/lerobot/visualize_dataset
+
+## Local Visualization
+
+You can also visualize episodes from a dataset locally using our command-line tool.
+
+**From the Hugging Face Hub:**
+
+```bash
+lerobot-dataset-viz \
+    --repo-id lerobot/pusht \
+    --episode-index 0
+```
+
+**From a local folder:**
+Add the `--root` option and set `--mode local`. For example, to search in `./my_local_data_dir/lerobot/pusht`:
+
+```bash
+lerobot-dataset-viz \
+    --repo-id lerobot/pusht \
+    --root ./my_local_data_dir \
+    --mode local \
+    --episode-index 0
+```
+
+Once executed, the tool opens `rerun.io` and displays the camera streams, robot states, and actions for the selected episode.
+
+For advanced usage—including visualizing datasets stored on a remote server—run:
+
+```bash
+lerobot-dataset-viz --help
+```
@@ -24,7 +24,7 @@ Built from pure Transformer encoders, X-VLA scales naturally with model size and
  <img
    src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot/xvla-architecture2.png"
    alt="XVLA Architecture 2"
-    style="width: 32%; max-width: 450px; height: auto;"
+    style="width: 60%; height: auto;"
  />
 </p>

@@ -120,7 +120,7 @@ Adapted for Google Robot platforms.

 ### Recommended Training Configuration

-When fine-tuning X-VLA for a new embodiment or task, we recommend the following freezing strategy:
+When fine-tuning X-VLA for a new embodiment or task, we recommend not freezing the VLM, and also setting the `policy.dtype=bfloat16` to not hit OOM errors.

 ```bash
 lerobot-train \
@@ -129,25 +129,26 @@ lerobot-train \
  --job_name=xvla_training \
  --policy.path="lerobot/xvla-base" \
  --policy.repo_id="HF_USER/xvla-your-robot" \
-  --steps=3000 \
+  --policy.dtype=bfloat16 \
+  --policy.action_mode=auto \
+  --steps=20000 \
  --policy.device=cuda \
-  --policy.freeze_vision_encoder=True \
-  --policy.freeze_language_encoder=True \
-  --policy.train_policy_transformer=True \
-  --policy.train_soft_prompts=True \
-  --policy.action_mode=YOUR_ACTION_MODE
+  --policy.freeze_vision_encoder=false \
+  --policy.freeze_language_encoder=false \
+  --policy.train_policy_transformer=true \
+  --policy.train_soft_prompts=true \
 ```

 ### Training Parameters Explained

-| Parameter                  | Default | Description                              |
-| -------------------------- | ------- | ---------------------------------------- |
-| `freeze_vision_encoder`    | `True`  | Freeze the VLM vision encoder weights    |
-| `freeze_language_encoder`  | `True`  | Freeze the VLM language encoder weights  |
-| `train_policy_transformer` | `True`  | Allow policy transformer layers to train |
-| `train_soft_prompts`       | `True`  | Allow soft prompts to train              |
+| Parameter                  | Default | Description                                    |
+| -------------------------- | ------- | ---------------------------------------------- |
+| `freeze_vision_encoder`    | `false` | Do not freeze the VLM vision encoder weights   |
+| `freeze_language_encoder`  | `false` | Do not freeze the VLM language encoder weights |
+| `train_policy_transformer` | `true`  | Allow policy transformer layers to train       |
+| `train_soft_prompts`       | `true`  | Allow soft prompts to train                    |

-**💡 Best Practice**: For Phase II adaptation to new embodiments, freeze the VLM encoders and only train the policy transformer and soft prompts. This provides excellent sample efficiency with minimal compute.
+**💡 Best Practice**: For Phase II adaptation to new embodiments, do not freeze the VLM encoders and also train the policy transformer and soft prompts.

 ### Example: Training on Bimanual Robot

@@ -157,14 +158,15 @@ lerobot-train \
  --output_dir=./outputs/xvla_bimanual \
  --job_name=xvla_so101_training \
  --policy.path="lerobot/xvla-base" \
+  --policy.dtype=bfloat16 \
  --policy.repo_id="YOUR_USERNAME/xvla-biso101" \
  --steps=3000 \
  --policy.device=cuda \
  --policy.action_mode=so101_bimanual \
-  --policy.freeze_vision_encoder=True \
-  --policy.freeze_language_encoder=True \
-  --policy.train_policy_transformer=True \
-  --policy.train_soft_prompts=True
+  --policy.freeze_vision_encoder=false \
+  --policy.freeze_language_encoder=false \
+  --policy.train_policy_transformer=true \
+  --policy.train_soft_prompts=true
 ```

 💡 **Best Performance:** If you have sufficient computational resources and want to achieve best X-VLA finetuning performance, you should follow the official finetuning strategy:
@@ -172,71 +174,7 @@ lerobot-train \
 **🔥 Full-finetune all components with a custom learning-rate scheme**

 To ensure stable optimization, the Vision-Language Model (VLM) must be trained with only 1/10 of the base learning rate, while all other components use the full LR.
-This LR ratio is crucial for achieving strong and stable finetuning performance.
-To enable this behavior, you must:
-
-1. Implement a custom optimizer and register it in your training config
-
-```
-from dataclasses import dataclass, asdict
-from lerobot.optim.optimizers import OptimizerConfig
-import torch
-
-@OptimizerConfig.register_subclass("xvla-adamw")
-@dataclass
-class XVLAAdamW(OptimizerConfig):
-    lr: float = 1e-4
-    betas: tuple[float, float] = (0.9, 0.99)
-    eps: float = 1e-8
-    weight_decay: float = 0.0
-    grad_clip_norm: float = 10.0
-
-    def build(self, params: dict) -> torch.optim.Optimizer:
-        """
-        Expect `named_parameters()` as input.
-        Apply lr = lr / 10 for all VLM-related parameters.
-        """
-        assert isinstance(params, dict), \
-            "Custom LR optimizer requires `named_parameters()` as inputs."
-        kwargs = asdict(self)
-        kwargs.pop("grad_clip_norm")
-        vlm_group, other_group = [], []
-        for name, p in params.items():
-            if not p.requires_grad:
-                continue
-            if "vlm" in name.lower():
-                vlm_group.append(p)
-            else:
-                other_group.append(p)
-
-        param_groups = [
-            {"params": vlm_group, "lr": self.lr * 0.1, "weight_decay": self.weight_decay * 0.1},
-            {"params": other_group, "lr": self.lr, "weight_decay": self.weight_decay},
-        ]
-
-        return torch.optim.AdamW(param_groups, **kwargs)
-```
-
-2. Modify X-VLA’s get_optim_params to return named parameters
-
-Replace:
-
-```
-def get_optim_params(self) -> dict:
-    """Return only trainable parameters for optimization."""
-    return filter(lambda p: p.requires_grad, self.parameters())
-```
-
-with:
-
-```
-def get_optim_params(self):
-    """Return trainable named parameters."""
-    return filter(lambda kv: kv[1].requires_grad, self.named_parameters())
-```
-
-This ensures the optimizer receives a dict of named parameters, allowing it to correctly detect VLM modules and apply the 1/10 LR rule.
-
+This LR ratio is crucial for achieving strong and stable finetuning performance. This is already done for you by default.
 ❕Note

 Completely matching the official reported performance may require an additional warm-up LR schedule for soft-prompts, which can bring minor improvements.
@@ -326,6 +264,26 @@ domain_id = 3

 The domain_id is automatically added to observations by the `XVLAAddDomainIdProcessorStep` in the preprocessing pipeline.

+The `lerobot/xvla-base` model has been trained on the following domain IDs. It is recommended to choose one that most resembles your robot/configuration:
+
+#### Fine-tuning Datasets
+
+| Dataset Name     | Domain ID |
+| ---------------- | --------- |
+| Bridge           | 0         |
+| RT1              | 1         |
+| Calvin           | 2         |
+| libero           | 3         |
+| widowx-air       | 4         |
+| AIR-AGILEX-HQ    | 5         |
+| robotwin2_abs_ee | 6         |
+| robotwin2_clean  | 6         |
+| robocasa-human   | 7         |
+| VLABench         | 8         |
+| AGIBOT-challenge | 9         |
+| AIR-AGILEX       | 10        |
+| AIRBOT           | 18        |
+
 ### 3. Processor Steps

 X-VLA requires specific preprocessing and postprocessing steps for proper operation.
@@ -1,243 +0,0 @@
-# Synthetic Data Generation Script - Summary
-
-## ✅ What Was Created
-
-### Main Script: `annotate_pgen.py` (717 lines)
-A production-ready script implementing the Hi-Robot synthetic data generation pipeline.
-
-**Key Features:**
- ✅ Loads LeRobot datasets with skill annotations
- ✅ Generates synthetic user prompts and robot utterances using Qwen VLM
- ✅ **Temporal sampling** - generates dialogue every N seconds (default: 1s)
- ✅ Adds `task_index_high_level` feature to dataset parquets
- ✅ Saves high-level tasks to `meta/tasks_high_level.parquet`
- ✅ Exports debug JSONL for quality analysis
- ✅ Supports both Qwen2-VL and Qwen3-VL models
- ✅ Multi-view camera support
- ✅ Episode-aware processing with automatic first-frame sampling
- ✅ Modular architecture for easy extension
-
-### Supporting Files Created
-
-1. **`run_pgen.sh`** - Convenience script with sensible defaults
-2. **`README_PGEN.md`** - Comprehensive documentation with examples
-3. **`example_pgen_usage.md`** - Practical examples and performance estimates
-4. **`SAMPLING_DIAGRAM.md`** - Visual explanation of temporal sampling strategy
-5. **`PGEN_SUMMARY.md`** - This file
-
-## 🚀 Key Innovation: Temporal Sampling
-
-The script processes **ALL episodes** in the dataset efficiently via `--sample-interval`:
-
-```bash
-# Instead of calling VLM for every frame (expensive):
-# 15,000 frames × VLM call = ~5 hours
-
-# Generate dialogue every 1 second (efficient):
-python annotate_pgen.py --repo-id dataset --model qwen --sample-interval 1.0
-# 15,000 frames processed, only ~500 VLM calls (30x speedup!)
-```
-
-**How it works:**
- Process ALL frames in ALL episodes (complete coverage)
- Generate dialogue at sampled timepoints (e.g., every 1 second)
- Propagate task indices to intermediate frames
- Always sample first frame of each episode
- All frames get labeled, but VLM is only called for samples
- No dummy values or skipped episodes
-
-**Benefits:**
- 30-100x speedup depending on interval
- Maintains temporal coherence
- Reduces cost without losing quality
- Configurable based on skill duration
-
-## 📊 Efficiency Comparison
-
-For a typical 15,000 frame dataset at 30 fps:
-
-| Method | VLM Calls | Time | Cost |
-|--------|-----------|------|------|
-| Every frame | 15,000 | ~5 hours | $$$$ |
-| Every 0.5s | 1,000 | ~20 min | $$$ |
-| **Every 1s** (default) | **500** | **~10 min** | **$$** |
-| Every 2s | 250 | ~5 min | $ |
-
-## 🎯 Usage
-
-### Quick Test (5s sampling for fast iteration)
-```bash
-python examples/dataset/annotate_pgen.py \
-    --data-dir /fsx/jade_choghari/.cache/huggingface/lerobot/lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen2-VL-7B-Instruct \
-    --sample-interval 5.0 \
-    --output-dir ./outputs/test_quick
-```
-
-### Production Run (Recommended Settings)
-```bash
-python examples/dataset/annotate_pgen.py \
-    --data-dir /fsx/jade_choghari/.cache/huggingface/lerobot/lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen2-VL-7B-Instruct \
-    --sample-interval 1.0 \
-    --output-dir ./outputs/full_pgen
-```
-
-### High-Quality with Qwen3
-```bash
-python examples/dataset/annotate_pgen.py \
-    --data-dir /fsx/jade_choghari/.cache/huggingface/lerobot/lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen3-VL-30B-A3B-Instruct \
-    --sample-interval 0.5 \
-    --temperature 0.6 \
-    --output-dir ./outputs/high_quality
-```
-
-## 📦 Output Structure
-
-After running, you'll have:
-
-```
-dataset_root/
-├── meta/
-│   ├── tasks_high_level.parquet      # High-level tasks with prompts/utterances
-│   └── syn_annotations.jsonl         # Debug: full context for each sample
-└── data/
-    └── chunk-000/
-        └── file-000.parquet           # Updated with task_index_high_level
-```
-
-**New feature added to all parquet files:**
- `task_index_high_level` (int64): Links to tasks_high_level.parquet
-
-## 🔧 All Parameters
-
-| Parameter | Default | Description |
-|-----------|---------|-------------|
-| `--repo-id` / `--data-dir` | - | Dataset source |
-| `--model` | Qwen/Qwen2-VL-7B-Instruct | VLM model |
-| `--device` | cuda | Device to use |
-| `--dtype` | bfloat16 | Model precision |
-| `--temperature` | 0.7 | Sampling temperature |
-| **`--sample-interval`** | **1.0** | **Generate every N seconds (all episodes processed)** |
-| `--num-image-views-per-sample` | 1 | Number of cameras |
-| `--batch-size` | 1 | Batch size (currently unused) |
-| `--output-dir` | None | Output directory |
-| `--push-to-hub` | False | Push to HuggingFace |
-
-## 🎨 Generated Data Format
-
-Each sampled frame produces:
-
-```json
-{
-  "scenario_type": "specific_object",
-  "response_type": "confirmation",
-  "user_prompt": "Can you pick up the pink brick?",
-  "robot_utterance": "Sure, I'll grab the pink lego brick.",
-  "skill": "robot arm picks up pink lego brick",
-  "episode_id": 0,
-  "frame_index": 45,
-  "timestamp": 1.5,
-  "skill_history": ["robot arm moves towards pink lego brick"],
-  "task_description": "pink lego brick into the transparent box"
-}
-```
-
-**Scenario Types:**
- specific_object, negative_task, situated_correction, implicit_request, constraint_based
-
-**Response Types:**
- confirmation, clarification, acknowledgment, constraint_acknowledgment
-
-## 🔬 Code Architecture
-
-```python
-# Main components (modular design)
-
-class QwenPgen:
-    """VLM wrapper supporting Qwen2/3"""
-    def call_qwen(images, prompt) -> dict
-
-def construct_prompt(task, history, skill) -> str:
-    """Build contextual prompt with history"""
-
-def annotate_sample(pgen, images, ...) -> dict:
-    """Generate dialogue for one sample"""
-
-def generate_synthetic_data(dataset, pgen, ...) -> tuple:
-    """Process entire dataset with temporal sampling"""
-    # Core sampling logic:
-    # - Track last_sample_timestamp per episode
-    # - Sample if time_elapsed >= sample_interval
-    # - Always sample first frame of episodes
-    # - Propagate task_index to intermediate frames
-
-def main():
-    """CLI entrypoint with argparse"""
-```
-
-## ✨ Next Steps
-
-1. **Quick test with large interval:**
-   ```bash
-   # Fast iteration - samples every 5 seconds
-   python examples/dataset/annotate_pgen.py \
-       --data-dir /path/to/dataset \
-       --model Qwen/Qwen2-VL-7B-Instruct \
-       --sample-interval 5.0 \
-       --output-dir ./outputs/quick_test
-   ```
-
-2. **Verify output quality:**
-   ```bash
-   head outputs/quick_test/meta/syn_annotations.jsonl
-   ```
-
-3. **Production run:**
-   ```bash
-   # Standard 1 second sampling for production
-   bash examples/dataset/run_pgen.sh
-   ```
-
-4. **Use in training:**
-   ```python
-   from lerobot.datasets.lerobot_dataset import LeRobotDataset
-   
-   ds = LeRobotDataset(repo_id="...", root="outputs/pgen_annotations")
-   
-   # Access high-level task for each frame
-   frame = ds[100]
-   task_idx = frame["task_index_high_level"].item()
-   ```
-
-## 📚 Documentation Files
-
- **`README_PGEN.md`**: Full API reference and troubleshooting
- **`example_pgen_usage.md`**: Practical examples with performance estimates
- **`SAMPLING_DIAGRAM.md`**: Visual explanation of temporal sampling
- **`PGEN_SUMMARY.md`**: This overview document
-
-## 🎯 Success Criteria
-
-✅ Script generates synthetic dialogue using Qwen VLM  
-✅ Adds `task_index_high_level` feature to dataset  
-✅ Saves tasks to `tasks_high_level.parquet`  
-✅ Implements efficient temporal sampling (30-100x speedup)  
-✅ Handles episode boundaries correctly  
-✅ Produces diverse interaction types (scenarios + responses)  
-✅ Maintains temporal coherence within episodes  
-✅ Includes comprehensive documentation and examples  
-✅ Ready for production use on real datasets  
-
-## 💡 Key Takeaway
-
-**The script processes ALL episodes with intelligent sampling:**
- `--sample-interval` controls how often VLM is called (default: 1.0s)
- ALL frames in ALL episodes get labeled (complete coverage)
- Intermediate frames inherit from most recent sample (temporal coherence)
- Achieves 30-100x speedup while maintaining quality
- Adjust interval based on use case: 5.0s for testing, 1.0s for production, 0.5s for fine detail
-
-This makes the synthetic data generation **practical, scalable, and complete** for real-world datasets!
-
@@ -1,243 +0,0 @@
-# Synthetic Data Generation for Hierarchical Robot Policies
-
-This directory contains `annotate_pgen.py`, a script for generating synthetic user prompts and robot utterances for hierarchical policy training using Vision-Language Models (VLMs).
-
-## Overview
-
-The script implements the synthetic data generation pipeline described in the Hi-Robot paper:
-
-1. **Load** a LeRobot dataset with skill annotations (from `annotate.py`)
-2. **Generate** synthetic dialogue using Qwen VLM:
-   - User prompts (ℓ_t): Natural requests that lead to specific skills
-   - Robot utterances (u_t): Acknowledgments and clarifications
-3. **Save** results as a new dataset feature `task_index_high_level`
-
-## Prerequisites
-
-1. First, annotate your dataset with skills using `annotate.py`:
-
-```bash
-python examples/dataset/annotate.py \
-    --repo-id lerobot/svla_so101_pickplace \
-    --video-key observation.images.base \
-    --model Qwen/Qwen2-VL-7B-Instruct
-```
-
-This creates `meta/skills.json` with skill segmentation for each episode.
-
-## Usage
-
-### Basic Usage
-
-```bash
-python examples/dataset/annotate_pgen.py \
-    --repo-id lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen2-VL-7B-Instruct \
-    --sample-interval 1.0 \
-    --output-dir ./outputs/pgen_dataset
-```
-
-**Note**: The script processes **all episodes** in the dataset. It generates dialogue every 1 second (`--sample-interval 1.0`) using temporal sampling. Frames between samples reuse the last generated dialogue. This makes the process efficient while ensuring complete dataset coverage.
-
-### Advanced Options
-
-```bash
-python examples/dataset/annotate_pgen.py \
-    --repo-id lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen3-VL-30B-A3B-Instruct \
-    --temperature 0.8 \
-    --sample-interval 0.5 \
-    --num-image-views-per-sample 2 \
-    --output-dir ./outputs/pgen_dataset \
-    --push-to-hub
-```
-
-This example uses a more powerful model and samples every 0.5 seconds for finer granularity.
-
-### Fast Testing (larger interval)
-
-```bash
-python examples/dataset/annotate_pgen.py \
-    --repo-id lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen2-VL-7B-Instruct \
-    --sample-interval 5.0 \
-    --output-dir ./outputs/pgen_quick_test
-```
-
-Use a larger interval (5.0 seconds) for rapid iteration during development. All episodes are still processed.
-
-### Using Local Dataset
-
-```bash
-python examples/dataset/annotate_pgen.py \
-    --data-dir /fsx/jade_choghari/.cache/huggingface/lerobot/lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen2-VL-7B-Instruct \
-    --output-dir ./outputs/pgen_dataset
-```
-
-## Output Files
-
-The script produces several outputs:
-
-1. **`meta/tasks_high_level.parquet`**: High-level tasks with user prompts and robot utterances
-   - Columns: task_index, user_prompt, robot_utterance, skill, scenario_type, response_type
-
-2. **`meta/syn_annotations.jsonl`**: Debug file with all generated dialogues
-   - One JSON object per line with full context for each frame
-
-3. **Modified dataset**: New dataset with `task_index_high_level` feature added to all parquet files
-
-## Scenario and Response Types
-
-The generator produces diverse interaction types:
-
-### Scenario Types
- **specific_object**: Direct specification of objects/actions
- **negative_task**: Instructions about what NOT to do
- **situated_correction**: Adjustments based on current state
- **implicit_request**: Implied needs without direct commands
- **constraint_based**: Specific constraints or preferences
-
-### Response Types
- **confirmation**: Simple acknowledgment ("OK, I'll do X")
- **clarification**: Seeking confirmation ("Just to confirm...")
- **acknowledgment**: Action acknowledgment ("Got it, doing X")
- **constraint_acknowledgment**: Acknowledging constraints ("Sure, I'll X while Y")
-
-## Example Generated Data
-
-```json
-{
-  "episode_id": 0,
-  "frame_index": 45,
-  "timestamp": 2.5,
-  "skill_current": "robot arm picks up pink lego brick",
-  "skill_history": ["robot arm moves towards pink lego brick"],
-  "task_description": "pink lego brick into the transparent box",
-  "scenario_type": "specific_object",
-  "response_type": "confirmation",
-  "user_prompt": "Can you grab the pink brick?",
-  "robot_utterance": "Sure, I'll pick up the pink lego brick."
-}
-```
-
-## Accessing the Data
-
-After running the script, access the synthetic data in your code:
-
-```python
-from lerobot.datasets.lerobot_dataset import LeRobotDataset
-import pandas as pd
-
-# Load modified dataset
-dataset = LeRobotDataset(repo_id="lerobot/svla_so101_pickplace_with_high_level_tasks")
-
-# Access frame with high-level task
-frame = dataset[100]
-high_level_task_idx = frame["task_index_high_level"].item()
-
-# Load high-level tasks
-tasks_df = pd.read_parquet(dataset.root / "meta" / "tasks_high_level.parquet")
-task_info = tasks_df.iloc[high_level_task_idx]
-
-print(f"User prompt: {task_info['user_prompt']}")
-print(f"Robot utterance: {task_info['robot_utterance']}")
-print(f"Skill: {task_info['skill']}")
-```
-
-## Architecture
-
-The script is modular and extensible:
-
-```python
-# Core components
-class QwenPgen:
-    """VLM wrapper for generation"""
-    def call_qwen(images, prompt) -> dict
-    
-def construct_prompt(task, history, skill) -> str
-    """Build prompt for VLM"""
-    
-def annotate_sample(pgen, images, ...) -> dict
-    """Generate dialogue for one sample"""
-    
-def generate_synthetic_data(dataset, pgen, ...) -> tuple
-    """Process entire dataset"""
-```
-
-## Parameters
-
-| Parameter | Default | Description |
-|-----------|---------|-------------|
-| `--repo-id` | - | HuggingFace dataset ID |
-| `--data-dir` | - | Local dataset path |
-| `--model` | Qwen/Qwen2-VL-7B-Instruct | VLM model name |
-| `--device` | cuda | Device (cuda/cpu) |
-| `--dtype` | bfloat16 | Model precision |
-| `--temperature` | 0.7 | Sampling temperature |
-| `--sample-interval` | 1.0 | Generate dialogue every N seconds (all episodes processed) |
-| `--num-image-views-per-sample` | 1 | Number of cameras |
-| `--output-dir` | None | Output directory |
-| `--push-to-hub` | False | Push to HuggingFace Hub |
-
-## Sampling Strategy
-
-The script uses **temporal sampling** to efficiently generate dialogue:
-
- **Default**: Generate dialogue every 1 second (`--sample-interval 1.0`)
- **Efficiency**: If a dataset runs at 30fps, this samples ~3% of frames
- **Propagation**: Frames between samples reuse the last generated task_index
- **Episode-aware**: Always samples the first frame of each episode
-
-### Example with 30 fps dataset:
-```bash
-# Sample every 1 second (every 30 frames)
--sample-interval 1.0  # ~3,000 generations for a 100 episode dataset (3 sec/episode)
-
-# Sample every 0.5 seconds (every 15 frames)
--sample-interval 0.5  # ~6,000 generations (more granular)
-
-# Sample every 2 seconds (every 60 frames)
--sample-interval 2.0  # ~1,500 generations (more efficient)
-```
-
-### Why sampling works:
- Skills typically last 1-3 seconds
- Dialogue doesn't need to change every frame
- Reduces computational cost by 30-100x
- Still provides good coverage for training
-
-## Tips
-
-1. **Quick testing**: Use larger `--sample-interval` (e.g., 5.0 or 10.0) for rapid iteration
-2. **Monitor GPU**: VLM inference is memory-intensive
-3. **Check outputs**: Review `syn_annotations.jsonl` for quality
-4. **Adjust temperature**: Higher = more diverse, lower = more consistent
-5. **Multiple views**: Use `--num-image-views-per-sample 2+` for better context
-6. **Tune sampling**: Start with 1.0s, increase for speed (testing), decrease for granularity (production)
-
-## Troubleshooting
-
-### No skills.json found
-Run `annotate.py` first to generate skill annotations.
-
-### Out of memory
- Reduce batch size to 1
- Use smaller model (Qwen2-VL-7B instead of Qwen3-VL-30B)
- Process fewer samples at a time
-
-### Poor quality generations
- Adjust temperature (try 0.6-0.9)
- Check that skills.json has good annotations
- Ensure images are loading correctly
-
-## Citation
-
-Based on the Hi-Robot paper's synthetic data generation approach:
-```
-@article{hirobot2024,
-  title={Hi-Robot: Hierarchical Robot Learning with Vision-Language Models},
-  year={2024}
-}
-```
-
@@ -1,141 +0,0 @@
-# Temporal Sampling Strategy Visualization
-
-## How `--sample-interval` Works
-
-### Example: 30 fps dataset, `--sample-interval 1.0` (1 second)
-
-```
-Timeline (seconds):  0.0      0.5      1.0      1.5      2.0      2.5      3.0
-                     │        │        │        │        │        │        │
-Frames:              0───15───30───45───60───75───90───105──120──135──150
-                     │        │        │        │        │        │        │
-                     ▼                 ▼                 ▼                 ▼
-Sampled:            YES      NO       YES      NO       YES      NO       YES
-                     │                 │                 │                 │
-Task Index:         [0]──────────────>[1]──────────────>[2]──────────────>[3]
-                     │                 │                 │                 │
-VLM Called:         ✓ Gen             ✓ Gen             ✓ Gen             ✓ Gen
-                    dialogue          dialogue          dialogue          dialogue
-                     │                 │                 │                 │
-Frames 0-29    ─────┘                 │                 │                 │
-get task 0                             │                 │                 │
-                                       │                 │                 │
-Frames 30-59  ────────────────────────┘                 │                 │
-get task 1                                               │                 │
-                                                         │                 │
-Frames 60-89  ──────────────────────────────────────────┘                 │
-get task 2                                                                 │
-                                                                           │
-Frames 90-119 ────────────────────────────────────────────────────────────┘
-get task 3
-```
-
-## Comparison: Different Sampling Intervals
-
-### `--sample-interval 2.0` (every 2 seconds)
-```
-Timeline:    0.0      1.0      2.0      3.0      4.0      5.0      6.0
-             │        │        │        │        │        │        │
-Sampled:    YES      NO       YES      NO       YES      NO       YES
-             │                 │                 │                 │
-Tasks:      [0]───────────────>[1]───────────────>[2]───────────────>[3]
-             
-VLM Calls:   4 (fewer calls, faster but less granular)
-```
-
-### `--sample-interval 1.0` (every 1 second) - **DEFAULT**
-```
-Timeline:    0.0   0.5   1.0   1.5   2.0   2.5   3.0   3.5   4.0   4.5   5.0   5.5   6.0
-             │     │     │     │     │     │     │     │     │     │     │     │     │
-Sampled:    YES   NO   YES   NO   YES   NO   YES   NO   YES   NO   YES   NO   YES
-             │           │           │           │           │           │           │
-Tasks:      [0]─────────>[1]─────────>[2]─────────>[3]─────────>[4]─────────>[5]─────>[6]
-             
-VLM Calls:   7 (balanced coverage and speed)
-```
-
-### `--sample-interval 0.5` (every 0.5 seconds)
-```
-Timeline:    0.0  0.5  1.0  1.5  2.0  2.5  3.0  3.5  4.0  4.5  5.0  5.5  6.0
-             │    │    │    │    │    │    │    │    │    │    │    │    │
-Sampled:    YES  YES  YES  YES  YES  YES  YES  YES  YES  YES  YES  YES  YES
-             │    │    │    │    │    │    │    │    │    │    │    │    │
-Tasks:      [0]─>[1]─>[2]─>[3]─>[4]─>[5]─>[6]─>[7]─>[8]─>[9]─>[10]>[11]>[12]
-             
-VLM Calls:   13 (high granularity, slower but more detailed)
-```
-
-## Episode Boundaries
-
-The script always samples the **first frame** of each episode:
-
-```
-Episode 0                          Episode 1                          Episode 2
-├─────────────────────────────────┤├─────────────────────────────────┤├──────...
-│                                 ││                                 ││
-Frame: 0    30    60    90   120  130   160   190   220  250  260   290  320
-Time:  0.0  1.0   2.0   3.0  4.0  0.0   1.0   2.0   3.0  4.0  0.0   1.0  2.0
-       │    │     │     │    │    │     │     │     │    │    │     │    │
-       ▼    ▼     ▼     ▼    ▼    ▼     ▼     ▼     ▼    ▼    ▼     ▼    ▼
-Sample:YES  YES   YES   YES  YES  YES   YES   YES   YES  YES  YES   YES  YES
-       │    │     │     │    │    │     │     │     │    │    │     │    │
-Task:  0────1─────2─────3────4    5─────6─────7─────8────9    10────11───12
-
-Note: Frames 0, 130, 260 are ALWAYS sampled (episode starts)
-      Even if they're within the sample-interval window
-```
-
-## Real-World Example: svla_so101_pickplace Dataset
-
-Typical stats:
- **Total episodes**: 50
- **Avg episode length**: 300 frames (10 seconds at 30 fps)
- **Total frames**: 15,000
-
-### Without Sampling (every frame)
-```
-Frames processed:    15,000
-VLM calls:           15,000
-Time estimate:       ~5 hours
-Unique tasks:        ~12,000 (lots of duplicates)
-```
-
-### With `--sample-interval 1.0` (every 1 second)
-```
-Frames processed:    15,000 ✓
-VLM calls:           500
-Time estimate:       ~10 minutes
-Unique tasks:        ~450 (meaningful variety)
-Efficiency gain:     30x faster
-```
-
-### With `--sample-interval 2.0` (every 2 seconds)
-```
-Frames processed:    15,000 ✓
-VLM calls:           250
-Time estimate:       ~5 minutes
-Unique tasks:        ~220
-Efficiency gain:     60x faster
-```
-
-## Key Points
-
-1. **All frames get labeled**: Every frame gets a `task_index_high_level`
-2. **Only sampled frames call VLM**: Huge efficiency gain
-3. **Temporal coherence**: Nearby frames share the same task
-4. **Episode-aware**: Always samples episode starts
-5. **Configurable**: Adjust `--sample-interval` based on your needs
-
-## Choosing Your Sampling Interval
-
-| Use Case | Recommended Interval | Why |
-|----------|---------------------|-----|
-| Quick testing | 2.0s | Fastest iteration |
-| Standard training | 1.0s | Good balance |
-| High-quality dataset | 0.5s | Better coverage |
-| Fine-grained control | 0.33s | Very detailed |
-| Dense annotations | 0.1s | Nearly every frame |
-
-**Rule of thumb**: Match your sampling interval to your typical skill duration.
-If skills last 1-3 seconds, sampling every 1 second captures each skill multiple times.
-
@@ -1,138 +0,0 @@
-#!/usr/bin/env python
-
-"""
-Example demonstrating how to use the ActionTokenizerProcessorStep to tokenize actions.
-
-This example shows how to:
-1. Load a dataset with action data
-2. Apply the action tokenizer processor to tokenize actions with proper padding/truncation
-3. Access both the tokenized actions and the attention mask
-4. Decode tokenized actions back to their original form
-"""
-
-import torch
-from transformers import AutoProcessor
-
-from lerobot.datasets.lerobot_dataset import LeRobotDataset
-from lerobot.processor.core import EnvTransition, TransitionKey
-from lerobot.processor.tokenizer_processor import ActionTokenizerProcessorStep
-from lerobot.utils.constants import ACTION_TOKEN_MASK
-
-# Define delta timestamps for the dataset
-delta_timestamps = {
-    'action': [
-        0.0, 0.03333333333333333, 0.06666666666666667, 0.1, 0.13333333333333333,
-        0.16666666666666666, 0.2, 0.23333333333333334, 0.26666666666666666, 0.3,
-        0.3333333333333333, 0.36666666666666664, 0.4, 0.43333333333333335,
-        0.4666666666666667, 0.5, 0.5333333333333333, 0.5666666666666667, 0.6,
-        0.6333333333333333, 0.6666666666666666, 0.7, 0.7333333333333333,
-        0.7666666666666667, 0.8, 0.8333333333333334, 0.8666666666666667, 0.9,
-        0.9333333333333333, 0.9666666666666667, 1.0, 1.0333333333333334,
-        1.0666666666666667, 1.1, 1.1333333333333333, 1.1666666666666667, 1.2,
-        1.2333333333333334, 1.2666666666666666, 1.3, 1.3333333333333333,
-        1.3666666666666667, 1.4, 1.4333333333333333, 1.4666666666666666, 1.5,
-        1.5333333333333334, 1.5666666666666667, 1.6, 1.6333333333333333
-    ]
-}
-
-# Load the dataset
-print("Loading dataset...")
-dataset = LeRobotDataset(
-    repo_id="local",
-    root="/fsx/jade_choghari/outputs/pgen_annotations1",
-    delta_timestamps=delta_timestamps
-)
-
-# Create a dataloader
-dataloader = torch.utils.data.DataLoader(
-    dataset,
-    num_workers=0,
-    batch_size=4,
-    shuffle=True,
-)
-
-# Get a batch of data
-batch = next(iter(dataloader))
-action_data = batch["action"]  # Shape: (batch_size, action_horizon, action_dim)
-
-print(f"\nOriginal action shape: {action_data.shape}")
-print(f"Original action data (first sample, first timestep):\n{action_data[0, 0]}")
-
-# Method 1: Using the tokenizer directly (as in fast_tokenize.py)
-print("\n" + "="*80)
-print("Method 1: Direct tokenizer usage")
-print("="*80)
-
-tokenizer = AutoProcessor.from_pretrained("physical-intelligence/fast", trust_remote_code=True)
-
-# Tokenize directly
-tokens = tokenizer(action_data)
-print(f"\nDirect tokenization result type: {type(tokens)}")
-print(f"Tokens shape/length: {tokens.shape if isinstance(tokens, torch.Tensor) else len(tokens)}")
-
-# Decode
-decoded_actions = tokenizer.decode(tokens)
-print(f"Decoded actions shape: {decoded_actions.shape}")
-reconstruction_error = torch.abs(action_data - decoded_actions).mean()
-print(f"Mean absolute reconstruction error: {reconstruction_error.item():.6f}")
-
-# Method 2: Using the ActionTokenizerProcessorStep with proper padding/truncation
-print("\n" + "="*80)
-print("Method 2: Using ActionTokenizerProcessorStep (with padding & mask)")
-print("="*80)
-
-# Create the action tokenizer processor step
-action_tokenizer_processor = ActionTokenizerProcessorStep(
-    tokenizer_name="physical-intelligence/fast",
-    trust_remote_code=True,
-    max_action_tokens=32,  # Maximum number of tokens per action
-)
-
-# Create a transition with the action data
-transition = {
-    TransitionKey.ACTION: action_data,
-    TransitionKey.OBSERVATION: {},  # Empty for this example
-}
-
-# Apply the processor
-processed_transition = action_tokenizer_processor(transition)
-
-# Extract tokenized actions and mask
-tokenized_actions = processed_transition[TransitionKey.ACTION]
-complementary_data = processed_transition[TransitionKey.COMPLEMENTARY_DATA]
-action_mask = complementary_data[ACTION_TOKEN_MASK]
-
-print(f"\nTokenized actions shape: {tokenized_actions.shape}")  # (batch_size, max_action_tokens)
-print(f"Action mask shape: {action_mask.shape}")  # (batch_size, max_action_tokens)
-print(f"Tokenized actions dtype: {tokenized_actions.dtype}")
-print(f"Action mask dtype: {action_mask.dtype}")
-
-# Show token statistics
-print(f"\nFirst sample tokens: {tokenized_actions[0]}")
-print(f"First sample mask: {action_mask[0]}")
-num_real_tokens = action_mask[0].sum().item()
-print(f"Number of real tokens (non-padding): {num_real_tokens}")
-print(f"Number of padding tokens: {action_mask.shape[1] - num_real_tokens}")
-
-# Decode using the mask
-print("\nDecoding tokenized actions...")
-decoded_with_processor = tokenizer.decode(tokenized_actions)
-print(f"Decoded actions shape: {decoded_with_processor.shape}")
-
-# Calculate reconstruction error
-reconstruction_error_processor = torch.abs(action_data - decoded_with_processor).mean()
-print(f"Mean absolute reconstruction error: {reconstruction_error_processor.item():.6f}")
-
-# Show that masking works correctly
-print("\n" + "="*80)
-print("Mask demonstration")
-print("="*80)
-for i in range(min(4, tokenized_actions.shape[0])):
-    mask_i = action_mask[i]
-    num_real = mask_i.sum().item()
-    print(f"Sample {i}: {num_real} real tokens, {len(mask_i) - num_real} padding tokens")
-
-print("\n" + "="*80)
-print("Action tokenization example completed successfully!")
-print("="*80)
-
@@ -1,143 +0,0 @@
-# Example: Synthetic Data Generation with Sampling
-
-## Quick Start
-
-### 1. Test with 100 frames and 1 second sampling
-```bash
-python examples/dataset/annotate_pgen.py \
-    --data-dir /fsx/jade_choghari/.cache/huggingface/lerobot/lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen2-VL-7B-Instruct \
-    --num-samples 100 \
-    --sample-interval 1.0 \
-    --output-dir ./outputs/test_pgen
-```
-
-**Expected behavior** (assuming 30 fps):
- Total frames: 100
- Frames sampled: ~4 (every 30 frames = 1 second)
- Efficiency: 96% fewer VLM calls
- Output: All 100 frames get `task_index_high_level`, but only 4 unique dialogues generated
-
-### 2. Process full dataset with different sampling rates
-
-#### Conservative (every 2 seconds)
-```bash
-python examples/dataset/annotate_pgen.py \
-    --data-dir /fsx/jade_choghari/.cache/huggingface/lerobot/lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen2-VL-7B-Instruct \
-    --sample-interval 2.0 \
-    --output-dir ./outputs/pgen_2s
-```
-
-#### Standard (every 1 second) - **RECOMMENDED**
-```bash
-python examples/dataset/annotate_pgen.py \
-    --data-dir /fsx/jade_choghari/.cache/huggingface/lerobot/lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen2-VL-7B-Instruct \
-    --sample-interval 1.0 \
-    --output-dir ./outputs/pgen_1s
-```
-
-#### Fine-grained (every 0.5 seconds)
-```bash
-python examples/dataset/annotate_pgen.py \
-    --data-dir /fsx/jade_choghari/.cache/huggingface/lerobot/lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen2-VL-7B-Instruct \
-    --sample-interval 0.5 \
-    --output-dir ./outputs/pgen_0.5s
-```
-
-## Performance Estimates
-
-For a dataset with:
- 100 episodes
- 10 seconds per episode (average)
- 30 fps
- Total frames: 30,000
-
-| Sampling Interval | Frames Sampled | % Sampled | Speedup | Time Estimate |
-|-------------------|----------------|-----------|---------|---------------|
-| Every frame (0.033s) | 30,000 | 100% | 1x | ~10 hours |
-| 0.5 seconds | 2,000 | 6.7% | 15x | ~40 min |
-| **1.0 seconds** | **1,000** | **3.3%** | **30x** | **~20 min** |
-| 2.0 seconds | 500 | 1.7% | 60x | ~10 min |
-
-*Note: Times are approximate and depend on GPU, model size, and generation speed*
-
-## Understanding the Output
-
-### Console Output Example
-```
-[cyan]Generating synthetic data for 30000 frames...[/cyan]
-[cyan]Sampling interval: 1.0s (fps: 30)[/cyan]
-Generating synthetic dialogue: 100%|████████| 30000/30000 [20:15<00:00, 24.68it/s]
-[green]✓ Sampled 1000 frames out of 30000 (3.3%)[/green]
-[green]✓ Generated 450 unique high-level tasks[/green]
-```
-
-### What happens:
-1. **Frame 0 (t=0.0s)**: Generate dialogue → Task index 0
-2. **Frames 1-29 (t=0.033s-0.967s)**: Reuse task index 0
-3. **Frame 30 (t=1.0s)**: Generate new dialogue → Task index 1
-4. **Frames 31-59 (t=1.033s-1.967s)**: Reuse task index 1
-5. And so on...
-
-### Result:
- Every frame has a `task_index_high_level`
- Only sampled frames have unique dialogues generated
- Intermediate frames inherit from the most recent sample
- Maintains temporal coherence within episodes
-
-## Checking Your Results
-
-After running, verify the output:
-
-```bash
-# Check the generated tasks
-python -c "
-import pandas as pd
-from pathlib import Path
-
-tasks = pd.read_parquet('outputs/test_pgen/meta/tasks_high_level.parquet')
-print(f'Total unique tasks: {len(tasks)}')
-print(f'Sample tasks:')
-print(tasks[['user_prompt', 'robot_utterance', 'skill']].head())
-"
-
-# Check debug output
-head outputs/test_pgen/meta/syn_annotations.jsonl
-
-# Load and verify dataset
-python -c "
-from lerobot.datasets.lerobot_dataset import LeRobotDataset
-
-ds = LeRobotDataset(repo_id='local_with_high_level_tasks', 
-                    root='outputs/test_pgen')
-print(f'Dataset has {len(ds)} frames')
-print(f'Features: {list(ds.features.keys())}')
-assert 'task_index_high_level' in ds.features
-print('✓ task_index_high_level feature added successfully!')
-"
-```
-
-## Common Use Cases
-
-### Development/Testing
-```bash
--sample-interval 2.0  # Fast iteration
--num-samples 500      # Small subset
-```
-
-### Production Training
-```bash
--sample-interval 1.0  # Good coverage
-# Process all samples (no --num-samples)
-```
-
-### High-Quality Dataset
-```bash
--sample-interval 0.5  # Fine-grained
--temperature 0.6      # More consistent
--model Qwen/Qwen3-VL-30B-A3B-Instruct  # Larger model
-```
-
@@ -1,25 +0,0 @@
-import numpy as np
-from transformers import AutoProcessor
-import torch
-from lerobot.datasets.lerobot_dataset import LeRobotDataset, LeRobotDatasetMetadata
-
-delta_timestamps = {'action': [0.0, 0.03333333333333333, 0.06666666666666667, 0.1, 0.13333333333333333, 0.16666666666666666, 0.2, 0.23333333333333334, 0.26666666666666666, 0.3, 0.3333333333333333, 0.36666666666666664, 0.4, 0.43333333333333335, 0.4666666666666667, 0.5, 0.5333333333333333, 0.5666666666666667, 0.6, 0.6333333333333333, 0.6666666666666666, 0.7, 0.7333333333333333, 0.7666666666666667, 0.8, 0.8333333333333334, 0.8666666666666667, 0.9, 0.9333333333333333, 0.9666666666666667, 1.0, 1.0333333333333334, 1.0666666666666667, 1.1, 1.1333333333333333, 1.1666666666666667, 1.2, 1.2333333333333334, 1.2666666666666666, 1.3, 1.3333333333333333, 1.3666666666666667, 1.4, 1.4333333333333333, 1.4666666666666666, 1.5, 1.5333333333333334, 1.5666666666666667, 1.6, 1.6333333333333333]}
-dataset = LeRobotDataset(repo_id="local", root="/fsx/jade_choghari/outputs/pgen_annotations1", delta_timestamps=delta_timestamps)
-
-dataloader = torch.utils.data.DataLoader(
-        dataset,
-        num_workers=0,
-        batch_size=4,
-        shuffle=True,
-)
-
-batch = next(iter(dataloader))
-
-# Load the tokenizer from the Hugging Face hub
-tokenizer = AutoProcessor.from_pretrained("physical-intelligence/fast", trust_remote_code=True)
-
-# Tokenize & decode action chunks (we use dummy data here)
-action_data = batch["action"]    # one batch of action chunks
-tokens = tokenizer(action_data)              # tokens = list[int]
-decoded_actions = tokenizer.decode(tokens)
-print("tokenized actions: ", tokens)
@@ -1,17 +0,0 @@
-from transformers import AutoProcessor, PaliGemmaForConditionalGeneration
-
-model_id = "google/paligemma-3b-pt-224"
-model = PaliGemmaForConditionalGeneration.from_pretrained(model_id)
-processor = AutoProcessor.from_pretrained(model_id)
-
-breakpoint()
-prefix_output = model.language_model.forward(
-    inputs_embeds=inputs_embeds[0],
-    attention_mask=attention_mask,
-    position_ids=position_ids,
-    adarms_cond=adarms_cond[0] if adarms_cond is not None else None,
-)
-prefix_past_key_values = prefix_output.past_key_values
-# prefix_output to be used for the language head
-# shape: [batch_size, seq_len, hidden_size] with hidden_size = 2048
-prefix_output = prefix_output.last_hidden_state
@@ -1,91 +0,0 @@
-import torch
-from huggingface_hub import HfApi
-
-import lerobot
-from lerobot.datasets.lerobot_dataset import LeRobotDataset, LeRobotDatasetMetadata
-# import make_pre_post_processors
-from lerobot.policies.factory import make_pre_post_processors
-from lerobot.policies.pi05.configuration_pi05 import PI05Config
-from lerobot.policies.factory import make_policy, make_policy_config
-from lerobot.configs.policies import PreTrainedConfig
-
-cfg = PreTrainedConfig.from_pretrained(
-    pretrained_name_or_path="/fsx/jade_choghari/outputs/pi0_training/checkpoints/last/pretrained_model",
-)
-cfg.dtype = "bfloat16"
-
-pre_processor, post_processor = make_pre_post_processors(
-    policy_cfg=cfg,
-    pretrained_path="/fsx/jade_choghari/outputs/pi0_training/checkpoints/last/pretrained_model",
-)
-
-delta_timestamps = {'action': [0.0, 0.03333333333333333, 0.06666666666666667, 0.1, 0.13333333333333333, 0.16666666666666666, 0.2, 0.23333333333333334, 0.26666666666666666, 0.3, 0.3333333333333333, 0.36666666666666664, 0.4, 0.43333333333333335, 0.4666666666666667, 0.5, 0.5333333333333333, 0.5666666666666667, 0.6, 0.6333333333333333, 0.6666666666666666, 0.7, 0.7333333333333333, 0.7666666666666667, 0.8, 0.8333333333333334, 0.8666666666666667, 0.9, 0.9333333333333333, 0.9666666666666667, 1.0, 1.0333333333333334, 1.0666666666666667, 1.1, 1.1333333333333333, 1.1666666666666667, 1.2, 1.2333333333333334, 1.2666666666666666, 1.3, 1.3333333333333333, 1.3666666666666667, 1.4, 1.4333333333333333, 1.4666666666666666, 1.5, 1.5333333333333334, 1.5666666666666667, 1.6, 1.6333333333333333]}
-
-dataset = LeRobotDataset(repo_id="local", root="/fsx/jade_choghari/outputs/pgen_annotations1", delta_timestamps=delta_timestamps)
-
-# rename map --rename_map='{
-#         "observation.images.side": "observation.images.base_0_rgb",
-#         "observation.images.up": "observation.images.left_wrist_0_rgb"
-#         }'
-rename_map = {
-    "observation.images.side": "observation.images.base_0_rgb",
-    "observation.images.up": "observation.images.left_wrist_0_rgb"
-}
-policy = make_policy(
-    cfg=cfg,
-    ds_meta=dataset.meta,
-    rename_map=rename_map,
-)
-
-dataloader = torch.utils.data.DataLoader(
-        dataset,
-        num_workers=0,
-        batch_size=4,
-        shuffle=True,
-)
-
-batch = next(iter(dataloader))
-batch = pre_processor(batch)
-policy.train()
-# run inference
-# action = policy.select_action(batch)
-loss, loss_dict = policy.forward(batch)
-breakpoint()
-# import requests
-# from PIL import Image
-# from transformers import AutoProcessor
-# model = policy.model.paligemma_with_expert.paligemma
-# model = model.to(device="cuda", dtype=torch.bfloat16)
-# model.eval()
-# prompt = "Describe this image."
-# url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
-# image = Image.open(requests.get(url, stream=True).raw)
-# processor = AutoProcessor.from_pretrained(
-#     "google/paligemma-3b-pt-224",
-# )
-# inputs = processor(image, prompt, return_tensors="pt").to(model.device)
-# print("generating...")
-# output = model.generate(
-#     **inputs,
-#     max_new_tokens=50,
-#     use_cache=True,  # default dynamic cache
-# )
-# print(processor.decode(output[0], skip_special_tokens=True))
-
-
-# # other model
-# from transformers import PaliGemmaForConditionalGeneration
-# model = PaliGemmaForConditionalGeneration.from_pretrained(
-#     "google/paligemma2-3b-pt-224",
-#     torch_dtype=torch.bfloat16,
-#     device_map="auto",
-# )
-# model.eval()
-# print("generating...")
-# output = model.generate(
-#     **inputs,
-#     max_new_tokens=100,
-#     use_cache=True,  # default dynamic cache
-# )
-# print("Model 2 output:")
-# print(processor.decode(output[0], skip_special_tokens=True))
@@ -1,23 +0,0 @@
-import torch
-from huggingface_hub import HfApi
-
-import lerobot
-from lerobot.datasets.lerobot_dataset import LeRobotDataset, LeRobotDatasetMetadata
-
-dataset = LeRobotDataset(repo_id="local", root="/fsx/jade_choghari/outputs/pgen_annotations1")
-
-dataloader = torch.utils.data.DataLoader(
-        dataset,
-        num_workers=0,
-        batch_size=32,
-        shuffle=True,
-)
-
-batch = next(iter(dataloader))
-print(batch.keys())
-print(batch['task_index_high_level'].shape)
-print(batch['task_index_high_level'])
-print(batch['user_prompt'][0])
-print(batch['robot_utterance'][0])
-print(batch['task'][0])
-breakpoint()
@@ -1,159 +0,0 @@
-## One-sentence answer
-
-> `make_att_2d_masks(prefix_pad_masks, prefix_att_masks)` builds the **actual 2D attention mask** `[B, L, L]` that tells the transformer **which token positions may attend to which others**, combining **padding** and **causality**.
-
-Everything else you’ve seen so far was just metadata.
-
---
-
-## What goes in
-
-### Inputs
-
-```python
-prefix_pad_masks   # shape [B, L]
-prefix_att_masks   # shape [B, L]
-```
-
-Where:
-
-* `prefix_pad_masks[b, i] = True`
-  → token `i` exists (not padding)
-
-* `prefix_att_masks[b, i] = False`
-  → token `i` is **bidirectional**
-
-* `prefix_att_masks[b, i] = True`
-  → token `i` is **causal (autoregressive)**
-
---
-
-## What comes out
-
-```python
-att_2d_prefix  # shape [B, L, L]
-```
-
-Each entry:
-
-```text
-att_2d_prefix[b, i, j] = True
-```
-
-means:
-
-> “In batch `b`, **token i (query)** is allowed to attend to **token j (key)**.”
-
---
-
-## How it is constructed (conceptually)
-
-For **each batch b**, **each query position i**, **each key position j**:
-
-```python
-if not prefix_pad_masks[b, j]:
-    att[b, i, j] = False           # cannot attend to padding
-else if not prefix_att_masks[b, i]:
-    att[b, i, j] = True            # bidirectional token → can see all real tokens
-else:
-    att[b, i, j] = (j <= i)        # causal token → can see only past + itself
-```
-
-That’s it.
-
---
-
-## Tiny concrete example (exactly matching your code)
-
-Suppose:
-
-```python
-prefix_pad_masks[0] = [T, T, T, T, T, F]
-prefix_att_masks[0] = [F, F, F, T, T, T]
-```
-
-Tokens:
-
-```
-0: IMG
-1: IMG
-2: LANG
-3: SUB0
-4: SUB1
-5: PAD
-```
-
---
-
-### Resulting `att_2d_prefix[0]`
-
-`✓ = True, ✗ = False`
-
-| Q \ K      | 0 | 1 | 2 | 3 | 4 | 5 |
-| ---------- | - | - | - | - | - | - |
-| 0 (bi)     | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
-| 1 (bi)     | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
-| 2 (bi)     | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
-| 3 (causal) | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ |
-| 4 (causal) | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
-| 5 (pad)    | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
-
---
-
-## Why this matters for your training code
-
-This line:
-
-```python
-att_2d_prefix_4d = self._prepare_attention_masks_4d(att_2d_prefix)
-```
-
-Converts `[B, L, L] → [B, 1, L, L]` and possibly flips True/False to `0/-inf`.
-
-This is **exactly what Paligemma uses inside self-attention**.
-
---
-
-## Key implications (VERY important)
-
-### 1️⃣ This mask does **not isolate token groups**
-
-* Bidirectional tokens can attend to **everything**
-* Causal tokens only restrict *their own row*
-
-So **flow/action tokens must be blocked separately**.
-
---
-
-### 2️⃣ This is why your AR subtask prediction works
-
-* Subtask tokens are causal
-* Output at position `i` predicts token `i+1`
-* Padding is fully ignored
-
---
-
-### 3️⃣ Inference behavior
-
-When `subtask_tokens = None`:
-
-* `prefix_att_masks` contains only `False`
-* `att_2d_prefix` becomes **fully bidirectional**
-* No AR behavior remains
-
-Exactly what you want.
-
---
-
-## One-sentence takeaway (commit this)
-
-> `make_att_2d_masks` fuses **padding** and **causality** into a concrete `[B, L, L]` attention matrix that the transformer actually uses.
-
-If you want next, I can:
-
-* inspect `make_att_2d_masks()` source with you
-* show how to block **flow → subtask** attention
-* explain how this changes when suffix tokens are added
-* help you refactor this into a cleaner “grouped attention” API
-
-You’re now at the point where the model’s behavior should feel *predictable*, not magical.
@@ -1,334 +0,0 @@
-Generate annotate_pgen.py using Qwen for synthetic data generation
-
-You are writing a Python script called annotate_pgen.py.
-This script generates synthetic user prompts (ℓ_t) and robot utterances (u_t) for Hi Robot–style hierarchical policy training, using Qwen 3vl as the generator model (pgen).
-
-SCRIPT PURPOSE
-
-The script must:
-
-Load Dlabeled which is a LeRobot Dataset that has been annotate using the annotate.py script, which contains:
-
-images: list of image paths at time t
-
-skill_current: the annotated skill label (ℓ̂_t)
-
-skill_history: list of previous skill labels (ℓ̂₀ … ℓ̂_{t−1}), those where annotated, and you can find details on them stored in teh dataset inside the the DATA_PATH/meta/skills.json
-
-you will find something like 
-
-{
-  "coarse_description": "pink lego brick into the transparent box",
-  "skill_to_task_index": {
-    "robot arm picks up pink lego brick": 19,
-    "robot arm approaches transparent box": 3,
-    "robot arm retracts from transparent box": 28,
-    "robot arm moves towards pink lego brick": 12,
-    "robot arm releases red lego brick into box": 26,
-    "robot arm releases red lego brick into transparent box": 27,
-    "robot arm closes gripper to pick up the pink lego brick": 5,
-    "robot arm lifts the pink lego brick": 7,
-    etc..
-  },
-  "episodes": {
-    "0": {
-      "episode_index": 0,
-      "description": "pink lego brick into the transparent box",
-      "skills": [
-        {
-          "name": "robot arm moves towards pink lego brick",
-          "start": 0.0,
-          "end": 1.8
-        },
-        {
-          "name": "robot arm picks up pink lego brick",
-          "start": 1.8,
-          "end": 3.1
-        },
-        {
-          "name": "robot arm moves towards transparent box",
-          "start": 3.1,
-          "end": 5.5
-        },
-        {
-          "name": "robot arm releases pink lego brick into transparent box",
-          "start": 5.5,
-          "end": 7.0
-        },
-        {
-          "name": "robot arm retracts from transparent box",
-          "start": 7.0,
-          "end": 10.1
-        }
-      ]
-    },
-    "1": {
-      "episode_index": 1,
-      "description": "pink lego brick into the transparent box",
-      "skills": [
-        {
-          "name": "robot arm moves towards red lego brick",
-          "start": 0.0,
-          "end": 1.2
-        },
-        {
-          "name": "robot arm picks up red lego brick",
-          "start": 1.2,
-          "end": 2.0
-        },
-        {
-          "name": "robot arm moves towards transparent box",
-          "start": 2.0,
-          "end": 3.8
-        },
-        {
-          "name": "robot arm places red lego brick into transparent box",
-          "start": 3.8,
-          "end": 5.0
-        },
-        {
-          "name": "robot arm moves away from transparent box",
-          "start": 5.0,
-          "end": 8.9
-        }
-      ]
-    },
-
-notice how task_description: is a high-level description (e.g., "make a sandwich") stored in description for each episode
-
-For each sample, call Qwen VLM to generate:
-
-synthetic user prompt ℓ_t
-
-synthetic robot response u_t
-
-Save results to D_syn in Parquet format insdie DATA_PATH/meta/tasks.parquet ; note tasks.parquet already contains the other tasks, so you need to update
-
-Should be modular, clean, easy to extend, with:
-
-a PGEN_PROMPT_TEMPLATE
-
-a construct_prompt() method
-
-a call_qwen() method
-
-a annotate_sample() method
-
-a CLI entrypoint (if __name__ == "__main__":)
-
-📦 INPUT FORMAT (Dlabeled)
-
-The script should expect Dlabeled as a .jsonl file where each line has:
-
-{
-  "episode_id": "ep_001",
-  "t": 37,
-  "images": ["path/to/cam0_t.jpg", "path/to/cam1_t.jpg"],
-  "skill_current": "pick up the KitKat",
-  "skill_history": ["open fridge", "pick up lettuce", "place lettuce"],
-  "task_description": "making a sandwich"
-}
-
-📤 OUTPUT FORMAT (D_syn)
-
-Each line of synthetically generated data should be:
-
-{
-  "episode_id": "ep_001",
-  "t": 37,
-  "images": ["path/to/cam0_t.jpg", "path/to/cam1_t.jpg"],
-  "skill_current": "pick up the KitKat",
-  "skill_history": [...],
-  "user_prompt": "Can you grab me something sweet?",
-  "robot_utterance": "Sure, I can pick up the KitKat.",
-  "task_description": "making a sandwich"
-}
-
-
-Store as syn_annotations.jsonl. for debugging
-
-🧠 pgen MODEL (Qwen) REQUIREMENTS
-
-Use HuggingFace Transformers:
-
-Qwen/Qwen2-VL-7B-Instruct (or any Qwen2-VL Vision-Language model available)
-
-Use the image + text chat interface
-
-Vision inputs should be loaded with PIL
-
-Use a single forward pass that outputs BOTH ℓ_t and u_t in a structured JSON
-
-📝 PROMPT FORMAT FOR pgen
-
-Create a template like:
-
-You are a robot-assistant dialogue generator for hierarchical robot policies.
-
-You will receive:
- A list of images showing the current robot scene.
- The high-level task: {task_description}
- Previous skill steps completed: {skill_history}
- The next skill to be performed by the robot: {skill_current}
-
-Generate two things in JSON:
-1. "user_prompt": a natural-sounding user request that logically leads to the robot performing the skill "{skill_current}" given the task and history.
-2. "robot_utterance": a natural robot reply acknowledging or clarifying the request.
-
-The responses must be grounded in the visual scene, the task, and the skill history.
-
-Respond ONLY in JSON:
-{
-  "user_prompt": "...",
-  "robot_utterance": "..."
-}
-
-This resposne will have a corresponsing task_index, and the task will be saved in task.parqeut and you must update each dataset parquet in for example /fsx/jade_choghari/.cache/huggingface/lerobot/lerobot/svla_so101_pickplace/data/chunk-000/
-file-000.parquet to include this new feature called task_index_high_level consider udpatign the metadata in info.json as well
-📌 LOGIC REQUIRED
-construct_prompt(sample)
-
-Loads sample dict
-
-Inserts:
-
-task_description
-
-skill_history
-
-skill_current
-
-Returns a full text prompt string
-
-call_qwen(images, prompt)
-
-Loads images into Qwen-VL multimodal input format
-
-Calls model.generate
-
-Parses JSON output
-
-annotate_sample(sample)
-
-Builds prompt
-
-Calls Qwen
-
-Returns augmented sample with user_prompt + robot_utterance
-
-🚀 CLI Usage
-
-The script should run as:
-
-python annotate_pgen.py \
-  --output-dir PATH \
-  --model Qwen/Qwen2-VL-7B-Instruct \
-  --repo-id lerobot/svla_so101_pickplace \
-  --model Qwen/Qwen3-VL-30B-A3B-Instruct \
-  --batch-size 1
-
-
-Include arguments via argparse.
-
-🔧 OTHER REQUIREMENTS
-
-Use tqdm for progress bars
-
-Log errors gracefully and continue
-
-Support GPU acceleration (device="cuda")
-
-Cache model loading so it's not reloaded every call
-
-Make the prompt deterministic but allow temperature parameter
-
-Add a flag --num-image-views-per-sample
-
-Add automatic JSON parsing with helpful error messages
-
-🎯 FINAL DELIVERABLE
-
-Cursor must now generate:
-A full Python file named annotate_pgen.py implementing the above functionality end-to-end.
-
-It should be production-ready, runnable on real data, cleanly structured, and easy to modify.
-
-
-from the paper:
-Next, we use a large vision-language model (VLM) pgen
-to produce synthetic user prompts and interjections ℓt,
-and corresponding robot utterance ut. Given Dlabeled, we
-prompt pgen with both the visual context I1
-t ,...,In
-t and the
-skill labelˆ
-ℓt (e.g., pick up the lettuce). pgen then imag-
-ines an appropriate interaction that might have led toˆ
-ℓt in a
-real user interaction: it generates possible user prompts ℓt
-(e.g., “Can you add some lettuce for me?”) along with the
-robot’s verbal responses and clarifications ut. We detail the
-A. Synthetic Data Generation
-A.1. Scenario and Response Categorization
-To ensure the quality and diversity of the synthetic data,
-we incorporate structured scenario classification and re-
-sponse categorization into the prompt design for pgen, fol-
-lowing (Stephan et al., 2024). Specifically, we classify
-interactions into different scenario types, such as nega-
-tive task (where the user instructs the robot what not to
-do), situated correction (where the user adjusts an earlier
-command based on the evolving task state), and specific
-constraint (where the user specifies particular constraints,
-such as dietary preferences). In addition, we categorize
-the robot’s responses into types such as simple confirma-
-tions, clarifications, and error handling. These classifica-
-tions guide the generation process to ensure a broad range
-of user-robot interactions.
-A.2. Prompt Construction for Contextual Grounding
-In prompt P, we include a detailed description of the task
-(e.g., bussing a table, making a sandwich, grocery shop-
-ping) and instruct the model to ground responses in visual
-observations and prior context. A key advantage of lever-
-aging large pretrained VLMs is their ability to incorporate
-world knowledge when generating interactions. For in-
-stance, the model can infer dietary constraints when gener-
-ating prompts for sandwich-making, producing user com-
-mands such as “Can you make a sandwich for me? I’m
-lactose intolerant” and an appropriate robot response like
-“Sure, I won’t put cheese on it.” Similarly, it can reason
-over ambiguous or implicit requests, such as inferring that
-“I want something sweet” in a grocery shopping scenario
-should lead to suggestions like chocolate or candy.
-To maintain consistency in multi-step tasks, we condition
-pgen on prior skill labels within an episodeˆ
-ˆ
-ℓ0,...,
-ℓt−1,
-allowing it to generate coherent user commands that
-account for past actions. For instance, if the robot
-has already placed lettuce and tomato on a sandwich,
-the generated user prompt might request additional in-
-gredients that logically follow. This ensures that the
-synthetic interactions reflect realistic task progression
-rather than isolated commands. As such, we leverage
-ˆ
-ˆ
-ˆ
-pgen(ℓt,ut|I1
-t ,...,In
-t ,
-ℓ0,...,
-ℓt−1,
-ℓt,P) to produce a richer,
-more diverse synthetic dataset Dsyn that provides mean-
-ingful supervision for training our high-level policy.
-While in this work we generate a separate Dsyn and train
-a separate high-level policy for each task (e.g., sandwich
-making vs. table cleaning) for clarity and ease of bench-
-marking, the architecture is readily amenable to a unified
-multi-task formulation. In principle, the same hierarchical
-approach could be used to train a single high-level policy
-across a multitude of tasks, facilitating knowledge transfer
-
-
-The result should be a new LeRobotDataset with a new feature called task_index_high_level inside each dataset parquet
@@ -1,11 +0,0 @@
-python examples/dataset/annotate.py \
-    --repo-id jadechoghari/collect-data \
-    --video-key observation.images.base \
-    --model Qwen/Qwen3-VL-30B-A3B-Instruct \
-    --episodes 16 22
-
-# python examples/dataset/annotate.py \
-#     --repo-id lerobot/svla_so101_pickplace \
-#     --video-key observation.images.side \
-#     --model Qwen/Qwen3-VL-30B-A3B-Instruct \
-#     --episodes 5
@@ -1,43 +0,0 @@
-#!/bin/bash
-
-# Example script to run synthetic data generation with Qwen VLM
-# This generates user prompts and robot utterances for hierarchical policy training
-
-# Configuration
-REPO_ID="jadechoghari/collect-data"
-MODEL="Qwen/Qwen3-VL-30B-A3B-Instruct"
-# Alternative: MODEL="Qwen/Qwen2-VL-7B-Instruct"
-
-
-OUTPUT_DIR="/fsx/jade_choghari/outputs/collect-data-pgen"
-BATCH_SIZE=32
-TEMPERATURE=0.9
-SAMPLE_INTERVAL=5.0  # Generate dialogue every 1 second (all episodes processed)
-
-# Run synthetic data generation (processes ALL episodes)
-python examples/dataset/annotate_pgen.py \
-    --repo-id "$REPO_ID" \
-    --model "$MODEL" \
-    --output-dir "$OUTPUT_DIR" \
-    --temperature "$TEMPERATURE" \
-    --batch-size "$BATCH_SIZE" \
-    --sample-interval "$SAMPLE_INTERVAL" \
-    --image-key observation.images.base \
-    --num-image-views-per-sample 1
-
-# For faster testing, increase sample interval:
-# --sample-interval 5.0  # Samples every 5 seconds (much faster)
-
-# To push to hub after generation:
-# Add --push-to-hub flag
-
-# Efficient batch processing: 4 episodes at once
-# python examples/dataset/annotate_pgen.py \
-#     --repo-id "$REPO_ID" \
-#     --model "$MODEL" \
-#     --output-dir "$OUTPUT_DIR" \
-#     --video-mode \
-#     --video-key observation.images.up \
-#     --video-batch-size "$BATCH_SIZE" \
-#     --sample-interval 1.0
-
@@ -1 +0,0 @@
-srun --time 12:00:00     --qos=high     --gres=gpu:1     --mem=24G     --partition=hopper-prod     --container-image /fsx/michel_aractingi/docker_images/huggingface+lerobot-gpu+dev.sqsh     --container-mounts /fsx/jade_choghari   
@@ -1,44 +0,0 @@
-#!/bin/bash
-
-# Quick test to verify the fix for task_indices length mismatch
-# This should now work correctly even with --num-samples < full dataset length
-
-echo "Testing annotate_pgen.py with --num-samples=100 on full dataset..."
-
-python examples/dataset/annotate_pgen.py \
-    --data-dir /fsx/jade_choghari/.cache/huggingface/lerobot/lerobot/svla_so101_pickplace \
-    --model Qwen/Qwen3-VL-30B-A3B-Instruct \
-    --num-samples 100 \
-    --sample-interval 1.0 \
-    --output-dir /fsx/jade_choghari/outputs/pgen_test_fixed
-
-if [ $? -eq 0 ]; then
-    echo "✓ SUCCESS: Script completed without errors!"
-    echo ""
-    echo "Verifying output..."
-    
-    # Check that all frames have task_index_high_level
-    python -c "
-from lerobot.datasets.lerobot_dataset import LeRobotDataset
-import numpy as np
-
-ds = LeRobotDataset(repo_id='local_test', root='/fsx/jade_choghari/outputs/pgen_test_fixed')
-print(f'Dataset has {len(ds)} frames')
-print(f'Features: {list(ds.features.keys())}')
-
-# Check that task_index_high_level exists
-assert 'task_index_high_level' in ds.features, 'task_index_high_level not in features!'
-
-# Sample some frames
-for idx in [0, 50, 99, 100, 500, 1000, 11938]:
-    if idx < len(ds):
-        frame = ds[idx]
-        task_idx = frame['task_index_high_level'].item()
-        print(f'Frame {idx}: task_index_high_level = {task_idx}')
-
-print('✓ All checks passed!')
-"
-else
-    echo "✗ FAILED: Script exited with error code $?"
-fi
-
@@ -455,7 +455,18 @@ def demo_cli(cfg: RTCDemoConfig):
    if cfg.policy.type == "pi05" or cfg.policy.type == "pi0":
        config.compile_model = cfg.use_torch_compile

-    policy = policy_class.from_pretrained(cfg.policy.pretrained_path, config=config)
+    if config.use_peft:
+        from peft import PeftConfig, PeftModel
+
+        peft_pretrained_path = cfg.policy.pretrained_path
+        peft_config = PeftConfig.from_pretrained(peft_pretrained_path)
+
+        policy = policy_class.from_pretrained(
+            pretrained_name_or_path=peft_config.base_model_name_or_path, config=config
+        )
+        policy = PeftModel.from_pretrained(policy, peft_pretrained_path, config=peft_config)
+    else:
+        policy = policy_class.from_pretrained(cfg.policy.pretrained_path, config=config)

    # Turn on RTC
    policy.config.rtc_config = cfg.rtc
@@ -1,47 +0,0 @@
-# Voice Assistant Examples
-
-Voice-enabled robot assistant examples using speech-to-text (STT), and text-to-speech (TTS).
-
-## Overview
-
-These examples demonstrate how to build a voice interface for robot control:
-
-1. **Hold SPACE** → Push-to-talk recording starts
-2. **Release SPACE** → Recording stops
-3. **STT (Whisper)** → Converts speech to text (high-level task prompt)
-4. **Pi0.5** → Generates robot response/utterance
-5. **TTS (Kokoro)** → Speaks the response back
-
-## Requirements
-
-```bash
-pip install torch transformers sounddevice numpy pynput kokoro>=0.9.2
-```
-
-## Usage
-
-### With Pi0.5 Model
-
-```bash
-python examples/voice_assistant/voice_assistant_pi05.py \
-    --pretrained_path path/to/pi05/checkpoint
-```
-
-## How It Works
-
-### Pi0.5 Voice Integration
-
-Pi0.5 can generate robot utterances as part of its subtask prediction. The flow:
-
-1. **High-level prompt**: User voice command is transcribed and formatted as a task prompt
-2. **Subtask generation**: Pi0.5 autoregressively generates a response
-3. **Utterance extraction**: If the response contains `<utterance>...</utterance>` tags, the content is extracted
-4. **TTS output**: The response is spoken back to the user
-
-## Configuration Options
-
-| Option | Default | Description |
-|--------|---------|-------------|
-| `--pretrained_path` | None | Path to Pi0.5 checkpoint |
-| `--record_seconds` | 5.0 | Audio recording duration |
-| `--max_response_tokens` | 100 | Max tokens in generated response |
@@ -1,336 +0,0 @@
-#!/usr/bin/env python
-"""
-Voice Assistant with Pi0.5: Microphone → STT → Pi0.5 → TTS → Speaker
-
-This example demonstrates how to use Pi0.5 as a conversational robot assistant:
-1. Hold SPACE to record your voice command
-2. Speech-to-text (Whisper) converts speech to text
-3. Text is fed as a high-level prompt to Pi0.5
-4. Pi0.5 generates a response (robot utterance)
-5. Text-to-speech (Kokoro) speaks the response back
-
-Requirements:
-    pip install torch transformers sounddevice numpy pynput kokoro>=0.9.2
-
-Usage:
-    python examples/voice_assistant/voice_assistant_pi05.py \
-        --pretrained_path lerobot/pi0.5-base
-"""
-
-import os
-
-os.environ["TOKENIZERS_PARALLELISM"] = "false"
-
-import argparse
-import re
-import subprocess
-import threading
-import time
-
-import numpy as np
-import sounddevice as sd
-import torch
-from pynput import keyboard
-from transformers import AutoTokenizer, WhisperForConditionalGeneration, WhisperProcessor
-
-from lerobot.policies.pi05.configuration_pi05 import PI05Config
-from lerobot.policies.pi05.modeling_pi05 import PI05Pytorch
-
-SAMPLE_RATE = 16000
-
-
-def get_device():
-    if torch.cuda.is_available():
-        return torch.device("cuda")
-    elif torch.backends.mps.is_available():
-        return torch.device("mps")
-    return torch.device("cpu")
-
-
-class Pi05VoiceAssistant:
-    """Voice assistant using Pi0.5 for generating robot utterances."""
-
-    def __init__(
-        self,
-        pretrained_path: str | None = None,
-        max_response_tokens: int = 100,
-        max_record_seconds: float = 30.0,
-    ):
-        self.device = get_device()
-        self.dtype = torch.float32 if self.device.type == "mps" else torch.bfloat16
-        self.max_response_tokens = max_response_tokens
-        self.max_record_seconds = max_record_seconds
-
-        # Push-to-talk state
-        self._recording = False
-        self._audio_chunks: list[np.ndarray] = []
-        self._stream: sd.InputStream | None = None
-
-        print(f"Using device: {self.device}")
-        self._load_models(pretrained_path)
-
-    def _load_models(self, pretrained_path: str | None):
-        print("Loading STT (Whisper tiny)...")
-        self.stt_processor = WhisperProcessor.from_pretrained("openai/whisper-tiny.en")
-        self.stt_model = WhisperForConditionalGeneration.from_pretrained(
-            "openai/whisper-tiny.en", torch_dtype=self.dtype
-        ).to(self.device)
-
-        print("Loading Pi0.5 model...")
-        self._load_pi05(pretrained_path)
-
-        print("Loading tokenizer...")
-        self.tokenizer = AutoTokenizer.from_pretrained("google/paligemma-3b-pt-224")
-
-        self._load_tts()
-        print("Ready!\n")
-
-    def _load_pi05(self, pretrained_path: str | None):
-        """Load Pi0.5 model for utterance generation."""
-        config = PI05Config()
-        config.dtype = "float32" if self.device.type == "mps" else "bfloat16"
-
-        self.pi05_model = PI05Pytorch(config)
-
-        if pretrained_path:
-            try:
-                from safetensors.torch import load_file
-                state_dict = load_file(f"{pretrained_path}/model.safetensors")
-                self.pi05_model.load_state_dict(state_dict, strict=False)
-                print(f"✓ Loaded Pi0.5 weights from {pretrained_path}")
-            except Exception as e:
-                print(f"Warning: Could not load pretrained weights: {e}")
-                print("Using randomly initialized model for demo purposes")
-
-        self.pi05_model = self.pi05_model.to(self.device)
-        self.pi05_model.eval()
-
-    def _load_tts(self):
-        try:
-            print("Loading TTS (Kokoro 82M)...")
-            from kokoro import KPipeline
-
-            self.tts_pipeline = KPipeline(lang_code="a")  # American English
-            self.tts_voice = "af_heart"
-            self.tts_type = "kokoro"
-            print("Kokoro loaded!")
-        except Exception as e:
-            print(f"Kokoro not available ({e})")
-            print("Using macOS `say` for TTS")
-            self.tts_pipeline = None
-            self.tts_type = "system"
-
-    def _audio_callback(self, indata, frames, time_info, status):
-        """Callback for audio stream - collects chunks while recording."""
-        if self._recording:
-            self._audio_chunks.append(indata.copy())
-
-    def _start_recording(self):
-        """Start recording audio."""
-        if self._recording:
-            return
-        self._recording = True
-        self._audio_chunks = []
-        print("🎤 Recording... (release SPACE to stop)")
-
-    def _stop_recording(self) -> np.ndarray | None:
-        """Stop recording and return the audio."""
-        if not self._recording:
-            return None
-        self._recording = False
-
-        if not self._audio_chunks:
-            return None
-
-        audio = np.concatenate(self._audio_chunks, axis=0).flatten()
-        duration = len(audio) / SAMPLE_RATE
-        volume = np.abs(audio).max()
-        print(f"Recorded {duration:.1f}s, volume: {volume:.4f}")
-
-        if volume < 0.001:
-            print("⚠️  Very low audio - check microphone permissions!")
-            return None
-
-        return audio
-
-    def wait_for_spacebar(self) -> np.ndarray | None:
-        """Wait for spacebar press, record while held, return audio on release."""
-        audio_result = None
-        recording_done = threading.Event()
-
-        def on_press(key):
-            if key == keyboard.Key.space:
-                self._start_recording()
-
-        def on_release(key):
-            nonlocal audio_result
-            if key == keyboard.Key.space and self._recording:
-                audio_result = self._stop_recording()
-                recording_done.set()
-                return False  # Stop listener
-
-        # Start audio stream
-        self._stream = sd.InputStream(
-            samplerate=SAMPLE_RATE,
-            channels=1,
-            dtype="float32",
-            callback=self._audio_callback,
-            blocksize=int(SAMPLE_RATE * 0.1),  # 100ms blocks
-        )
-
-        with self._stream:
-            print("\n⏳ Press and hold SPACE to speak...")
-            with keyboard.Listener(on_press=on_press, on_release=on_release) as listener:
-                # Wait for recording to complete or timeout
-                recording_done.wait(timeout=self.max_record_seconds)
-                if self._recording:
-                    audio_result = self._stop_recording()
-
-        return audio_result
-
-    def transcribe(self, audio: np.ndarray) -> str:
-        start = time.perf_counter()
-        inputs = self.stt_processor(audio, sampling_rate=SAMPLE_RATE, return_tensors="pt")
-        input_features = inputs.input_features.to(self.device, dtype=self.dtype)
-        tokens = self.stt_model.generate(input_features)
-        text = self.stt_processor.batch_decode(tokens, skip_special_tokens=True)[0]
-        print(f"STT: {time.perf_counter() - start:.2f}s")
-        return text.strip()
-
-    def _create_dummy_images(self, batch_size: int = 1) -> tuple[list[torch.Tensor], list[torch.Tensor]]:
-        """Create placeholder images for Pi0.5 when no camera is available."""
-        image_shape = (batch_size, 3, 224, 224)
-        dummy_image = torch.zeros(image_shape, dtype=torch.float32, device=self.device)
-        dummy_mask = torch.ones(batch_size, dtype=torch.bool, device=self.device)
-        return [dummy_image], [dummy_mask]
-
-    def _tokenize_prompt(self, text: str) -> tuple[torch.Tensor, torch.Tensor]:
-        """Tokenize the user prompt for Pi0.5."""
-        prompt = f"User request: {text}\nRobot response:"
-        tokenized = self.tokenizer(
-            [prompt],
-            max_length=200,
-            truncation=True,
-            padding="max_length",
-            return_tensors="pt",
-        )
-        tokens = tokenized["input_ids"].to(self.device)
-        masks = tokenized["attention_mask"].to(self.device, dtype=torch.bool)
-        return tokens, masks
-
-    def generate_response(self, user_text: str) -> str:
-        """Generate robot utterance using Pi0.5's language generation."""
-        start = time.perf_counter()
-
-        images, img_masks = self._create_dummy_images()
-        tokens, masks = self._tokenize_prompt(user_text)
-
-        with torch.no_grad():
-            generated_tokens = self.pi05_model._generate_subtask_tokens(
-                images=images,
-                img_masks=img_masks,
-                tokens=tokens,
-                masks=masks,
-                tokenizer=self.tokenizer,
-                max_length=self.max_response_tokens,
-                device=self.device,
-            )
-
-        # Decode generated tokens
-        valid_tokens = generated_tokens[0][generated_tokens[0] != 0]
-        response = self.tokenizer.decode(valid_tokens, skip_special_tokens=True)
-
-        # Extract utterance if marked with special tokens
-        response = self._extract_utterance(response)
-
-        print(f"Pi0.5: {time.perf_counter() - start:.2f}s")
-        return response.strip()
-
-    def _extract_utterance(self, text: str) -> str:
-        """Extract utterance from between <utterance> tokens if present."""
-        pattern = r"<utterance>(.*?)</utterance>"
-        match = re.search(pattern, text, re.DOTALL)
-        if match:
-            return match.group(1).strip()
-        return text
-
-    def speak(self, text: str):
-        start = time.perf_counter()
-        if self.tts_type == "kokoro":
-            generator = self.tts_pipeline(text, voice=self.tts_voice)
-            audio_chunks = [audio for _, _, audio in generator]
-            if audio_chunks:
-                audio = np.concatenate(audio_chunks)
-                sd.play(audio, 24000)
-                sd.wait()
-        else:
-            subprocess.run(["say", text], check=True)
-        print(f"TTS: {time.perf_counter() - start:.2f}s")
-
-    def run(self):
-        print("=" * 50)
-        print("Pi0.5 Voice Assistant")
-        print("=" * 50)
-        print("• Hold SPACE to record your voice command")
-        print("• Release SPACE when done speaking")
-        print("• Press Ctrl+C to exit")
-        print("=" * 50)
-
-        while True:
-            try:
-                audio = self.wait_for_spacebar()
-
-                if audio is None:
-                    print("(no audio captured)\n")
-                    continue
-
-                user_text = self.transcribe(audio)
-
-                if not user_text:
-                    print("(no speech detected)\n")
-                    continue
-
-                print(f"You: {user_text}")
-
-                response = self.generate_response(user_text)
-                print(f"Robot: {response}\n")
-
-                self.speak(response)
-
-            except KeyboardInterrupt:
-                print("\nGoodbye!")
-                break
-
-
-def main():
-    parser = argparse.ArgumentParser(description="Pi0.5 Voice Assistant")
-    parser.add_argument(
-        "--pretrained_path",
-        type=str,
-        default=None,
-        help="Path to pretrained Pi0.5 model (optional)",
-    )
-    parser.add_argument(
-        "--max_response_tokens",
-        type=int,
-        default=100,
-        help="Maximum tokens in generated response",
-    )
-    parser.add_argument(
-        "--max_record_seconds",
-        type=float,
-        default=30.0,
-        help="Maximum recording duration in seconds",
-    )
-    args = parser.parse_args()
-
-    assistant = Pi05VoiceAssistant(
-        pretrained_path=args.pretrained_path,
-        max_response_tokens=args.max_response_tokens,
-        max_record_seconds=args.max_record_seconds,
-    )
-    assistant.run()
-
-
-if __name__ == "__main__":
-    main()
@@ -1,26 +0,0 @@
-{
-  "repo_id": "local",
-  "vocab_size": 1024,
-  "scale": 10.0,
-  "encoded_dims": "0:15",
-  "encoded_dim_ranges": [
-    [
-      0,
-      15
-    ]
-  ],
-  "total_encoded_dims": 15,
-  "delta_dims": null,
-  "delta_dim_list": null,
-  "use_delta_transform": false,
-  "state_key": "observation.state",
-  "action_horizon": 50,
-  "num_training_chunks": 4900,
-  "compression_stats": {
-    "compression_ratio": 15.85791309863622,
-    "mean_token_length": 47.295,
-    "p99_token_length": 90.0,
-    "min_token_length": 9.0,
-    "max_token_length": 109.0
-  }
-}
@@ -1,158 +0,0 @@
-import logging
-from typing import ClassVar
-
-import numpy as np
-from scipy.fft import dct
-from scipy.fft import idct
-from tokenizers import ByteLevelBPETokenizer
-from tokenizers.trainers import BpeTrainer
-from transformers import PreTrainedTokenizerFast
-from transformers.processing_utils import ProcessorMixin
-
-
-class UniversalActionProcessor(ProcessorMixin):
-    attributes: ClassVar[list[str]] = ["bpe_tokenizer"]
-    bpe_tokenizer_class: str = "AutoTokenizer"
-
-    def __init__(
-        self,
-        bpe_tokenizer: PreTrainedTokenizerFast,
-        scale: float = 10,
-        vocab_size: int = 1024,
-        min_token: int = 0,
-        *,
-        action_dim: int | None = None,
-        time_horizon: int | None = None,
-    ):
-        self.scale = scale
-        self.vocab_size = vocab_size
-        self.min_token = min_token
-
-        # Action horizon and dimension needed during decoding. These can be specified
-        # in three ways (in order of priority):
-        # 1. passed in as kwargs to decode()
-        # 2. in the constructor
-        # 3. cached from the last time decode() was called
-        self.time_horizon = time_horizon
-        self.action_dim = action_dim
-        self.called_time_horizon = time_horizon
-        self.called_action_dim = action_dim
-
-        super().__init__(bpe_tokenizer)
-
-    def __call__(self, action_chunk: np.array) -> np.array:
-        assert action_chunk.ndim <= 3, "Only 3 dimensions supported: [batch, timesteps, action_dim]"
-        if action_chunk.ndim == 2:
-            action_chunk = action_chunk[None, ...]
-
-        # Cache the time horizon and action dimension for decoding
-        self.called_time_horizon = action_chunk.shape[-2]
-        self.called_action_dim = action_chunk.shape[-1]
-
-        dct_coeff = dct(action_chunk, axis=1, norm="ortho")
-        dct_coeff = np.around(dct_coeff * self.scale)
-        tokens = []
-        for elem in dct_coeff:
-            token_str = "".join(map(chr, np.maximum(elem.flatten() - self.min_token, 0).astype(int)))
-            tokens.append(self.bpe_tokenizer(token_str)["input_ids"])
-        return tokens
-
-    def decode(
-        self,
-        tokens: list[list[int]],
-        *,
-        time_horizon: int | None = None,
-        action_dim: int | None = None,
-    ) -> np.array:
-        self.time_horizon = time_horizon or self.time_horizon or self.called_time_horizon
-        self.action_dim = action_dim or self.action_dim or self.called_action_dim
-
-        # Cache the time horizon and action dimension for the next call
-        self.called_time_horizon = self.time_horizon
-        self.called_action_dim = self.action_dim
-
-        assert (
-            self.time_horizon is not None and self.action_dim is not None
-        ), "Tokenizer not initialized, call encode() once or pass in time_horizon and action_dim."
-
-        decoded_actions = []
-        for token in tokens:
-            try:
-                decoded_tokens = self.bpe_tokenizer.decode(token)
-                decoded_dct_coeff = np.array(list(map(ord, decoded_tokens))) + self.min_token
-                decoded_dct_coeff = decoded_dct_coeff.reshape(-1, self.action_dim)
-                assert (
-                    decoded_dct_coeff.shape
-                    == (
-                        self.time_horizon,
-                        self.action_dim,
-                    )
-                ), f"Decoded DCT coefficients have shape {decoded_dct_coeff.shape}, expected ({self.time_horizon}, {self.action_dim})"
-            except Exception as e:
-                print(f"Error decoding tokens: {e}")
-                print(f"Tokens: {token}")
-                decoded_dct_coeff = np.zeros((self.time_horizon, self.action_dim))
-            decoded_actions.append(idct(decoded_dct_coeff / self.scale, axis=0, norm="ortho"))
-        return np.stack(decoded_actions)
-
-    @classmethod
-    def fit(
-        cls,
-        action_data: list[np.array],
-        scale: float = 10,
-        vocab_size: int = 1024,
-        *,
-        time_horizon: int | None = None,
-        action_dim: int | None = None,
-    ) -> "UniversalActionProcessor":
-        # Run DCT over all inputs
-        dct_tokens = [dct(a, axis=0, norm="ortho").flatten() for a in action_data]
-
-        # Quantize and find min token
-        max_token = int(np.around(np.concatenate(dct_tokens) * scale).max())
-        min_token = int(np.around(np.concatenate(dct_tokens) * scale).min())
-        min_vocab_size = max_token - min_token
-
-        assert (
-            min_vocab_size <= vocab_size
-        ), f"Vocab size {vocab_size} is too small for the range of tokens {min_vocab_size}"
-        if min_vocab_size + 100 > vocab_size:
-            logging.warning(
-                f"Initial alphabet size {min_vocab_size} is almost as large as the vocab"
-                f"size {vocab_size}, consider increasing vocab size"
-            )
-
-        # Make token iterator for BPE training
-        def _token_iter():
-            for tokens in dct_tokens:
-                rounded_tokens = np.around(tokens * scale) - min_token
-                rounded_tokens = rounded_tokens.astype(int)
-                string = "".join(map(chr, rounded_tokens))
-                yield string
-
-        # Train BPE tokenizer
-        bpe = ByteLevelBPETokenizer()
-
-        # Set up the entire range of possible tokens as the initial alphabet
-        alphabet = [chr(i) for i in range(max_token - min_token + 1)]
-        trainer = BpeTrainer(
-            vocab_size=vocab_size,
-            min_frequency=2,
-            show_progress=True,
-            special_tokens=[],
-            initial_alphabet=alphabet,
-            max_token_length=10000,
-        )
-
-        # Train the inner tokenizer (don't use ByteLevelBPETokenizer.train_from_iterator()
-        # because it doesn't support custom alphabets)
-        bpe._tokenizer.train_from_iterator(_token_iter(), trainer=trainer)
-
-        return cls(
-            PreTrainedTokenizerFast(tokenizer_object=bpe, clean_up_tokenization_spaces=False),
-            scale=scale,
-            vocab_size=vocab_size,
-            min_token=min_token,
-            time_horizon=time_horizon,
-            action_dim=action_dim,
-        )
@@ -1,11 +0,0 @@
-{
-  "action_dim": 15,
-  "auto_map": {
-    "AutoProcessor": "processing_action_tokenizer.UniversalActionProcessor"
-  },
-  "min_token": -71,
-  "processor_class": "UniversalActionProcessor",
-  "scale": 10.0,
-  "time_horizon": 50,
-  "vocab_size": 1024
-}
@@ -1 +0,0 @@
-{}
@@ -1,11 +0,0 @@
-{
-  "added_tokens_decoder": {},
-  "auto_map": {
-    "AutoProcessor": "processing_action_tokenizer.UniversalActionProcessor"
-  },
-  "clean_up_tokenization_spaces": false,
-  "extra_special_tokens": {},
-  "model_max_length": 1000000000000000019884624838656,
-  "processor_class": "UniversalActionProcessor",
-  "tokenizer_class": "PreTrainedTokenizerFast"
-}
@@ -96,7 +96,7 @@ dependencies = [
 # Common
 pygame-dep = ["pygame>=2.5.1,<2.7.0"]
 placo-dep = ["placo>=0.9.6,<0.10.0"]
-transformers-dep = ["transformers>=4.53.0,<5.0.0"]
+transformers-dep = ["transformers>=4.57.1,<5.0.0"]
 grpcio-dep = ["grpcio==1.73.1", "protobuf==6.31.0"] # TODO: Bumb dependency (compatible with wandb)

 # Motors
@@ -120,6 +120,13 @@ intelrealsense = [
 phone = ["hebi-py>=2.8.0,<2.12.0", "teleop>=0.1.0,<0.2.0", "fastapi<1.0"]

 # Policies
+wallx = [
+    "transformers==4.49.0",
+    "peft==0.17.1",
+    "scipy==1.15.3",
+    "torchdiffeq==0.2.5",
+    "qwen_vl_utils==0.0.11"
+]
 pi = ["transformers @ git+https://github.com/huggingface/transformers.git@fix/lerobot_openpi"]
 smolvla = ["lerobot[transformers-dep]", "num2words>=0.5.14,<0.6.0", "accelerate>=1.7.0,<2.0.0", "safetensors>=0.4.3,<1.0.0"]
 groot = [
@@ -133,14 +140,16 @@ groot = [
    "ninja>=1.11.1,<2.0.0",
    "flash-attn>=2.5.9,<3.0.0 ; sys_platform != 'darwin'"
 ]
+sarm = ["lerobot[transformers-dep]", "faker>=33.0.0,<35.0.0", "matplotlib>=3.10.3,<4.0.0", "qwen-vl-utils>=0.0.14"]
 xvla = ["lerobot[transformers-dep]"]
 hilserl = ["lerobot[transformers-dep]", "gym-hil>=0.1.13,<0.2.0", "lerobot[grpcio-dep]", "lerobot[placo-dep]"]

 # Features
 async = ["lerobot[grpcio-dep]", "matplotlib>=3.10.3,<4.0.0"]
+peft = ["lerobot[transformers-dep]", "peft>=0.18.0"]

 # Development
-dev = ["pre-commit>=3.7.0,<5.0.0", "debugpy>=1.8.1,<1.9.0", "lerobot[grpcio-dep]", "grpcio-tools==1.73.1"]
+dev = ["pre-commit>=3.7.0,<5.0.0", "debugpy>=1.8.1,<1.9.0", "lerobot[grpcio-dep]", "grpcio-tools==1.73.1", "mypy>=1.19.1"]
 test = ["pytest>=8.1.0,<9.0.0", "pytest-timeout>=2.4.0,<3.0.0", "pytest-cov>=5.0.0,<8.0.0", "mock-serial>=0.0.1,<0.1.0 ; sys_platform != 'win32'"]
 video_benchmark = ["scikit-image>=0.23.2,<0.26.0", "pandas>=2.2.2,<2.4.0"]

@@ -159,6 +168,7 @@ all = [
    "lerobot[reachy2]",
    "lerobot[kinematics]",
    "lerobot[intelrealsense]",
+    # "lerobot[wallx]",
    "lerobot[pi]",
    "lerobot[smolvla]",
    # "lerobot[groot]", TODO(Steven): Gr00t requires specific installation instructions for flash-attn
@@ -173,6 +183,8 @@ all = [
    "lerobot[phone]",
    "lerobot[libero]",
    "lerobot[metaworld]",
+    "lerobot[sarm]",
+    "lerobot[peft]",
 ]

 [project.scripts]
@@ -227,6 +239,7 @@ ignore = [

 [tool.ruff.lint.per-file-ignores]
 "__init__.py" = ["F401", "F403"]
+"src/lerobot/policies/wall_x/**" = ["N801", "N812", "SIM102", "SIM108", "SIM210", "SIM211", "B006", "B007", "SIM118"] # Supprese these as they are coming from original Qwen2_5_vl code TODO(pepijn): refactor original

 [tool.ruff.lint.isort]
 combine-as-imports = true
@@ -263,6 +276,7 @@ default.extend-ignore-identifiers-re = [
    "ein",
    "thw",
    "inpt",
+    "ROBOTIS",
 ]

 # TODO: Uncomment when ready to use
@@ -317,9 +331,9 @@ disallow_untyped_defs = true
 disallow_incomplete_defs = true
 check_untyped_defs = true

-# [[tool.mypy.overrides]]
-# module = "lerobot.optim.*"
-# ignore_errors = false
+[[tool.mypy.overrides]]
+module = "lerobot.optim.*"
+ignore_errors = false

 [[tool.mypy.overrides]]
 module = "lerobot.model.*"
@@ -369,3 +383,40 @@ ignore_errors = false
 # [[tool.mypy.overrides]]
 # module = "lerobot.scripts.*"
 # ignore_errors = false
+
+[tool.uv]
+# wallx requires transformers==4.49.0 which conflicts with other extras that need >=4.53.0
+conflicts = [
+    [
+        { extra = "wallx" },
+        { extra = "transformers-dep" },
+    ],
+    [
+        { extra = "wallx" },
+        { extra = "pi" },
+    ],
+    [
+        { extra = "wallx" },
+        { extra = "smolvla" },
+    ],
+    [
+        { extra = "wallx" },
+        { extra = "groot" },
+    ],
+    [
+        { extra = "wallx" },
+        { extra = "xvla" },
+    ],
+    [
+        { extra = "wallx" },
+        { extra = "hilserl" },
+    ],
+    [
+        { extra = "wallx" },
+        { extra = "libero" },
+    ],
+    [
+        { extra = "wallx" },
+        { extra = "all" },
+    ],
+]
@@ -26,4 +26,4 @@ DEFAULT_OBS_QUEUE_TIMEOUT = 2
 SUPPORTED_POLICIES = ["act", "smolvla", "diffusion", "tdmpc", "vqbet", "pi0", "pi05"]

 # TODO: Add all other robots
-SUPPORTED_ROBOTS = ["so100_follower", "so101_follower", "bi_so100_follower"]
+SUPPORTED_ROBOTS = ["so100_follower", "so101_follower", "bi_so100_follower", "omx_follower"]
@@ -54,6 +54,7 @@ from lerobot.robots import (  # noqa: F401
    bi_so100_follower,
    koch_follower,
    make_robot_from_config,
+    omx_follower,
    so100_follower,
    so101_follower,
 )
@@ -67,3 +67,31 @@ class EvalConfig:
                f"to increase the number of episodes to match the batch size (e.g. `eval.n_episodes={self.batch_size}`), "
                f"or lower the batch size (e.g. `eval.batch_size={self.n_episodes}`)."
            )
+
+
+@dataclass
+class PeftConfig:
+    # PEFT offers many fine-tuning methods, layer adapters being the most common and currently also the most
+    # effective methods so we'll focus on those in this high-level config interface.
+
+    # Either a string (module name suffix or 'all-linear'), a list of module name suffixes or a regular expression
+    # describing module names to target with the configured PEFT method. Some policies have a default value for this
+    # so that you don't *have* to choose which layers to adapt but it might still be worthwhile depending on your case.
+    target_modules: list[str] | str | None = None
+
+    # Names/suffixes of modules to fully fine-tune and store alongside adapter weights. Useful for layers that are
+    # not part of a pre-trained model (e.g., action state projections). Depending on the policy this defaults to layers
+    # that are newly created in pre-trained policies. If you're fine-tuning an already trained policy you might want
+    # to set this to `[]`. Corresponds to PEFT's `modules_to_save`.
+    full_training_modules: list[str] | None = None
+
+    # The PEFT (adapter) method to apply to the policy. Needs to be a valid PEFT type.
+    method_type: str = "LORA"
+
+    # Adapter initialization method. Look at the specific PEFT adapter documentation for defaults.
+    init_type: str | None = None
+
+    # We expect that all PEFT adapters are in some way doing rank-decomposition therefore this parameter specifies
+    # the rank used for the adapter. In general a higher rank means more trainable parameters and closer to full
+    # fine-tuning.
+    r: int = 16
@@ -55,14 +55,18 @@ class PreTrainedConfig(draccus.ChoiceRegistry, HubMixin, abc.ABC):  # type: igno

    n_obs_steps: int = 1

-    input_features: dict[str, PolicyFeature] = field(default_factory=dict)
-    output_features: dict[str, PolicyFeature] = field(default_factory=dict)
+    # `input_features` can be set to None/null in order to infer those values from the dataset.
+    input_features: dict[str, PolicyFeature] | None = field(default_factory=dict)
+    output_features: dict[str, PolicyFeature] | None = field(default_factory=dict)

    device: str | None = None  # e.g. "cuda", "cuda:0", "cpu", or "mps"
    # `use_amp` determines whether to use Automatic Mixed Precision (AMP) for training and evaluation. With AMP,
    # automatic gradient scaling is used.
    use_amp: bool = False

+    # Whether the policy employed PEFT for training.
+    use_peft: bool = False
+
    push_to_hub: bool = True  # type: ignore[assignment] # TODO: use a different name to avoid override
    repo_id: str | None = None

@@ -125,6 +129,8 @@ class PreTrainedConfig(draccus.ChoiceRegistry, HubMixin, abc.ABC):  # type: igno

    @property
    def robot_state_feature(self) -> PolicyFeature | None:
+        if self.input_features is None:
+            return None
        for ft_name, ft in self.input_features.items():
            if ft.type is FeatureType.STATE and ft_name == OBS_STATE:
                return ft
@@ -132,6 +138,8 @@ class PreTrainedConfig(draccus.ChoiceRegistry, HubMixin, abc.ABC):  # type: igno

    @property
    def env_state_feature(self) -> PolicyFeature | None:
+        if self.input_features is None:
+            return None
        for _, ft in self.input_features.items():
            if ft.type is FeatureType.ENV:
                return ft
@@ -139,10 +147,14 @@ class PreTrainedConfig(draccus.ChoiceRegistry, HubMixin, abc.ABC):  # type: igno

    @property
    def image_features(self) -> dict[str, PolicyFeature]:
+        if self.input_features is None:
+            return {}
        return {key: ft for key, ft in self.input_features.items() if ft.type is FeatureType.VISUAL}

    @property
    def action_feature(self) -> PolicyFeature | None:
+        if self.output_features is None:
+            return None
        for ft_name, ft in self.output_features.items():
            if ft.type is FeatureType.ACTION and ft_name == ACTION:
                return ft
@@ -24,7 +24,7 @@ from huggingface_hub.errors import HfHubHTTPError

 from lerobot import envs
 from lerobot.configs import parser
-from lerobot.configs.default import DatasetConfig, EvalConfig, WandBConfig
+from lerobot.configs.default import DatasetConfig, EvalConfig, PeftConfig, WandBConfig
 from lerobot.configs.policies import PreTrainedConfig
 from lerobot.optim import OptimizerConfig
 from lerobot.optim.schedulers import LRSchedulerConfig
@@ -56,6 +56,7 @@ class TrainPipelineConfig(HubMixin):
    steps: int = 100_000
    eval_freq: int = 20_000
    log_freq: int = 200
+    tolerance_s: float = 1e-4
    save_checkpoint: bool = True
    # Checkpoint is saved every `save_freq` training iterations and after the last training step.
    save_freq: int = 20_000
@@ -64,9 +65,18 @@ class TrainPipelineConfig(HubMixin):
    scheduler: LRSchedulerConfig | None = None
    eval: EvalConfig = field(default_factory=EvalConfig)
    wandb: WandBConfig = field(default_factory=WandBConfig)
-    checkpoint_path: Path | None = field(init=False, default=None)
+    peft: PeftConfig | None = None
+
+    # RA-BC (Reward-Aligned Behavior Cloning) parameters
+    use_rabc: bool = False  # Enable reward-weighted training
+    rabc_progress_path: str | None = None  # Path to precomputed SARM progress parquet file
+    rabc_kappa: float = 0.01  # Hard threshold for high-quality samples
+    rabc_epsilon: float = 1e-6  # Small constant for numerical stability
+    rabc_head_mode: str | None = "sparse"  # For dual-head models: "sparse" or "dense"
+
    # Rename map for the observation to override the image and state keys
    rename_map: dict[str, str] = field(default_factory=dict)
+    checkpoint_path: Path | None = field(init=False, default=None)

    def validate(self) -> None:
        # HACK: We parse again the cli args here to get the pretrained paths if there was some.
@@ -130,6 +140,14 @@ class TrainPipelineConfig(HubMixin):
                "'policy.repo_id' argument missing. Please specify it to push the model to the hub."
            )

+        if self.use_rabc and not self.rabc_progress_path:
+            # Auto-detect from dataset path
+            repo_id = self.dataset.repo_id
+            if self.dataset.root:
+                self.rabc_progress_path = str(Path(self.dataset.root) / "sarm_progress.parquet")
+            else:
+                self.rabc_progress_path = f"hf://datasets/{repo_id}/sarm_progress.parquet"
+
    @classmethod
    def __get_path_fields__(cls) -> list[str]:
        """This enables the parser to load config from the policy using `--policy.path=local/dir`"""
@@ -0,0 +1,13 @@
+# Copyright 2025 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
@@ -0,0 +1,13 @@
+# Copyright 2025 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
@@ -59,6 +59,7 @@ python examples/dataset_annotation/subtask_annotation.py \
 import argparse
 import json
 import multiprocessing as mp
+import random
 import re
 import subprocess
 import tempfile
@@ -66,21 +67,100 @@ import textwrap
 import time
 from concurrent.futures import ProcessPoolExecutor, as_completed
 from pathlib import Path
+from typing import Any

 import cv2
+import numpy as np
 import pandas as pd
 import torch
-from qwen_vl_utils import process_vision_info
-from rich.console import Console
+from pydantic import BaseModel, Field
 from transformers import AutoProcessor, Qwen3VLMoeForConditionalGeneration

 from lerobot.datasets.lerobot_dataset import LeRobotDataset
-from lerobot.policies.sarm.sarm_utils import (
-    Subtask,
-    SubtaskAnnotation,
-    Timestamp,
-    compute_temporal_proportions,
-)
+
+
+# Pydantic Models for SARM Subtask Annotation
+class Timestamp(BaseModel):
+    """Timestamp in MM:SS or SS format"""
+
+    start: str = Field(description="Start timestamp (MM:SS or just seconds)")
+    end: str = Field(description="End timestamp (MM:SS or just seconds)")
+
+
+class Subtask(BaseModel):
+    """Individual subtask/stage - must use EXACT names from provided list"""
+
+    name: str = Field(description="Subtask name - MUST match one from the predefined list exactly")
+    timestamps: Timestamp
+
+
+class SubtaskAnnotation(BaseModel):
+    """Complete annotation for a robot manipulation episode"""
+
+    subtasks: list[Subtask] = Field(description="List of all subtasks in temporal order")
+
+
+def compute_temporal_proportions(
+    annotations: dict[int, Any], fps: int = 30, subtask_order: list[str] | None = None
+) -> dict[str, float]:
+    """
+    Compute dataset-level temporal proportions (priors) for each subtask.
+
+    Implements SARM Paper Formula (1): ᾱ_k = (1/M) × Σ_i (L_{i,k} / T_i)
+
+    Args:
+        annotations: Dict mapping episode index to SubtaskAnnotation object.
+        fps: Frames per second (unused, kept for API compatibility)
+        subtask_order: Optional list defining the output order of subtasks.
+
+    Returns:
+        Dict mapping subtask name to its temporal proportion (ᾱ_k), ordered by subtask_order if provided.
+    """
+    subtask_proportions: dict[str, list[float]] = {}
+
+    for annotation in annotations.values():
+        total_duration = 0
+        durations: dict[str, int] = {}
+
+        for subtask in annotation.subtasks:
+            start_parts = subtask.timestamps.start.split(":")
+            end_parts = subtask.timestamps.end.split(":")
+
+            start_seconds = (
+                int(start_parts[0]) * 60 + int(start_parts[1])
+                if len(start_parts) == 2
+                else int(start_parts[0])
+            )
+            end_seconds = (
+                int(end_parts[0]) * 60 + int(end_parts[1]) if len(end_parts) == 2 else int(end_parts[0])
+            )
+
+            duration = end_seconds - start_seconds
+            durations[subtask.name] = duration
+            total_duration += duration
+
+        if total_duration > 0:
+            for name, duration in durations.items():
+                if name not in subtask_proportions:
+                    subtask_proportions[name] = []
+                subtask_proportions[name].append(duration / total_duration)
+
+    if not subtask_proportions:
+        return {}
+
+    avg_proportions = {name: sum(props) / len(props) for name, props in subtask_proportions.items()}
+
+    total = sum(avg_proportions.values())
+    if total > 0:
+        avg_proportions = {name: prop / total for name, prop in avg_proportions.items()}
+
+    # Reorder according to subtask_order if provided
+    if subtask_order:
+        avg_proportions = {
+            name: avg_proportions.get(name, 0.0) for name in subtask_order if name in avg_proportions
+        }
+
+    return avg_proportions


 def create_sarm_prompt(subtask_list: list[str]) -> str:
@@ -177,8 +257,8 @@ class VideoAnnotator:
        model_name: str = "Qwen/Qwen3-VL-30B-A3B-Instruct",
        device: str = "cuda",
        torch_dtype: torch.dtype = torch.bfloat16,
-        model: "Qwen3VLMoeForConditionalGeneration | None" = None,
-        processor: "AutoProcessor | None" = None,
+        model: Qwen3VLMoeForConditionalGeneration | None = None,  # noqa: F821
+        processor: AutoProcessor | None = None,  # noqa: F821
    ):
        """
        Initialize the video annotator with local model.
@@ -193,16 +273,17 @@ class VideoAnnotator:
        """
        self.subtask_list = subtask_list
        self.prompt = create_sarm_prompt(subtask_list)
-        self.console = Console()
        self.device = device

        # Use provided model/processor or load new ones
        if model is not None and processor is not None:
            self.model = model
            self.processor = processor
-            self.console.print(f"[green]✓ Using shared model on {device}[/green]")
+            print(f"Using shared model on {device}")
        else:
-            self.console.print(f"[cyan]Loading model: {model_name}...[/cyan]")
+            from transformers import AutoProcessor, Qwen3VLMoeForConditionalGeneration
+
+            print(f"Loading model: {model_name}...")

            self.model = Qwen3VLMoeForConditionalGeneration.from_pretrained(
                model_name, torch_dtype=torch_dtype, device_map=device, trust_remote_code=True
@@ -210,7 +291,7 @@ class VideoAnnotator:

            self.processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)

-            self.console.print(f"[green]✓ Model loaded successfully on {device}[/green]")
+            print(f"Model loaded successfully on {device}")

    def extract_episode_segment(
        self, file_path: Path, start_timestamp: float, end_timestamp: float, target_fps: int = 1
@@ -229,25 +310,22 @@ class VideoAnnotator:
            Path to extracted video file
        """
        # Create temporary file for extracted video
-        tmp_file = tempfile.NamedTemporaryFile(suffix=".mp4", delete=False)
-        tmp_path = Path(tmp_file.name)
-        tmp_file.close()
+        with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as tmp_file:
+            tmp_path = Path(tmp_file.name)

        try:
            # Check if ffmpeg is available
-            subprocess.run(
+            subprocess.run(  # nosec B607
                ["ffmpeg", "-version"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True
            )
-        except (subprocess.CalledProcessError, FileNotFoundError):
-            raise RuntimeError("ffmpeg not found, cannot extract episode segment") from e
+        except (subprocess.CalledProcessError, FileNotFoundError) as err:
+            raise RuntimeError("ffmpeg not found, cannot extract episode segment") from err

        try:
            # Calculate duration
            duration = end_timestamp - start_timestamp

-            self.console.print(
-                f"[cyan]Extracting episode: {start_timestamp:.1f}s-{end_timestamp:.1f}s ({duration:.1f}s)[/cyan]"
-            )
+            print(f"Extracting episode: {start_timestamp:.1f}s-{end_timestamp:.1f}s ({duration:.1f}s)")

            # Use ffmpeg to extract segment with minimal quality loss
            cmd = [
@@ -275,7 +353,7 @@ class VideoAnnotator:

            # Verify the output file was created and is not empty
            if not tmp_path.exists() or tmp_path.stat().st_size == 0:
-                self.console.print("[red]✗ Video extraction failed (0 bytes) - skipping episode[/red]")
+                print("Video extraction failed (0 bytes) - skipping episode")
                if tmp_path.exists():
                    tmp_path.unlink()
                raise RuntimeError("FFmpeg produced empty video file")
@@ -285,13 +363,11 @@ class VideoAnnotator:

            # Fail if file is too small (< 100KB likely means extraction failed)
            if file_size_mb < 0.1:
-                self.console.print(
-                    f"[red]✗ Extracted video too small ({file_size_mb:.2f}MB) - skipping episode[/red]"
-                )
+                print(f"Extracted video too small ({file_size_mb:.2f}MB) - skipping episode")
                tmp_path.unlink()
                raise RuntimeError(f"Video extraction produced invalid file ({file_size_mb:.2f}MB)")

-            self.console.print(f"[green]✓ Extracted: {file_size_mb:.1f}MB ({target_fps} FPS)[/green]")
+            print(f"Extracted: {file_size_mb:.1f}MB ({target_fps} FPS)")

            return tmp_path

@@ -307,6 +383,8 @@ class VideoAnnotator:
        max_retries: int = 3,
    ) -> SubtaskAnnotation:
        """Annotate a video segment using local GPU."""
+        from qwen_vl_utils import process_vision_info
+
        file_path = Path(file_path)

        if end_timestamp is None:
@@ -355,7 +433,7 @@ class VideoAnnotator:
                        )

                    response = self.processor.batch_decode(
-                        [out[len(inp) :] for inp, out in zip(inputs.input_ids, generated_ids)],
+                        [out[len(inp) :] for inp, out in zip(inputs.input_ids, generated_ids, strict=True)],
                        skip_special_tokens=True,
                    )[0].strip()

@@ -371,7 +449,7 @@ class VideoAnnotator:
                        match = re.search(r"\{.*\}", response, re.DOTALL)
                        if match:
                            return SubtaskAnnotation.model_validate(json.loads(match.group()))
-                        raise ValueError("No JSON found")
+                        raise ValueError("No JSON found") from None
                except Exception as e:
                    if attempt == max_retries - 1:
                        raise RuntimeError(f"Failed after {max_retries} attempts") from e
@@ -381,16 +459,12 @@ class VideoAnnotator:
                extracted_path.unlink()


-def display_annotation(
-    annotation: SubtaskAnnotation, console: Console, episode_idx: int, fps: int, prefix: str = ""
-):
+def display_annotation(annotation: SubtaskAnnotation, episode_idx: int, fps: int, prefix: str = ""):
    """Display annotation summary."""
    subtask_summary = ", ".join(
        f"{s.name}({s.timestamps.start}-{s.timestamps.end})" for s in annotation.subtasks
    )
-    console.print(
-        f"[green]Episode {episode_idx} {prefix}: {len(annotation.subtasks)} subtasks - {subtask_summary}[/green]"
-    )
+    print(f"Episode {episode_idx} {prefix}: {len(annotation.subtasks)} subtasks - {subtask_summary}")


 def timestamp_to_seconds(timestamp: str) -> float:
@@ -402,6 +476,272 @@ def timestamp_to_seconds(timestamp: str) -> float:
        return int(parts[0])


+def extract_frame(video_path: Path, timestamp: float) -> np.ndarray | None:
+    """Extract a single frame from video at given timestamp."""
+    cap = cv2.VideoCapture(str(video_path))
+    if not cap.isOpened():
+        return None
+    cap.set(cv2.CAP_PROP_POS_MSEC, timestamp * 1000)
+    ret, frame = cap.read()
+    cap.release()
+    return cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) if ret else None
+
+
+def draw_timeline(ax, subtasks, total_duration, colors):
+    """Draw a timeline with color-coded subtask segments."""
+    import matplotlib.patches as mpatches
+
+    bar_height, bar_y = 0.6, 0.5
+
+    for i, subtask in enumerate(subtasks):
+        start = timestamp_to_seconds(subtask.timestamps.start)
+        end = timestamp_to_seconds(subtask.timestamps.end)
+        color = colors[i % len(colors)]
+
+        rect = mpatches.FancyBboxPatch(
+            (start, bar_y - bar_height / 2),
+            end - start,
+            bar_height,
+            boxstyle="round,pad=0.02,rounding_size=0.1",
+            facecolor=color,
+            edgecolor="white",
+            linewidth=1.5,
+            alpha=0.85,
+        )
+        ax.add_patch(rect)
+
+        # Add label if segment is wide enough
+        duration = end - start
+        if duration > total_duration * 0.06:
+            ax.text(
+                (start + end) / 2,
+                bar_y,
+                subtask.name,
+                ha="center",
+                va="center",
+                fontsize=8,
+                fontweight="bold",
+                color="white",
+                rotation=0 if duration > total_duration * 0.12 else 45,
+            )
+
+        if i > 0:
+            ax.axvline(x=start, ymin=0.1, ymax=0.9, color="white", linestyle="--", linewidth=1.5, alpha=0.7)
+
+    ax.axvline(x=0, ymin=0.1, ymax=0.9, color="#00ff00", linestyle="-", linewidth=2, alpha=0.9)
+    if subtasks:
+        ax.axvline(
+            x=timestamp_to_seconds(subtasks[-1].timestamps.end),
+            ymin=0.1,
+            ymax=0.9,
+            color="white",
+            linestyle="--",
+            linewidth=1.5,
+            alpha=0.7,
+        )
+
+    ax.set_xlim(-total_duration * 0.02, total_duration * 1.02)
+    ax.set_ylim(-0.1, 1.1)
+    ax.set_xlabel("Time (seconds)", fontsize=10, color="white", labelpad=5)
+    for spine in ["top", "right", "left"]:
+        ax.spines[spine].set_visible(False)
+    ax.spines["bottom"].set_color("#444444")
+    ax.tick_params(axis="x", colors="#888888", labelsize=8)
+    ax.tick_params(axis="y", left=False, labelleft=False)
+
+
+def visualize_episode(
+    ep_idx: int,
+    annotation: SubtaskAnnotation,
+    video_path: Path,
+    video_start: float,
+    video_end: float,
+    output_path: Path,
+    video_key: str,
+    ann_type: str,
+):
+    """Create visualization for a single episode with frames and timeline."""
+    import matplotlib.pyplot as plt
+
+    if annotation is None:
+        print(f"No {ann_type} annotation for episode {ep_idx}")
+        return
+
+    subtasks = annotation.subtasks
+    if not subtasks:
+        print(f"No subtasks for episode {ep_idx}")
+        return
+
+    colors = plt.cm.tab10(np.linspace(0, 1, max(len(subtasks), 10)))
+    total_duration = timestamp_to_seconds(subtasks[-1].timestamps.end)
+
+    # Extract middle frame from each subtask
+    sample_frames, frame_times = [], []
+    for subtask in subtasks:
+        start = timestamp_to_seconds(subtask.timestamps.start)
+        end = timestamp_to_seconds(subtask.timestamps.end)
+        mid = (start + end) / 2
+        frame_times.append(mid)
+        sample_frames.append(extract_frame(video_path, video_start + mid))
+
+    # Create figure
+    fig_width = max(16, len(subtasks) * 2.5)
+    fig = plt.figure(figsize=(fig_width, 10))
+    fig.patch.set_facecolor("#1a1a2e")
+
+    gs = fig.add_gridspec(
+        2,
+        max(len(subtasks), 1),
+        height_ratios=[2, 1],
+        hspace=0.3,
+        wspace=0.1,
+        left=0.05,
+        right=0.95,
+        top=0.88,
+        bottom=0.1,
+    )
+
+    fig.suptitle(
+        f"Episode {ep_idx} - {ann_type.capitalize()} Annotations",
+        fontsize=18,
+        fontweight="bold",
+        color="white",
+        y=0.96,
+    )
+    fig.text(
+        0.5,
+        0.91,
+        f"Camera: {video_key} | Duration: {video_end - video_start:.1f}s | {len(subtasks)} subtasks",
+        ha="center",
+        fontsize=11,
+        color="#888888",
+    )
+
+    # Plot frames
+    for i, (frame, subtask) in enumerate(zip(sample_frames, subtasks, strict=True)):
+        ax = fig.add_subplot(gs[0, i])
+        ax.set_facecolor("#16213e")
+        if frame is not None:
+            ax.imshow(frame)
+        else:
+            ax.text(
+                0.5, 0.5, "N/A", ha="center", va="center", fontsize=12, color="white", transform=ax.transAxes
+            )
+        ax.set_title(subtask.name, fontsize=10, fontweight="bold", color=colors[i % len(colors)], pad=8)
+        ax.axis("off")
+        ax.text(
+            0.5,
+            -0.08,
+            f"t={frame_times[i]:.1f}s",
+            ha="center",
+            fontsize=9,
+            color="#888888",
+            transform=ax.transAxes,
+        )
+
+    # Plot timeline
+    ax_timeline = fig.add_subplot(gs[1, :])
+    ax_timeline.set_facecolor("#16213e")
+    draw_timeline(ax_timeline, subtasks, total_duration, colors)
+
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    plt.savefig(output_path, dpi=150, facecolor=fig.get_facecolor(), edgecolor="none", bbox_inches="tight")
+    plt.close()
+    print(f"Saved: {output_path}")
+
+
+def visualize_annotations(
+    dataset: LeRobotDataset,
+    sparse_annotations: dict[int, SubtaskAnnotation],
+    dense_annotations: dict[int, SubtaskAnnotation] | None,
+    video_key: str,
+    output_dir: Path,
+    num_episodes: int = 5,
+    annotation_type: str = "sparse",
+    episode_indices: list[int] | None = None,
+):
+    """
+    Visualize subtask annotations for a set of episodes.
+
+    Args:
+        dataset: LeRobotDataset instance
+        sparse_annotations: Dict mapping episode index to sparse annotations
+        dense_annotations: Dict mapping episode index to dense annotations (or None)
+        video_key: Camera/video key to use
+        output_dir: Directory to save visualization images
+        num_episodes: Number of episodes to visualize (ignored if episode_indices provided)
+        annotation_type: "sparse", "dense", or "both"
+        episode_indices: Specific episode indices to visualize (optional)
+    """
+    # Determine available episodes based on annotation type
+    if annotation_type == "sparse":
+        available = set(sparse_annotations.keys())
+    elif annotation_type == "dense":
+        available = set(dense_annotations.keys()) if dense_annotations else set()
+    else:  # both
+        sparse_set = set(sparse_annotations.keys())
+        dense_set = set(dense_annotations.keys()) if dense_annotations else set()
+        available = sparse_set | dense_set
+
+    if not available:
+        print("Error: No annotations found to visualize.")
+        return
+
+    # Select episodes to visualize
+    if episode_indices:
+        episodes = sorted([e for e in episode_indices if e in available])
+        missing = set(episode_indices) - available
+        if missing:
+            print(f"Episodes not found in annotations: {sorted(missing)}")
+    else:
+        episodes = sorted(random.sample(list(available), min(num_episodes, len(available))))
+    print(f"Visualizing {len(episodes)} episodes: {episodes}")
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    # Generate visualizations
+    for i, ep_idx in enumerate(episodes, 1):
+        print(f"Processing episode {ep_idx} ({i}/{len(episodes)})")
+        video_path = dataset.root / dataset.meta.get_video_file_path(ep_idx, video_key)
+        if not video_path.exists():
+            print(f"Video not found: {video_path}")
+            continue
+
+        video_start = float(dataset.meta.episodes[f"videos/{video_key}/from_timestamp"][ep_idx])
+        video_end = float(dataset.meta.episodes[f"videos/{video_key}/to_timestamp"][ep_idx])
+
+        if annotation_type == "both":
+            # Visualize both sparse and dense
+            for ann_type, annotations in [("sparse", sparse_annotations), ("dense", dense_annotations)]:
+                if annotations and ep_idx in annotations:
+                    output_path = output_dir / f"episode_{ep_idx:04d}_{ann_type}.png"
+                    visualize_episode(
+                        ep_idx,
+                        annotations.get(ep_idx),
+                        video_path,
+                        video_start,
+                        video_end,
+                        output_path,
+                        video_key,
+                        ann_type,
+                    )
+        else:
+            annotations = sparse_annotations if annotation_type == "sparse" else dense_annotations
+            if annotations and ep_idx in annotations:
+                output_path = output_dir / f"episode_{ep_idx:04d}_{annotation_type}.png"
+                visualize_episode(
+                    ep_idx,
+                    annotations.get(ep_idx),
+                    video_path,
+                    video_start,
+                    video_end,
+                    output_path,
+                    video_key,
+                    annotation_type,
+                )
+
+    print(f"Visualizations saved to: {output_dir.absolute()}")
+
+
 def save_annotations_to_dataset(
    dataset_path: Path, annotations: dict[int, SubtaskAnnotation], fps: int, prefix: str = "sparse"
 ):
@@ -533,7 +873,7 @@ def load_annotations_from_dataset(dataset_path: Path, prefix: str = "sparse") ->
                        end=f"{int(e) // 60:02d}:{int(e) % 60:02d}",
                    ),
                )
-                for n, s, e in zip(names, starts, ends)
+                for n, s, e in zip(names, starts, ends, strict=True)
            ]
        )
    return annotations
@@ -546,7 +886,6 @@ def process_single_episode(
    video_key: str,
    fps: int,
    annotator: VideoAnnotator,
-    console: Console,
 ) -> tuple[int, SubtaskAnnotation | None, str | None]:
    """Process a single episode annotation."""
    try:
@@ -574,7 +913,6 @@ def worker_process_episodes(
 ) -> tuple[dict, dict | None]:
    """Worker for parallel processing across GPUs."""
    device = f"cuda:{gpu_id}"
-    console = Console()
    dataset = LeRobotDataset(repo_id, download_videos=False)

    sparse_annotator = VideoAnnotator(sparse_subtask_list, model_name, device, torch_dtype)
@@ -595,14 +933,14 @@ def worker_process_episodes(

    for ep_idx in episode_indices:
        _, sparse_ann, err = process_single_episode(
-            ep_idx, dataset.root, dataset.meta, video_key, dataset.fps, sparse_annotator, console
+            ep_idx, dataset.root, dataset.meta, video_key, dataset.fps, sparse_annotator
        )
        if sparse_ann:
            sparse_annotations[ep_idx] = sparse_ann

        if dense_annotator:
            _, dense_ann, _ = process_single_episode(
-                ep_idx, dataset.root, dataset.meta, video_key, dataset.fps, dense_annotator, console
+                ep_idx, dataset.root, dataset.meta, video_key, dataset.fps, dense_annotator
            )
            if dense_ann:
                dense_annotations[ep_idx] = dense_ann
@@ -632,25 +970,36 @@ def main():
    parser.add_argument("--dtype", type=str, default="bfloat16", choices=["bfloat16", "float16", "float32"])
    parser.add_argument("--num-workers", type=int, default=1, help="Parallel workers for multi-GPU")
    parser.add_argument("--gpu-ids", type=int, nargs="+", default=None, help="GPU IDs to use")
+    # Visualization options
+    parser.add_argument(
+        "--visualize-only",
+        action="store_true",
+        help="Only visualize existing annotations (no generation)",
+    )
+    parser.add_argument(
+        "--num-visualizations",
+        type=int,
+        default=5,
+        help="Number of episodes to visualize (default: 5)",
+    )
+    parser.add_argument(
+        "--visualize-type",
+        type=str,
+        default="sparse",
+        choices=["sparse", "dense", "both"],
+        help="Type of annotations to visualize (default: sparse)",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default="./subtask_viz",
+        help="Output directory for visualizations (default: ./subtask_viz)",
+    )

    args = parser.parse_args()
-    console = Console()

-    # Validate arguments
-    if args.dense_only and not args.dense_subtasks:
-        return console.print("[red]Error: --dense-only requires --dense-subtasks[/red]")
-    if args.dense_subtasks and not args.sparse_subtasks and not args.dense_only:
-        return console.print("[red]Error: --dense-subtasks requires --sparse-subtasks or --dense-only[/red]")
-
-    sparse_subtask_list = (
-        [s.strip() for s in args.sparse_subtasks.split(",")] if args.sparse_subtasks else None
-    )
-    dense_subtask_list = [s.strip() for s in args.dense_subtasks.split(",")] if args.dense_subtasks else None
-    auto_sparse = sparse_subtask_list is None
-    dense_mode = dense_subtask_list is not None
-    torch_dtype = {"bfloat16": torch.bfloat16, "float16": torch.float16, "float32": torch.float32}[args.dtype]
-
-    console.print(f"[cyan]Loading dataset: {args.repo_id}[/cyan]")
+    # Load dataset first (needed for both annotation and visualization)
+    print(f"Loading dataset: {args.repo_id}")
    dataset = LeRobotDataset(args.repo_id, download_videos=True)
    fps = dataset.fps

@@ -660,7 +1009,44 @@ def main():
    video_key = (
        args.video_key if args.video_key in (dataset.meta.video_keys or []) else dataset.meta.video_keys[0]
    )
-    console.print(f"[cyan]Using camera: {video_key}, FPS: {fps}[/cyan]")
+    print(f"Using camera: {video_key}, FPS: {fps}")
+
+    # Handle visualization-only mode
+    if args.visualize_only:
+        print("Visualization-only mode")
+        sparse_annotations = load_annotations_from_dataset(dataset.root, prefix="sparse")
+        dense_annotations = load_annotations_from_dataset(dataset.root, prefix="dense")
+
+        if not sparse_annotations and not dense_annotations:
+            return print("Error: No annotations found. Run annotation first.")
+
+        print(f"Found {len(sparse_annotations)} sparse, {len(dense_annotations)} dense annotations")
+
+        visualize_annotations(
+            dataset=dataset,
+            sparse_annotations=sparse_annotations,
+            dense_annotations=dense_annotations if dense_annotations else None,
+            video_key=video_key,
+            output_dir=Path(args.output_dir),
+            num_episodes=args.num_visualizations,
+            annotation_type=args.visualize_type,
+            episode_indices=args.episodes,
+        )
+        return
+
+    # Validate arguments for annotation mode
+    if args.dense_only and not args.dense_subtasks:
+        return print("Error: --dense-only requires --dense-subtasks")
+    if args.dense_subtasks and not args.sparse_subtasks and not args.dense_only:
+        return print("Error: --dense-subtasks requires --sparse-subtasks or --dense-only")
+
+    sparse_subtask_list = (
+        [s.strip() for s in args.sparse_subtasks.split(",")] if args.sparse_subtasks else None
+    )
+    dense_subtask_list = [s.strip() for s in args.dense_subtasks.split(",")] if args.dense_subtasks else None
+    auto_sparse = sparse_subtask_list is None
+    dense_mode = dense_subtask_list is not None
+    torch_dtype = {"bfloat16": torch.bfloat16, "float16": torch.float16, "float32": torch.float32}[args.dtype]

    # Determine episodes
    episode_indices = args.episodes or list(range(dataset.meta.total_episodes))
@@ -670,8 +1056,8 @@ def main():
        episode_indices = [ep for ep in episode_indices if ep not in existing_annotations]

    if not episode_indices:
-        return console.print("[green]All episodes already annotated![/green]")
-    console.print(f"[cyan]Annotating {len(episode_indices)} episodes[/cyan]")
+        return print("All episodes already annotated!")
+    print(f"Annotating {len(episode_indices)} episodes")

    # GPU setup
    gpu_ids = args.gpu_ids or list(
@@ -686,7 +1072,7 @@ def main():
    if auto_sparse:
        sparse_annotations.update(generate_auto_sparse_annotations(dataset, episode_indices, video_key))
        save_annotations_to_dataset(dataset.root, sparse_annotations, fps, prefix="sparse")
-        console.print(f"[green]Auto-generated {len(episode_indices)} sparse 'task' annotations[/green]")
+        print(f"Auto-generated {len(episode_indices)} sparse 'task' annotations")

    # VLM annotation (for sparse if not auto, and for dense)
    need_vlm = (not auto_sparse) or dense_mode
@@ -694,7 +1080,7 @@ def main():
    if need_vlm:
        if args.num_workers > 1 and not auto_sparse:
            # Parallel processing
-            console.print(f"[cyan]Parallel processing with {args.num_workers} workers[/cyan]")
+            print(f"Parallel processing with {args.num_workers} workers")
            episodes_per_worker = [[] for _ in range(args.num_workers)]
            for i, ep_idx in enumerate(episode_indices):
                episodes_per_worker[i % args.num_workers].append(ep_idx)
@@ -751,52 +1137,66 @@ def main():
            )

            for i, ep_idx in enumerate(episode_indices):
-                console.print(f"[cyan]Episode {ep_idx} ({i + 1}/{len(episode_indices)})[/cyan]")
+                print(f"Episode {ep_idx} ({i + 1}/{len(episode_indices)})")

                if sparse_annotator:
                    _, sparse_ann, err = process_single_episode(
-                        ep_idx, dataset.root, dataset.meta, video_key, fps, sparse_annotator, console
+                        ep_idx, dataset.root, dataset.meta, video_key, fps, sparse_annotator
                    )
                    if sparse_ann:
                        sparse_annotations[ep_idx] = sparse_ann
                        save_annotations_to_dataset(dataset.root, sparse_annotations, fps, prefix="sparse")
                    elif err:
-                        console.print(f"[red]Sparse failed: {err}[/red]")
+                        print(f"Sparse failed: {err}")

                if dense_annotator:
                    _, dense_ann, err = process_single_episode(
-                        ep_idx, dataset.root, dataset.meta, video_key, fps, dense_annotator, console
+                        ep_idx, dataset.root, dataset.meta, video_key, fps, dense_annotator
                    )
                    if dense_ann:
                        dense_annotations[ep_idx] = dense_ann
                        save_annotations_to_dataset(dataset.root, dense_annotations, fps, prefix="dense")
                    elif err:
-                        console.print(f"[red]Dense failed: {err}[/red]")
+                        print(f"Dense failed: {err}")

    # Save temporal proportions
-    def save_proportions(annotations, prefix, is_auto=False):
-        props: dict[str, float] = {"task": 1.0} if is_auto else compute_temporal_proportions(annotations, fps)
+    def save_proportions(annotations, prefix, subtask_list=None, is_auto=False):
+        props: dict[str, float] = (
+            {"task": 1.0} if is_auto else compute_temporal_proportions(annotations, fps, subtask_list)
+        )
        path = dataset.root / "meta" / f"temporal_proportions_{prefix}.json"
        path.parent.mkdir(parents=True, exist_ok=True)
        with open(path, "w") as f:
            json.dump(props, f, indent=2)
-        console.print(f"[green]Saved {prefix} temporal proportions[/green]")
+        print(f"Saved {prefix} temporal proportions")

-    save_proportions(sparse_annotations, "sparse", auto_sparse)
+    save_proportions(sparse_annotations, "sparse", sparse_subtask_list, auto_sparse)
    if dense_mode and dense_annotations:
-        save_proportions(dense_annotations, "dense")
+        save_proportions(dense_annotations, "dense", dense_subtask_list)

-    console.print(
-        f"\n[bold green]Complete! {len(sparse_annotations)} sparse, {len(dense_annotations or {})} dense annotations[/bold green]"
-    )
+    print(f"\nComplete! {len(sparse_annotations)} sparse, {len(dense_annotations or {})} dense annotations")
+
+    # Visualize annotations after generation
+    if args.num_visualizations > 0:
+        print(f"\nGenerating {args.num_visualizations} visualizations...")
+        visualize_type = "both" if dense_mode else "sparse"
+        visualize_annotations(
+            dataset=dataset,
+            sparse_annotations=sparse_annotations,
+            dense_annotations=dense_annotations,
+            video_key=video_key,
+            output_dir=Path(args.output_dir),
+            num_episodes=args.num_visualizations,
+            annotation_type=visualize_type,
+        )

    if args.push_to_hub:
        try:
            dataset.push_to_hub(push_videos=True)
-            console.print(f"[green]Pushed to {args.output_repo_id or args.repo_id}[/green]")
+            print(f"Pushed to {args.output_repo_id or args.repo_id}")
        except Exception as e:
-            console.print(f"[red]Push failed: {e}[/red]")
+            print(f"Push failed: {e}")


 if __name__ == "__main__":
-    main()
+    main()
@@ -98,6 +98,7 @@ def make_dataset(cfg: TrainPipelineConfig) -> LeRobotDataset | MultiLeRobotDatas
                image_transforms=image_transforms,
                revision=cfg.dataset.revision,
                video_backend=cfg.dataset.video_backend,
+                tolerance_s=cfg.tolerance_s,
            )
        else:
            dataset = StreamingLeRobotDataset(
@@ -108,6 +109,7 @@ def make_dataset(cfg: TrainPipelineConfig) -> LeRobotDataset | MultiLeRobotDatas
                image_transforms=image_transforms,
                revision=cfg.dataset.revision,
                max_num_shards=cfg.num_workers,
+                tolerance_s=cfg.tolerance_s,
            )
    else:
        raise NotImplementedError("The MultiLeRobotDataset isn't supported for now.")
@@ -58,7 +58,6 @@ from lerobot.datasets.utils import (
    load_nested_dataset,
    load_stats,
    load_tasks,
-    load_tasks_high_level,
    update_chunk_file_indices,
    validate_episode_buffer,
    validate_frame,
@@ -162,7 +161,6 @@ class LeRobotDatasetMetadata:
        self.info = load_info(self.root)
        check_version_compatibility(self.repo_id, self._version, CODEBASE_VERSION)
        self.tasks = load_tasks(self.root)
-        self.tasks_high_level = load_tasks_high_level(self.root)
        self.episodes = load_episodes(self.root)
        self.stats = load_stats(self.root)

@@ -1052,12 +1050,6 @@ class LeRobotDataset(torch.utils.data.Dataset):
        # Add task as a string
        task_idx = item["task_index"].item()
        item["task"] = self.meta.tasks.iloc[task_idx].name
-        # Optionally add high level task index
-        if "task_index_high_level" in self.features:
-            high_level_task_idx = item["task_index_high_level"].item()
-            item["robot_utterance"] = self.meta.tasks_high_level.iloc[high_level_task_idx]["robot_utterance"]
-            item["user_prompt"] = self.meta.tasks_high_level.iloc[high_level_task_idx]["user_prompt"]
-
        return item

    def __repr__(self):
@@ -60,7 +60,6 @@ VIDEO_DIR = "videos"

 CHUNK_FILE_PATTERN = "chunk-{chunk_index:03d}/file-{file_index:03d}"
 DEFAULT_TASKS_PATH = "meta/tasks.parquet"
-DEFAULT_TASKS_HIGH_LEVEL_PATH = "meta/tasks_high_level.parquet"
 DEFAULT_EPISODES_PATH = EPISODES_DIR + "/" + CHUNK_FILE_PATTERN + ".parquet"
 DEFAULT_DATA_PATH = DATA_DIR + "/" + CHUNK_FILE_PATTERN + ".parquet"
 DEFAULT_VIDEO_PATH = VIDEO_DIR + "/{video_key}/" + CHUNK_FILE_PATTERN + ".mp4"
@@ -353,9 +352,6 @@ def load_tasks(local_dir: Path) -> pandas.DataFrame:
    tasks = pd.read_parquet(local_dir / DEFAULT_TASKS_PATH)
    return tasks

-def load_tasks_high_level(local_dir: Path) -> pandas.DataFrame:
-    tasks = pd.read_parquet(local_dir / DEFAULT_TASKS_HIGH_LEVEL_PATH)
-    return tasks

 def write_episodes(episodes: Dataset, local_dir: Path) -> None:
    """Write episode metadata to a parquet file in the LeRobot v3.0 format.
@@ -35,6 +35,8 @@ def make_optimizer_and_scheduler(
        tuple[Optimizer, LRScheduler | None]: The couple (Optimizer, Scheduler). Scheduler can be `None`.
    """
    params = policy.get_optim_params() if cfg.use_policy_training_preset else policy.parameters()
+    if cfg.optimizer is None:
+        raise ValueError("Optimizer config is required but not provided in TrainPipelineConfig")
    optimizer = cfg.optimizer.build(params)
    lr_scheduler = cfg.scheduler.build(optimizer, cfg.steps) if cfg.scheduler is not None else None
    return optimizer, lr_scheduler
@@ -14,6 +14,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 import abc
+from collections.abc import Iterable
 from dataclasses import asdict, dataclass, field
 from pathlib import Path
 from typing import Any
@@ -29,6 +30,17 @@ from lerobot.utils.constants import (
 )
 from lerobot.utils.io_utils import deserialize_json_into_object

+# Type alias for parameters accepted by optimizer build() methods.
+# This matches PyTorch's optimizer signature while also supporting:
+# - dict[str, Parameter]: Named parameters for differential LR by name (e.g., XVLA)
+# - dict[str, Iterable]: Multiple parameter groups for multi-optimizer configs (e.g., SAC)
+OptimizerParams = (
+    Iterable[torch.nn.Parameter]  # From model.parameters()
+    | Iterable[dict[str, Any]]  # List of param groups with lr/weight_decay overrides
+    | dict[str, torch.nn.Parameter]  # From dict(model.named_parameters()) for name-based LR
+    | dict[str, Any]  # For multi-optimizer configs (SAC) with multiple param groups
+)
+

@dataclass
 class OptimizerConfig(draccus.ChoiceRegistry, abc.ABC):
@@ -45,13 +57,24 @@ class OptimizerConfig(draccus.ChoiceRegistry, abc.ABC):
        return "adam"

    @abc.abstractmethod
-    def build(self) -> torch.optim.Optimizer | dict[str, torch.optim.Optimizer]:
+    def build(self, params: OptimizerParams) -> torch.optim.Optimizer | dict[str, torch.optim.Optimizer]:
        """
        Build the optimizer. It can be a single optimizer or a dictionary of optimizers.
+
        NOTE: Multiple optimizers are useful when you have different models to optimize.
        For example, you can have one optimizer for the policy and another one for the value function
        in reinforcement learning settings.

+        Args:
+            params: Parameters to optimize. Accepts multiple formats depending on the optimizer:
+                - Iterable[Parameter]: From model.parameters() - standard PyTorch usage
+                - Iterable[dict]: List of param groups with 'params' key and optional
+                  'lr', 'weight_decay' overrides (e.g., ACT, VQBeT policies)
+                - dict[str, Parameter]: From dict(model.named_parameters()) for optimizers
+                  that apply differential learning rates by parameter name (e.g., XVLA)
+                - dict[str, Iterable]: For multi-optimizer configs where each key maps to
+                  a separate optimizer's parameters (e.g., SAC with actor/critic/temperature)
+
        Returns:
            The optimizer or a dictionary of optimizers.
        """
@@ -67,7 +90,7 @@ class AdamConfig(OptimizerConfig):
    weight_decay: float = 0.0
    grad_clip_norm: float = 10.0

-    def build(self, params: dict) -> torch.optim.Optimizer:
+    def build(self, params: OptimizerParams) -> torch.optim.Optimizer:
        kwargs = asdict(self)
        kwargs.pop("grad_clip_norm")
        return torch.optim.Adam(params, **kwargs)
@@ -82,7 +105,7 @@ class AdamWConfig(OptimizerConfig):
    weight_decay: float = 1e-2
    grad_clip_norm: float = 10.0

-    def build(self, params: dict) -> torch.optim.Optimizer:
+    def build(self, params: OptimizerParams) -> torch.optim.Optimizer:
        kwargs = asdict(self)
        kwargs.pop("grad_clip_norm")
        return torch.optim.AdamW(params, **kwargs)
@@ -98,7 +121,7 @@ class SGDConfig(OptimizerConfig):
    weight_decay: float = 0.0
    grad_clip_norm: float = 10.0

-    def build(self, params: dict) -> torch.optim.Optimizer:
+    def build(self, params: OptimizerParams) -> torch.optim.Optimizer:
        kwargs = asdict(self)
        kwargs.pop("grad_clip_norm")
        return torch.optim.SGD(params, **kwargs)
@@ -139,21 +162,19 @@ class XVLAAdamWConfig(OptimizerConfig):
    soft_prompt_lr_scale: float = 1.0  # Scale factor for soft-prompt LR (1.0 = same as base LR)
    soft_prompt_warmup_lr_scale: float | None = None  # If set, start soft-prompts at this scale (e.g., 0.01)

-    def build(self, params: dict) -> torch.optim.Optimizer:
+    def build(self, params: OptimizerParams) -> torch.optim.Optimizer:
        """
        Build AdamW optimizer with differential learning rates.

-        Expects `named_parameters()` as input (dict of name -> param).
-        Applies:
-        - lr * 0.1 for all VLM-related parameters
-        - lr * soft_prompt_lr_scale for soft-prompt parameters (with optional warmup)
-        - full lr for all other parameters
-
        Args:
-            params: Dictionary of parameter names to parameters (from named_parameters())
+            params: Must be a dict[str, Parameter] from dict(model.named_parameters())
+                or equivalent.

        Returns:
            AdamW optimizer with parameter groups for VLM, soft-prompts, and other components
+
+        Raises:
+            AssertionError: If params is not a dict (e.g., from model.parameters())
        """
        assert isinstance(params, dict), "Custom LR optimizer requires `named_parameters()` as inputs."

@@ -174,7 +195,7 @@ class XVLAAdamWConfig(OptimizerConfig):
            # Start at warmup scale, scheduler will warm up to soft_prompt_lr
            soft_prompt_lr = self.lr * self.soft_prompt_warmup_lr_scale

-        param_groups = [
+        param_groups: list[dict[str, Any]] = [
            {
                "params": vlm_group,
                "lr": self.lr * 0.1,
@@ -224,19 +245,25 @@ class MultiAdamConfig(OptimizerConfig):
    grad_clip_norm: float = 10.0
    optimizer_groups: dict[str, dict[str, Any]] = field(default_factory=dict)

-    def build(self, params_dict: dict[str, list]) -> dict[str, torch.optim.Optimizer]:
+    def build(self, params: OptimizerParams) -> dict[str, torch.optim.Optimizer]:
        """Build multiple Adam optimizers.

        Args:
-            params_dict: Dictionary mapping parameter group names to lists of parameters
-                         The keys should match the keys in optimizer_groups
+            params: Must be a dict[str, Iterable[Parameter]] mapping parameter group names
+                to iterables of parameters. The keys should match the keys in optimizer_groups.
+                Typically from policies that need separate optimizers (e.g., SAC with
+                actor/critic/temperature).

        Returns:
            Dictionary mapping parameter group names to their optimizers
+
+        Raises:
+            AssertionError: If params is not a dict
        """
+        assert isinstance(params, dict), "MultiAdamConfig requires a dict of parameter groups as inputs."
        optimizers = {}

-        for name, params in params_dict.items():
+        for name, group_params in params.items():
            # Get group-specific hyperparameters or use defaults
            group_config = self.optimizer_groups.get(name, {})

@@ -248,7 +275,7 @@ class MultiAdamConfig(OptimizerConfig):
                "weight_decay": group_config.get("weight_decay", self.weight_decay),
            }

-            optimizers[name] = torch.optim.Adam(params, **optimizer_kwargs)
+            optimizers[name] = torch.optim.Adam(group_params, **optimizer_kwargs)

        return optimizers

@@ -30,7 +30,7 @@ from lerobot.utils.io_utils import deserialize_json_into_object

@dataclass
 class LRSchedulerConfig(draccus.ChoiceRegistry, abc.ABC):
-    num_warmup_steps: int
+    num_warmup_steps: int | None

    @property
    def type(self) -> str:
@@ -21,6 +21,7 @@ from .smolvla.configuration_smolvla import SmolVLAConfig as SmolVLAConfig
 from .smolvla.processor_smolvla import SmolVLANewLineProcessor
 from .tdmpc.configuration_tdmpc import TDMPCConfig as TDMPCConfig
 from .vqbet.configuration_vqbet import VQBeTConfig as VQBeTConfig
+from .wall_x.configuration_wall_x import WallXConfig as WallXConfig
 from .xvla.configuration_xvla import XVLAConfig as XVLAConfig

 __all__ = [
@@ -29,8 +30,10 @@ __all__ = [
    "PI0Config",
    "PI05Config",
    "SmolVLAConfig",
+    "SARMConfig",
    "TDMPCConfig",
    "VQBeTConfig",
    "GrootConfig",
    "XVLAConfig",
+    "WallXConfig",
 ]
@@ -50,6 +50,7 @@ class ACTPolicy(PreTrainedPolicy):
    def __init__(
        self,
        config: ACTConfig,
+        **kwargs,
    ):
        """
        Args:
@@ -56,6 +56,7 @@ class DiffusionPolicy(PreTrainedPolicy):
    def __init__(
        self,
        config: DiffusionConfig,
+        **kwargs,
    ):
        """
        Args:
@@ -37,10 +37,12 @@ from lerobot.policies.pi05.configuration_pi05 import PI05Config
 from lerobot.policies.pretrained import PreTrainedPolicy
 from lerobot.policies.sac.configuration_sac import SACConfig
 from lerobot.policies.sac.reward_model.configuration_classifier import RewardClassifierConfig
+from lerobot.policies.sarm.configuration_sarm import SARMConfig
 from lerobot.policies.smolvla.configuration_smolvla import SmolVLAConfig
 from lerobot.policies.tdmpc.configuration_tdmpc import TDMPCConfig
 from lerobot.policies.utils import validate_visual_features_consistency
 from lerobot.policies.vqbet.configuration_vqbet import VQBeTConfig
+from lerobot.policies.wall_x.configuration_wall_x import WallXConfig
 from lerobot.policies.xvla.configuration_xvla import XVLAConfig
 from lerobot.processor import PolicyAction, PolicyProcessorPipeline
 from lerobot.processor.converters import (
@@ -61,7 +63,7 @@ def get_policy_class(name: str) -> type[PreTrainedPolicy]:

    Args:
        name: The name of the policy. Supported names are "tdmpc", "diffusion", "act",
-              "vqbet", "pi0", "pi05", "sac", "reward_classifier", "smolvla".
+              "vqbet", "pi0", "pi05", "sac", "reward_classifier", "smolvla", "wall_x".

    Returns:
        The policy class corresponding to the given name.
@@ -105,6 +107,10 @@ def get_policy_class(name: str) -> type[PreTrainedPolicy]:
        from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy

        return SmolVLAPolicy
+    elif name == "sarm":
+        from lerobot.policies.sarm.modeling_sarm import SARMRewardModel
+
+        return SARMRewardModel
    elif name == "groot":
        from lerobot.policies.groot.modeling_groot import GrootPolicy

@@ -113,6 +119,10 @@ def get_policy_class(name: str) -> type[PreTrainedPolicy]:
        from lerobot.policies.xvla.modeling_xvla import XVLAPolicy

        return XVLAPolicy
+    elif name == "wall_x":
+        from lerobot.policies.wall_x.modeling_wall_x import WallXPolicy
+
+        return WallXPolicy
    else:
        try:
            return _get_policy_cls_from_policy_name(name=name)
@@ -130,7 +140,7 @@ def make_policy_config(policy_type: str, **kwargs) -> PreTrainedConfig:
    Args:
        policy_type: The type of the policy. Supported types include "tdmpc",
                     "diffusion", "act", "vqbet", "pi0", "pi05", "sac", "smolvla",
-                     "reward_classifier".
+                     "reward_classifier", "wall_x".
        **kwargs: Keyword arguments to be passed to the configuration class constructor.

    Returns:
@@ -161,6 +171,8 @@ def make_policy_config(policy_type: str, **kwargs) -> PreTrainedConfig:
        return GrootConfig(**kwargs)
    elif policy_type == "xvla":
        return XVLAConfig(**kwargs)
+    elif policy_type == "wall_x":
+        return WallXConfig(**kwargs)
    else:
        try:
            config_cls = PreTrainedConfig.get_choice_class(policy_type)
@@ -337,6 +349,14 @@ def make_pre_post_processors(
            dataset_stats=kwargs.get("dataset_stats"),
        )

+    elif isinstance(policy_cfg, SARMConfig):
+        from lerobot.policies.sarm.processor_sarm import make_sarm_pre_post_processors
+
+        processors = make_sarm_pre_post_processors(
+            config=policy_cfg,
+            dataset_stats=kwargs.get("dataset_stats"),
+            dataset_meta=kwargs.get("dataset_meta"),
+        )
    elif isinstance(policy_cfg, GrootConfig):
        from lerobot.policies.groot.processor_groot import make_groot_pre_post_processors

@@ -344,6 +364,7 @@ def make_pre_post_processors(
            config=policy_cfg,
            dataset_stats=kwargs.get("dataset_stats"),
        )
+
    elif isinstance(policy_cfg, XVLAConfig):
        from lerobot.policies.xvla.processor_xvla import (
            make_xvla_pre_post_processors,
@@ -354,6 +375,14 @@ def make_pre_post_processors(
            dataset_stats=kwargs.get("dataset_stats"),
        )

+    elif isinstance(policy_cfg, WallXConfig):
+        from lerobot.policies.wall_x.processor_wall_x import make_wall_x_pre_post_processors
+
+        processors = make_wall_x_pre_post_processors(
+            config=policy_cfg,
+            dataset_stats=kwargs.get("dataset_stats"),
+        )
+
    else:
        try:
            processors = _make_processors_from_policy_config(
@@ -435,11 +464,47 @@ def make_policy(
        cfg.input_features = {key: ft for key, ft in features.items() if key not in cfg.output_features}
    kwargs["config"] = cfg

-    if cfg.pretrained_path:
+    # Pass dataset_stats to the policy if available (needed for some policies like SARM)
+    if ds_meta is not None and hasattr(ds_meta, "stats"):
+        kwargs["dataset_stats"] = ds_meta.stats
+
+    if ds_meta is not None:
+        kwargs["dataset_meta"] = ds_meta
+
+    if not cfg.pretrained_path and cfg.use_peft:
+        raise ValueError(
+            "Instantiating a policy with `use_peft=True` without a checkpoint is not supported since that requires "
+            "the PEFT config parameters to be set. For training with PEFT, see `lerobot_train.py` on how to do that."
+        )
+
+    if cfg.pretrained_path and not cfg.use_peft:
        # Load a pretrained policy and override the config if needed (for example, if there are inference-time
        # hyperparameters that we want to vary).
        kwargs["pretrained_name_or_path"] = cfg.pretrained_path
        policy = policy_cls.from_pretrained(**kwargs)
+    elif cfg.pretrained_path and cfg.use_peft:
+        # Load a pretrained PEFT model on top of the policy. The pretrained path points to the folder/repo
+        # of the adapter and the adapter's config contains the path to the base policy. So we need the
+        # adapter config first, then load the correct policy and then apply PEFT.
+        from peft import PeftConfig, PeftModel
+
+        logging.info("Loading policy's PEFT adapter.")
+
+        peft_pretrained_path = cfg.pretrained_path
+        peft_config = PeftConfig.from_pretrained(peft_pretrained_path)
+
+        kwargs["pretrained_name_or_path"] = peft_config.base_model_name_or_path
+        if not kwargs["pretrained_name_or_path"]:
+            # This means that there's a bug or we trained a policy from scratch using PEFT.
+            # It is more likely that this is a bug so we'll raise an error.
+            raise ValueError(
+                "No pretrained model name found in adapter config. Can't instantiate the pre-trained policy on which "
+                "the adapter was trained."
+            )
+
+        policy = policy_cls.from_pretrained(**kwargs)
+        policy = PeftModel.from_pretrained(policy, peft_pretrained_path, config=peft_config)
+
    else:
        # Make a fresh policy.
        policy = policy_cls(**kwargs)
@@ -49,7 +49,7 @@ class GrootPolicy(PreTrainedPolicy):
    name = "groot"
    config_class = GrootConfig

-    def __init__(self, config: GrootConfig):
+    def __init__(self, config: GrootConfig, **kwargs):
        """Initialize Groot policy wrapper."""
        super().__init__(config)
        config.validate_features()
@@ -23,6 +23,8 @@ from lerobot.optim.schedulers import CosineDecayWithWarmupSchedulerConfig
 from lerobot.policies.rtc.configuration_rtc import RTCConfig
 from lerobot.utils.constants import OBS_IMAGES

+DEFAULT_IMAGE_SIZE = 224
+

@PreTrainedConfig.register_subclass("pi0")
@dataclass
@@ -51,7 +53,10 @@ class PI0Config(PreTrainedConfig):
    # Real-Time Chunking (RTC) configuration
    rtc_config: RTCConfig | None = None

-    image_resolution: tuple[int, int] = (224, 224)  # see openpi `preprocessing_pytorch.py`
+    image_resolution: tuple[int, int] = (
+        DEFAULT_IMAGE_SIZE,
+        DEFAULT_IMAGE_SIZE,
+    )  # see openpi `preprocessing_pytorch.py`

    # Add empty images. Used to add empty cameras when no image features are present.
    empty_cameras: int = 0
@@ -41,7 +41,7 @@ else:
    PaliGemmaForConditionalGeneration = None

 from lerobot.configs.policies import PreTrainedConfig
-from lerobot.policies.pi0.configuration_pi0 import PI0Config
+from lerobot.policies.pi0.configuration_pi0 import DEFAULT_IMAGE_SIZE, PI0Config
 from lerobot.policies.pretrained import PreTrainedPolicy, T
 from lerobot.policies.rtc.modeling_rtc import RTCProcessor
 from lerobot.utils.constants import (
@@ -337,6 +337,7 @@ class PaliGemmaWithExpertModel(
        action_expert_config,
        use_adarms=None,
        precision: Literal["bfloat16", "float32"] = "bfloat16",
+        image_size: int = DEFAULT_IMAGE_SIZE,
    ):
        if use_adarms is None:
            use_adarms = [False, False]
@@ -356,6 +357,7 @@ class PaliGemmaWithExpertModel(
        vlm_config_hf.text_config.vocab_size = 257152
        vlm_config_hf.text_config.use_adarms = use_adarms[0]
        vlm_config_hf.text_config.adarms_cond_dim = vlm_config.width if use_adarms[0] else None
+        vlm_config_hf.vision_config.image_size = image_size
        vlm_config_hf.vision_config.intermediate_size = 4304
        vlm_config_hf.vision_config.projection_dim = 2048
        vlm_config_hf.vision_config.projector_hidden_act = "gelu_fast"
@@ -519,11 +521,17 @@ class PI0Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        paligemma_config = get_gemma_config(config.paligemma_variant)
        action_expert_config = get_gemma_config(config.action_expert_variant)

+        if config.image_resolution[0] != config.image_resolution[1]:
+            raise ValueError(
+                f"PaliGemma expects square image resolution, invalid resolution: {config.image_resolution}"
+            )
+
        self.paligemma_with_expert = PaliGemmaWithExpertModel(
            paligemma_config,
            action_expert_config,
            use_adarms=[False, False],
            precision=config.dtype,
+            image_size=config.image_resolution[0],
        )

        self.action_in_proj = nn.Linear(config.max_action_dim, action_expert_config.width)
@@ -812,16 +820,13 @@ class PI0Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        )

        dt = -1.0 / num_steps
-        dt = torch.tensor(dt, dtype=torch.float32, device=device)

        x_t = noise
-        time = torch.tensor(1.0, dtype=torch.float32, device=device)
-        while time >= -dt / 2:
-            expanded_time = time.expand(bsize)
+        for step in range(num_steps):
+            time = 1.0 + step * dt
+            time_tensor = torch.tensor(time, dtype=torch.float32, device=device).expand(bsize)

-            # Define a closure function to properly capture expanded_time
-            # This avoids the lambda expression (E731) and loop variable binding (B023) issues
-            def denoise_step_partial_call(input_x_t, current_timestep=expanded_time):
+            def denoise_step_partial_call(input_x_t, current_timestep=time_tensor):
                return self.denoise_step(
                    state=state,
                    prefix_pad_masks=prefix_pad_masks,
@@ -846,15 +851,11 @@ class PI0Pytorch(nn.Module):  # see openpi `PI0Pytorch`
            else:
                v_t = denoise_step_partial_call(x_t)

-            # Euler step
-            x_t += dt * v_t
+            x_t = x_t + dt * v_t

-            # Record x_t and v_t after Euler step
            if self.rtc_processor is not None and self.rtc_processor.is_debug_enabled():
                self.rtc_processor.track(time=time, x_t=x_t, v_t=v_t)

-            time += dt
-
        return x_t

    def denoise_step(
@@ -906,6 +907,7 @@ class PI0Policy(PreTrainedPolicy):
    def __init__(
        self,
        config: PI0Config,
+        **kwargs,
    ):
        """
        Args:
@@ -1234,9 +1236,15 @@ class PI0Policy(PreTrainedPolicy):

        return actions

-    def forward(self, batch: dict[str, Tensor]) -> tuple[Tensor, dict]:
-        """Run the batch through the model and compute the loss for training."""
+    def forward(self, batch: dict[str, Tensor], reduction: str = "mean") -> tuple[Tensor, dict]:
+        """Run the batch through the model and compute the loss for training.

+        Args:
+            batch: Training batch containing observations and actions.
+            reduction: How to reduce the loss. Options:
+                - "mean": Return scalar mean loss (default, backward compatible)
+                - "none": Return per-sample losses of shape (batch_size,) for RA-BC weighting
+        """
        # Prepare inputs
        images, img_masks = self._preprocess_images(batch)
        lang_tokens, lang_masks = batch[f"{OBS_LANGUAGE_TOKENS}"], batch[f"{OBS_LANGUAGE_ATTENTION_MASK}"]
@@ -1250,11 +1258,17 @@ class PI0Policy(PreTrainedPolicy):
        original_action_dim = self.config.output_features[ACTION].shape[0]
        losses = losses[:, :, :original_action_dim]

-        loss = losses.mean()
-
        loss_dict = {
-            "loss": loss.item(),
            "loss_per_dim": losses.mean(dim=[0, 1]).detach().cpu().numpy().tolist(),
        }

-        return loss, loss_dict
+        if reduction == "none":
+            # Return per-sample losses (B,) by averaging over time and action dims
+            per_sample_loss = losses.mean(dim=(1, 2))
+            loss_dict["loss"] = per_sample_loss.mean().item()
+            return per_sample_loss, loss_dict
+        else:
+            # Default: return scalar mean loss
+            loss = losses.mean()
+            loss_dict["loss"] = loss.item()
+            return loss, loss_dict
@@ -1,196 +0,0 @@
-# FAST Tokenizer Training for LeRobotDataset
-
-This directory contains tools for training a FAST (Factorized Action Sequence Tokenizer) on LeRobot datasets.
-
-## Files
-
- **`train_fast_tokenizer.py`**: Main training script (refactored for LeRobotDataset)
- **`train_fast_tokenizer_example.md`**: Usage examples and parameter documentation
- **`MIGRATION_NOTES.md`**: Migration guide from B1K to LeRobotDataset
-
-## Quick Start
-
-```bash
-# Basic usage
-python train_fast_tokenizer.py \
-    --repo_id "lerobot/aloha_sim_insertion_human" \
-    --action_horizon 10 \
-    --encoded_dims "0:14"
-
-# With delta transform
-python train_fast_tokenizer.py \
-    --repo_id "lerobot/aloha_sim_insertion_human" \
-    --action_horizon 10 \
-    --encoded_dims "0:14" \
-    --delta_dims "0,1,2,3,4,5,6,7,8,9,10,11,12,13" \
-    --state_key "observation.state" \
-    --vocab_size 1024
-```
-
-## What is FAST?
-
-FAST is a tokenizer for robotic action sequences that:
-1. Applies DCT (Discrete Cosine Transform) to action chunks
-2. Quantizes DCT coefficients 
-3. Uses BPE (Byte-Pair Encoding) to compress the quantized sequence
-4. Achieves high compression ratios (e.g., 10-20x) while maintaining accuracy
-
-This enables efficient storage and processing of long action sequences in vision-language-action models.
-
-## Requirements
-
- Python 3.10+
- LeRobot dataset (either local or from HuggingFace Hub)
- transformers (for AutoProcessor)
- numpy
- torch
- tyro
-
-## Workflow
-
-```
-LeRobotDataset → Extract Episodes → Apply Delta Transform 
-    ↓
-Select Dimensions → Normalize (q01, q99) → Create Chunks
-    ↓
-Train FAST Tokenizer → Compute Stats → Save
-```
-
-## Parameters Guide
-
-### Essential Parameters
-
- **`repo_id`**: HuggingFace dataset repository ID
-  - Example: `"lerobot/aloha_sim_insertion_human"`
-  
- **`action_horizon`**: Length of action sequences to tokenize
-  - Typical: 10-16 steps
-  
- **`encoded_dims`**: Which action dimensions to encode
-  - Format: `"start:end,start:end"`
-  - Example: `"0:7"` = dimensions 0-6
-  - Example: `"0:3,7:10"` = dimensions 0-2 and 7-9
-
-### Optional Parameters
-
- **`delta_dims`**: Apply delta transform (action - state) to these dimensions
-  - Format: `"0,1,2,3,4,5"`
-  - Use for position-based actions
-  
- **`state_key`**: Dataset key containing state observations
-  - Default: `"observation.state"`
-  
- **`vocab_size`**: BPE vocabulary size
-  - Default: 1024
-  - Larger = better compression but more memory
-  
- **`scale`**: DCT quantization scale
-  - Default: 10.0
-  - Smaller = finer quantization, larger = coarser
-
- **`sample_fraction`**: Fraction of action chunks to use per episode
-  - Default: 0.1 (10%)
-  - Increase for small datasets, decrease for large datasets
-
-## Output
-
-The script creates a directory (default: `./fast_tokenizer_{repo_id}`) containing:
-
-1. **Tokenizer files**: Can be loaded with `AutoProcessor.from_pretrained()`
-2. **`metadata.json`**: Contains:
-   - Training configuration
-   - Compression statistics
-   - Dataset information
-
-## Example Output
-
-```
-Loading dataset: lerobot/aloha_sim_insertion_human
-Dataset loaded: 50 episodes, 5000 frames
-Encoding 14 dimensions: 0:14
-Delta dimensions: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
-Action horizon: 10
-Processing 50 episodes...
-Collected 4500 action chunks
-Extracted 14 encoded dimensions
-
-Before normalization - overall stats:
-  Min: -2.3451, Max: 3.1234, Mean: 0.0234, Std: 0.8765
-
-Applied quantile normalization [q01, q99] → [-1, 1]
-
-After normalization - overall stats:
-  Min: -1.0000, Max: 1.0000, Mean: 0.0156, Std: 0.4321
-
-Training FAST tokenizer on 4500 action chunks...
-Action chunk shape: (4500, 10, 14)
-Vocab size: 1024
-DCT scale: 10.0
-✓ Tokenizer training complete!
-
-Compression Statistics:
-  Average compression ratio: 14.23x
-  Mean token length: 9.8
-  P99 token length: 15
-  Min token length: 6
-  Max token length: 18
-
-✅ Saved FAST tokenizer to ./fast_tokenizer_lerobot_aloha_sim_insertion_human
-```
-
-## Using the Trained Tokenizer
-
-```python
-from transformers import AutoProcessor
-
-# Load tokenizer
-tokenizer = AutoProcessor.from_pretrained(
-    "./fast_tokenizer_lerobot_aloha_sim_insertion_human",
-    trust_remote_code=True
-)
-
-# Encode action chunk [horizon, action_dim]
-action_chunk = np.random.randn(10, 14)  # Example
-tokens = tokenizer(action_chunk[None])[0]  # Returns token IDs
-
-# Decode tokens back to actions
-reconstructed = tokenizer.decode(tokens)
-```
-
-## Tips
-
-1. **Start Small**: Use `--max_episodes 10` for initial testing
-2. **Check Dimensions**: Verify encoded dimensions match your robot's action space
-3. **Delta Transform**: Use for position-based actions, not velocity-based
-4. **Normalization**: Ensure dataset has proper statistics computed
-5. **Compression Ratio**: Aim for 10-20x for good balance of compression and accuracy
-
-## Troubleshooting
-
-**Issue**: "No normalization stats found"
- **Solution**: Compute dataset statistics first, or use raw actions
-
-**Issue**: "Episode too short for action horizon"
- **Solution**: Reduce `--action_horizon` or filter short episodes
-
-**Issue**: "State key not found"
- **Solution**: Check dataset features and use correct `--state_key`
-
-**Issue**: Memory error with large datasets
- **Solution**: Reduce `--sample_fraction` or `--max_episodes`
-
-## Citation
-
-If you use FAST in your research, please cite:
-
-```bibtex
-@article{black2023fast,
-  title={FAST: Factorized Action Sequence Tokenizer for Vision-Language-Action Models},
-  author={Black, Kevin and others},
-  journal={arXiv preprint},
-  year={2023}
-}
-```
-
-
-
@@ -22,6 +22,8 @@ from lerobot.optim.optimizers import AdamWConfig
 from lerobot.optim.schedulers import CosineDecayWithWarmupSchedulerConfig
 from lerobot.policies.rtc.configuration_rtc import RTCConfig

+DEFAULT_IMAGE_SIZE = 224
+

@PreTrainedConfig.register_subclass("pi05")
@dataclass
@@ -37,9 +39,6 @@ class PI05Config(PreTrainedConfig):
    # Shorter state and action vectors will be padded to these dimensions
    max_state_dim: int = 32
    max_action_dim: int = 32
-    max_action_tokens: int = 32
-    fast_vocab_size: int = 2048
-    

    # Flow matching parameters: see openpi `PI0Pytorch`
    num_inference_steps: int = 10
@@ -53,7 +52,10 @@ class PI05Config(PreTrainedConfig):
    # Real-Time Chunking (RTC) configuration
    rtc_config: RTCConfig | None = None

-    image_resolution: tuple[int, int] = (224, 224)  # see openpi `preprocessing_pytorch.py`
+    image_resolution: tuple[int, int] = (
+        DEFAULT_IMAGE_SIZE,
+        DEFAULT_IMAGE_SIZE,
+    )  # see openpi `preprocessing_pytorch.py`

    # Add empty images. Used to add empty cameras when no image features are present.
    empty_cameras: int = 0
@@ -63,8 +65,8 @@ class PI05Config(PreTrainedConfig):
    normalization_mapping: dict[str, NormalizationMode] = field(
        default_factory=lambda: {
            "VISUAL": NormalizationMode.IDENTITY,
-            "STATE": NormalizationMode.MEAN_STD,  # Pi0.5 uses quantiles for state
-            "ACTION": NormalizationMode.MEAN_STD,  # Pi0.5 uses quantiles for action
+            "STATE": NormalizationMode.QUANTILES,  # Pi0.5 uses quantiles for state
+            "ACTION": NormalizationMode.QUANTILES,  # Pi0.5 uses quantiles for action
        }
    )

@@ -1,21 +0,0 @@
-lerobot-train \
-    --dataset.repo_id=lerobot \
-    --dataset.root=/fsx/jade_choghari/outputs/collect-data-pgen \
-    --output_dir=/fsx/jade_choghari/outputs/pi0test1 \
-    --job_name=pi0_training \
-    --policy.repo_id=jade_choghari/pi0-base \
-    --policy.path=/fsx/jade_choghari/outputs/pi0_fast_fruit1/checkpoints/last/pretrained_model \
-    --policy.dtype=bfloat16 \
-    --steps=3000 \
-    --save_freq=1000 \
-    --rename_map='{
-        "observation.images.base": "observation.images.base_0_rgb",
-        "observation.images.left_wrist": "observation.images.left_wrist_0_rgb",
-        "observation.images.right_wrist": "observation.images.right_wrist_0_rgb",
-        }' \
-    --batch_size=4 \
-    --policy.device=cuda \
-    # --wandb.enable=true \
-    # --wandb.disable_artifact=true \
-    # --wandb.project=pi05hi-training \
-
@@ -41,17 +41,13 @@ else:
    PaliGemmaForConditionalGeneration = None

 from lerobot.configs.policies import PreTrainedConfig
-from lerobot.policies.pi05.configuration_pi05 import PI05Config
+from lerobot.policies.pi05.configuration_pi05 import DEFAULT_IMAGE_SIZE, PI05Config
 from lerobot.policies.pretrained import PreTrainedPolicy, T
 from lerobot.policies.rtc.modeling_rtc import RTCProcessor
 from lerobot.utils.constants import (
    ACTION,
    OBS_LANGUAGE_ATTENTION_MASK,
    OBS_LANGUAGE_TOKENS,
-    OBS_LANGUAGE_HIGH_LEVEL_TASK_TOKENS,
-    OBS_LANGUAGE_HIGH_LEVEL_TASK_ATTENTION_MASK,
-    OBS_LANGUAGE_SUBTASK_ONLY_TOKENS,
-    OBS_LANGUAGE_SUBTASK_ONLY_ATTENTION_MASK,
    OPENPI_ATTENTION_MASK_VALUE,
 )

@@ -340,6 +336,7 @@ class PaliGemmaWithExpertModel(
        action_expert_config,
        use_adarms=None,
        precision: Literal["bfloat16", "float32"] = "bfloat16",
+        image_size: int = DEFAULT_IMAGE_SIZE,
    ):
        if use_adarms is None:
            use_adarms = [False, False]
@@ -359,6 +356,7 @@ class PaliGemmaWithExpertModel(
        vlm_config_hf.text_config.vocab_size = 257152
        vlm_config_hf.text_config.use_adarms = use_adarms[0]
        vlm_config_hf.text_config.adarms_cond_dim = vlm_config.width if use_adarms[0] else None
+        vlm_config_hf.vision_config.image_size = image_size
        vlm_config_hf.vision_config.intermediate_size = 4304
        vlm_config_hf.vision_config.projection_dim = 2048
        vlm_config_hf.vision_config.projector_hidden_act = "gelu_fast"
@@ -433,8 +431,6 @@ class PaliGemmaWithExpertModel(
                adarms_cond=adarms_cond[0] if adarms_cond is not None else None,
            )
            prefix_past_key_values = prefix_output.past_key_values
-            # prefix_output to be used for the language head
-            # shape: [batch_size, seq_len, hidden_size] with hidden_size = 2048
            prefix_output = prefix_output.last_hidden_state
            suffix_output = None
        elif inputs_embeds[0] is None:
@@ -524,11 +520,17 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        paligemma_config = get_gemma_config(config.paligemma_variant)
        action_expert_config = get_gemma_config(config.action_expert_variant)

+        if config.image_resolution[0] != config.image_resolution[1]:
+            raise ValueError(
+                f"PaliGemma expects square image resolution, invalid resolution: {config.image_resolution}"
+            )
+
        self.paligemma_with_expert = PaliGemmaWithExpertModel(
            paligemma_config,
            action_expert_config,
            use_adarms=[False, True],
            precision=config.dtype,
+            image_size=config.image_resolution[0],
        )

        self.action_in_proj = nn.Linear(config.max_action_dim, action_expert_config.width)
@@ -537,18 +539,6 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        self.time_mlp_in = nn.Linear(action_expert_config.width, action_expert_config.width)
        self.time_mlp_out = nn.Linear(action_expert_config.width, action_expert_config.width)

-        # FAST action token embedding and prediction head
-        self.fast_action_embedding = nn.Embedding(config.fast_vocab_size, paligemma_config.width)
-        self.fast_action_lm_head = nn.Linear(paligemma_config.width, config.fast_vocab_size)
-
-        # Apply dtype conversion to FAST layers to match model precision
-        if config.dtype == "bfloat16":
-            self.fast_action_embedding = self.fast_action_embedding.to(dtype=torch.bfloat16)
-            self.fast_action_lm_head = self.fast_action_lm_head.to(dtype=torch.bfloat16)
-        elif config.dtype == "float32":
-            self.fast_action_embedding = self.fast_action_embedding.to(dtype=torch.float32)
-            self.fast_action_lm_head = self.fast_action_lm_head.to(dtype=torch.float32)
-
        # Initialize gradient checkpointing flag
        self.gradient_checkpointing_enabled = False

@@ -596,201 +586,10 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
            )
        return func(*args, **kwargs)

-    def _prepare_attention_masks_4d(self, att_2d_masks, dtype=None):
+    def _prepare_attention_masks_4d(self, att_2d_masks):
        """Helper method to prepare 4D attention masks for transformer."""
        att_2d_masks_4d = att_2d_masks[:, None, :, :]
-        result = torch.where(att_2d_masks_4d, 0.0, OPENPI_ATTENTION_MASK_VALUE)
-        if dtype is not None:
-            result = result.to(dtype=dtype)
-        return result
-
-    def _create_custom_attention_mask(self, att_mask_segments, pad_masks, bsize):
-        """Create custom 2D attention mask for the new attention pattern.
-        
-        Attention rules:
-        - Images + Language: bidirectional among themselves, don't attend to subtask or FAST
-        - Subtask: attend to images + language, causal among themselves, don't attend to FAST
-        - FAST: attend to images + language + subtask, causal among themselves
-        
-        Args:
-            att_mask_segments: List of (type, length) tuples
-            pad_masks: Padding masks [B, total_seq_len]
-            bsize: Batch size
-        
-        Returns:
-            att_2d_masks: 2D attention mask [B, total_seq_len, total_seq_len]
-        """
-        total_len = sum(length for _, length in att_mask_segments)
-        device = pad_masks.device
-        
-        # Initialize attention mask as False (cannot attend)
-        att_2d_masks = torch.zeros(bsize, total_len, total_len, dtype=torch.bool, device=device)
-        
-        # Track positions for each segment
-        positions = []
-        current_pos = 0
-        for seg_type, seg_len in att_mask_segments:
-            positions.append((seg_type, current_pos, current_pos + seg_len))
-            current_pos += seg_len
-        
-        # Apply attention rules
-        for i, (query_type, query_start, query_end) in enumerate(positions):
-            for j, (key_type, key_start, key_end) in enumerate(positions):
-                # Images and Language can attend to each other bidirectionally
-                if query_type in ['image', 'language'] and key_type in ['image', 'language']:
-                    att_2d_masks[:, query_start:query_end, key_start:key_end] = True
-                
-                # Subtask tokens attend to images + language
-                elif query_type == 'subtask' and key_type in ['image', 'language']:
-                    att_2d_masks[:, query_start:query_end, key_start:key_end] = True
-                
-                # Subtask tokens attend causally to themselves
-                elif query_type == 'subtask' and key_type == 'subtask':
-                    # Create causal mask for subtask tokens
-                    subtask_len = query_end - query_start
-                    causal_mask = torch.tril(torch.ones(subtask_len, subtask_len, dtype=torch.bool, device=device))
-                    att_2d_masks[:, query_start:query_end, key_start:key_end] = causal_mask[None, :, :]
-                
-                # FAST tokens attend to images + language + subtask
-                elif query_type == 'fast' and key_type in ['image', 'language', 'subtask']:
-                    att_2d_masks[:, query_start:query_end, key_start:key_end] = True
-                
-                # FAST tokens attend causally to themselves
-                elif query_type == 'fast' and key_type == 'fast':
-                    fast_len = query_end - query_start
-                    causal_mask = torch.tril(torch.ones(fast_len, fast_len, dtype=torch.bool, device=device))
-                    att_2d_masks[:, query_start:query_end, key_start:key_end] = causal_mask[None, :, :]
-        
-        # Apply padding masks
-        pad_2d_masks = pad_masks[:, None, :] * pad_masks[:, :, None]
-        att_2d_masks = att_2d_masks & pad_2d_masks
-        
-        return att_2d_masks
-
-    def visualize_attention_mask(
-        self,
-        att_mask_segments,
-        att_2d_masks,
-        save_path,
-        batch_idx=0,
-        dpi=150,
-        max_display_tokens=None
-    ):
-        """Visualize the attention mask with labeled segments.
-        
-        Args:
-            att_mask_segments: List of (type, length) tuples defining the segments
-            att_2d_masks: 2D attention mask tensor [B, total_seq_len, total_seq_len]
-            save_path: Path where to save the visualization image
-            batch_idx: Which batch item to visualize (default: 0)
-            dpi: DPI for the saved image (default: 150)
-            max_display_tokens: Maximum number of tokens to display (for very long sequences)
-        """
-        try:
-            import matplotlib.pyplot as plt
-            import matplotlib.patches as mpatches
-            from matplotlib.colors import LinearSegmentedColormap
-        except ImportError:
-            logging.warning("matplotlib not available, skipping attention mask visualization")
-            return
-        
-        # Extract the mask for the specified batch
-        mask = att_2d_masks[batch_idx].cpu().float().numpy()
-        
-        # If sequence is too long, downsample for visualization
-        if max_display_tokens is not None and mask.shape[0] > max_display_tokens:
-            # Simple downsampling by taking every Nth token
-            step = mask.shape[0] // max_display_tokens
-            mask = mask[::step, ::step]
-            # Adjust segments accordingly
-            att_mask_segments = [(seg_type, max(1, seg_len // step)) for seg_type, seg_len in att_mask_segments]
-        
-        # Calculate positions for each segment
-        positions = []
-        current_pos = 0
-        for seg_type, seg_len in att_mask_segments:
-            positions.append((seg_type, current_pos, current_pos + seg_len))
-            current_pos += seg_len
-        
-        # Create figure
-        fig, ax = plt.subplots(figsize=(12, 10))
-        
-        # Create custom colormap: white for False (no attention), blue for True (attention)
-        colors = ['white', '#2E86AB']
-        n_bins = 2
-        cmap = LinearSegmentedColormap.from_list('attention', colors, N=n_bins)
-        
-        # Display the mask
-        im = ax.imshow(mask, cmap=cmap, aspect='auto', interpolation='nearest', vmin=0, vmax=1)
-        
-        # Add colorbar
-        cbar = plt.colorbar(im, ax=ax, fraction=0.046, pad=0.04)
-        cbar.set_label('Attention Enabled', rotation=270, labelpad=20)
-        cbar.set_ticks([0.25, 0.75])
-        cbar.set_ticklabels(['No', 'Yes'])
-        
-        # Define colors for each segment type
-        segment_colors = {
-            'image': '#A23B72',
-            'language': '#F18F01',
-            'subtask': '#C73E1D',
-            'fast': '#6A994E'
-        }
-        
-        # Draw segment boundaries and labels
-        for seg_type, start, end in positions:
-            color = segment_colors.get(seg_type, '#666666')
-            
-            # Draw vertical lines for columns (keys)
-            ax.axvline(x=start - 0.5, color=color, linewidth=2, alpha=0.7)
-            ax.axvline(x=end - 0.5, color=color, linewidth=2, alpha=0.7)
-            
-            # Draw horizontal lines for rows (queries)
-            ax.axhline(y=start - 0.5, color=color, linewidth=2, alpha=0.7)
-            ax.axhline(y=end - 0.5, color=color, linewidth=2, alpha=0.7)
-            
-            # Add labels at the top
-            mid_pos = (start + end) / 2
-            ax.text(mid_pos, -mask.shape[0] * 0.02, f"{seg_type.upper()}\n({end - start})",
-                   ha='center', va='top', fontsize=10, fontweight='bold', color=color)
-            
-            # Add labels on the left
-            ax.text(-mask.shape[1] * 0.02, mid_pos, f"{seg_type.upper()}\n({end - start})",
-                   ha='right', va='center', fontsize=10, fontweight='bold', color=color, rotation=0)
-        
-        # Set axis labels
-        ax.set_xlabel('Key Position (tokens being attended to)', fontsize=12, fontweight='bold')
-        ax.set_ylabel('Query Position (tokens attending)', fontsize=12, fontweight='bold')
-        ax.set_title('Attention Mask Pattern\n(White = No Attention, Blue = Attention Allowed)', 
-                    fontsize=14, fontweight='bold', pad=20)
-        
-        # Create legend for segment types
-        legend_patches = []
-        attention_rules = {
-            'image': 'Bidirectional with lang',
-            'language': 'Bidirectional with images',
-            'subtask': 'Attends to img+lang, causal self',
-            'fast': 'Attends to all, causal self'
-        }
-        for seg_type, color in segment_colors.items():
-            if any(seg[0] == seg_type for seg in att_mask_segments):
-                rule = attention_rules.get(seg_type, '')
-                legend_patches.append(mpatches.Patch(color=color, label=f'{seg_type.upper()}: {rule}'))
-        
-        ax.legend(handles=legend_patches, loc='upper right', bbox_to_anchor=(1.15, 1.0),
-                 framealpha=0.9, fontsize=9)
-        
-        # Adjust layout and save
-        plt.tight_layout()
-        
-        # Ensure the directory exists
-        save_path = Path(save_path)
-        save_path.parent.mkdir(parents=True, exist_ok=True)
-        
-        plt.savefig(save_path, dpi=dpi, bbox_inches='tight')
-        plt.close()
-        
-        logging.info(f"Attention mask visualization saved to: {save_path}")
+        return torch.where(att_2d_masks_4d, 0.0, OPENPI_ATTENTION_MASK_VALUE)

    def sample_noise(self, shape, device):
        return torch.normal(
@@ -807,44 +606,15 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        )
        time = time_beta * self.config.time_sampling_scale + self.config.time_sampling_offset
        return time.to(dtype=torch.float32, device=device)
-    
+
    def embed_prefix(
-        self, 
-        images, 
-        img_masks, 
-        tokens, 
-        subtask_tokens, 
-        masks, 
-        subtask_masks, 
-        fast_action_tokens=None, 
-        fast_action_masks=None,
-    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, int, int, int]:
-        """Embed images with SigLIP, tokens, and optionally subtask tokens with embedding layer.
-        
-        Args:
-            images: List of image tensors
-            img_masks: List of image masks
-            tokens: Language instruction tokens
-            subtask_tokens: Subtask tokens to predict (can be None for inference)
-            masks: Attention masks for tokens
-            fast_action_tokens: FAST action tokens for auxiliary prediction (can be None) - discrete token IDs
-            fast_action_masks: Padding masks for FAST action tokens (can be None)
-            
-        Returns:
-            embs: Concatenated embeddings [images, tokens, (subtask_tokens if provided), (fast_action_tokens if provided)]
-            pad_masks: Padding masks
-            att_masks: Custom 2D attention mask implementing the required pattern
-            total_T_images: Total number of image tokens
-            num_subtask_embs: Number of subtask token embeddings
-            num_fast_embs: Number of FAST action token embeddings
-        """
+        self, images, img_masks, tokens, masks
+    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
+        """Embed images with SigLIP and language tokens with embedding layer."""
        embs = []
        pad_masks = []
-        att_mask_segments = []  # Store info about each segment for custom mask creation
-        total_T_images = 0
-        num_subtask_embs = 0
-        num_fast_embs = 0
-        
+        att_masks = []
+
        # Process images
        for img, img_mask in zip(images, img_masks, strict=True):

@@ -856,10 +626,9 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`

            embs.append(img_emb)
            pad_masks.append(img_mask[:, None].expand(bsize, num_img_embs))
-            att_mask_segments.append(('image', num_img_embs))
-            total_T_images += num_img_embs
-            
-        # Process language instruction tokens
+            att_masks += [0] * num_img_embs
+
+        # Process language tokens
        def lang_embed_func(tokens):
            lang_emb = self.paligemma_with_expert.embed_language_tokens(tokens)
            lang_emb_dim = lang_emb.shape[-1]
@@ -870,65 +639,16 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        pad_masks.append(masks)

        num_lang_embs = lang_emb.shape[1]
-        att_mask_segments.append(('language', num_lang_embs))
-
-        # Process subtask tokens if provided (these are predicted, so use causal masking)
-        if subtask_tokens is not None:
-            def subtask_embed_func(subtask_tokens):
-                subtask_emb = self.paligemma_with_expert.embed_language_tokens(subtask_tokens)
-                subtask_emb_dim = subtask_emb.shape[-1]
-                return subtask_emb * math.sqrt(subtask_emb_dim)
-
-            subtask_emb = self._apply_checkpoint(subtask_embed_func, subtask_tokens)
-            embs.append(subtask_emb)
-            
-            # Create subtask pad masks (non-zero tokens are valid)
-            pad_masks.append(subtask_masks)
-
-            num_subtask_embs = subtask_emb.shape[1]
-            att_mask_segments.append(('subtask', num_subtask_embs))
-        # Process FAST action tokens if provided (these are discrete token IDs)
-        if fast_action_tokens is not None:
-            def fast_action_embed_func(fast_action_tokens):
-                fast_emb = self.fast_action_embedding(fast_action_tokens)
-                fast_emb_dim = fast_emb.shape[-1]
-                return fast_emb * math.sqrt(fast_emb_dim)
-            
-            fast_action_emb = self._apply_checkpoint(fast_action_embed_func, fast_action_tokens)
-            embs.append(fast_action_emb)
-            
-            # Use provided mask or create default (all valid)
-            if fast_action_masks is not None:
-                fast_pad_mask = fast_action_masks
-            else:
-                bsize = fast_action_tokens.shape[0]
-                num_fast_embs = fast_action_tokens.shape[1]
-                fast_pad_mask = torch.ones(bsize, num_fast_embs, dtype=torch.bool, device=fast_action_tokens.device)
-            
-            num_fast_embs = fast_action_tokens.shape[1]
-            pad_masks.append(fast_pad_mask)
-            att_mask_segments.append(('fast', num_fast_embs))
+        att_masks += [0] * num_lang_embs

        embs = torch.cat(embs, dim=1)
        pad_masks = torch.cat(pad_masks, dim=1)
-        
-        # Create custom 2D attention mask
-        # Attention rules:
-        # - Images + Language: bidirectional among themselves, don't attend to subtask or FAST
-        # - Subtask: attend to images + language, causal among themselves, don't attend to FAST
-        # - FAST: attend to images + language + subtask, causal among themselves
-        att_masks = self._create_custom_attention_mask(att_mask_segments, pad_masks, bsize)
+        att_masks = torch.tensor(att_masks, dtype=torch.bool, device=pad_masks.device)

-        # # Optionally visualize the attention mask
-        # self.visualize_attention_mask(
-        #     att_mask_segments=att_mask_segments,
-        #     att_2d_masks=att_masks,
-        #     save_path="/admin/home/jade_choghari/lerobot/src/lerobot/policies/pi05/attention_mask_visualization.png",
-        #     batch_idx=0,
-        #     max_display_tokens=512  # Limit display for very long sequences
-        # )
+        bsize = pad_masks.shape[0]
+        att_masks = att_masks[None, :].expand(bsize, len(att_masks))

-        return embs, pad_masks, att_masks, total_T_images, num_subtask_embs, num_fast_embs
+        return embs, pad_masks, att_masks

    def embed_suffix(self, noisy_actions, timestep):
        """Embed noisy_actions, timestep to prepare for Expert Gemma processing."""
@@ -977,23 +697,8 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`

        return embs, pad_masks, att_masks, adarms_cond

-    #  loss_dict = self.model.forward(images, img_masks, high_level_task, tokens, masks, subtask_tokens, subtask_masks, actions, fast_action_tokens, fast_action_masks)
-    def forward(self, images, img_masks, high_level_task, high_level_task_masks, subtask_tokens, subtask_masks, actions, fast_action_tokens=None, fast_action_masks=None, noise=None, time=None) -> Tensor:
-        """Do a full training forward pass and compute the loss.
-        
-        Args:
-            images: List of image tensors
-            img_masks: List of image masks
-            high_level_task: Instruction tokens WITHOUT subtask (e.g., "High level task: X; State: Y; Subtask:")
-            high_level_task_masks: Attention masks for high_level_task
-            subtask_tokens: Subtask tokens to predict (e.g., tokens for "pick up the cup")
-            subtask_masks: Attention masks for subtask_tokens
-            actions: Ground truth actions [B, chunk_size, action_dim]
-            fast_action_tokens: Discrete action token IDs [B, max_action_tokens]
-            fast_action_masks: Padding masks for fast action tokens [B, max_action_tokens]
-            noise: Optional noise for flow matching
-            time: Optional time for flow matching
-        """
+    def forward(self, images, img_masks, tokens, masks, actions, noise=None, time=None) -> Tensor:
+        """Do a full training forward pass and compute the loss."""
        if noise is None:
            noise = self.sample_noise(actions.shape, actions.device)

@@ -1004,183 +709,23 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        x_t = time_expanded * noise + (1 - time_expanded) * actions
        u_t = noise - actions

-        # Initialize FAST loss to 0 (will be computed only if FAST tokens are provided)
-        fast_loss = torch.tensor(0.0, device=actions.device, dtype=actions.dtype)
-
-        # ========== PASS 1: Prefix with FAST tokens for subtask + FAST prediction ==========
-        # Only run this pass if FAST action tokens are provided
-        if fast_action_tokens is not None and fast_action_masks is not None:
-            # Embed prefix (images + high_level_task + subtask_tokens + FAST tokens)
-            # FAST tokens are provided as discrete token IDs
-            prefix_with_fast_embs, prefix_with_fast_pad_masks, prefix_with_fast_att_masks, total_T_images, num_subtask_embs, num_fast_embs = self.embed_prefix(
-                images, img_masks, high_level_task, subtask_tokens, high_level_task_masks, subtask_masks, 
-                fast_action_tokens=fast_action_tokens, fast_action_masks=fast_action_masks
-            )
-
-            # Convert embeddings to bfloat16 if needed for the model
-            if (
-                self.paligemma_with_expert.paligemma.language_model.layers[0].self_attn.q_proj.weight.dtype
-                == torch.bfloat16
-            ):
-                prefix_with_fast_embs = prefix_with_fast_embs.to(dtype=torch.bfloat16)
-
-            # Prepare attention masks for prefix pass with FAST tokens
-            position_ids_prefix_with_fast = torch.cumsum(prefix_with_fast_pad_masks, dim=1) - 1
-            att_2d_prefix_with_fast_4d = self._prepare_attention_masks_4d(prefix_with_fast_att_masks, dtype=prefix_with_fast_embs.dtype)
-
-            # Forward pass through paligemma for subtask + FAST prediction
-            (prefix_with_fast_out, _), _ = self.paligemma_with_expert.forward(
-                attention_mask=att_2d_prefix_with_fast_4d,
-                position_ids=position_ids_prefix_with_fast,
-                past_key_values=None,
-                inputs_embeds=[prefix_with_fast_embs, None],  # SUFFIX = None
-                use_cache=False,
-                adarms_cond=[None, None],
-            )
-
-            # LM HEAD → SUBTASK LOGITS
-            lm_head = self.paligemma_with_expert.paligemma.lm_head
-            logits = lm_head(prefix_with_fast_out)  # (B, T_prefix_with_fast, vocab)
-
-            # Extract logits for subtask token prediction
-            T_high_level_task = high_level_task.size(1)
-            T_subtask = subtask_tokens.size(1)
-            start_index = total_T_images + T_high_level_task
-            end_index = start_index + T_subtask
-            logits_subtask = logits[:, start_index-1:end_index-1, :]  # (B, T_subtask, vocab)
-
-            targets = subtask_tokens  # (B, T_subtask)
-            # Compute cross-entropy loss for subtask
-            loss_fct = torch.nn.CrossEntropyLoss(reduction='none')
-            logits_flat = logits_subtask.reshape(-1, logits_subtask.size(-1))
-            targets_flat = targets.reshape(-1)
-            loss_per_token = loss_fct(logits_flat, targets_flat)
-            loss_per_token = loss_per_token.reshape(targets.shape)
-            masked_loss = loss_per_token * subtask_masks.float()
-            subtask_loss = masked_loss.sum() / subtask_masks.sum().clamp(min=1)
-
-            # Extract outputs for FAST action token prediction and compute auxiliary loss
-            # FAST outputs start after subtask tokens
-            # Similar to subtask, we use autoregressive prediction where position i predicts token i+1
-            fast_start_index = end_index
-            fast_end_index = fast_start_index + num_fast_embs
-            
-            # Get logits for FAST action tokens using the FAST LM head
-            fast_logits = self.fast_action_lm_head(prefix_with_fast_out)  # (B, T_prefix_with_fast, fast_vocab_size)
-            
-            # Extract logits for FAST token prediction (autoregressive: position i predicts token i+1)
-            # - Position (fast_start_index-1) predicts fast_action_tokens[0]
-            # - Position (fast_start_index) predicts fast_action_tokens[1], etc.
-            fast_logits_for_pred = fast_logits[:, fast_start_index-1:fast_end_index-1, :]  # (B, max_action_tokens, fast_vocab_size)
-            
-            # Compute cross-entropy loss for FAST action tokens
-            fast_targets = fast_action_tokens  # (B, max_action_tokens)
-            loss_fct_fast = torch.nn.CrossEntropyLoss(reduction='none')
-            fast_logits_flat = fast_logits_for_pred.reshape(-1, fast_logits_for_pred.size(-1))  # (B*max_action_tokens, fast_vocab_size)
-            fast_targets_flat = fast_targets.reshape(-1)  # (B*max_action_tokens)
-            
-            fast_loss_per_token = loss_fct_fast(fast_logits_flat, fast_targets_flat)  # (B*max_action_tokens)
-            fast_loss_per_token = fast_loss_per_token.reshape(fast_targets.shape)  # (B, max_action_tokens)
-            
-            # Apply mask and compute mean loss over valid tokens
-            masked_fast_loss = fast_loss_per_token * fast_action_masks.float()
-            fast_loss = masked_fast_loss.sum() / fast_action_masks.sum().clamp(min=1)
-        else:
-            # If no FAST tokens provided, compute subtask loss without FAST tokens
-            # This is the fallback for backward compatibility
-            prefix_embs_for_subtask, prefix_pad_masks_for_subtask, prefix_att_masks_for_subtask, total_T_images, _, _ = self.embed_prefix(
-                images, img_masks, high_level_task, subtask_tokens, high_level_task_masks, subtask_masks,
-                fast_action_tokens=None, fast_action_masks=None
-            )
-            
-            # Convert embeddings to bfloat16 if needed for the model
-            if (
-                self.paligemma_with_expert.paligemma.language_model.layers[0].self_attn.q_proj.weight.dtype
-                == torch.bfloat16
-            ):
-                prefix_embs_for_subtask = prefix_embs_for_subtask.to(dtype=torch.bfloat16)
-            
-            position_ids_prefix = torch.cumsum(prefix_pad_masks_for_subtask, dim=1) - 1
-            att_2d_prefix_4d = self._prepare_attention_masks_4d(prefix_att_masks_for_subtask, dtype=prefix_embs_for_subtask.dtype)
-            
-            (prefix_out, _), _ = self.paligemma_with_expert.forward(
-                attention_mask=att_2d_prefix_4d,
-                position_ids=position_ids_prefix,
-                past_key_values=None,
-                inputs_embeds=[prefix_embs_for_subtask, None],
-                use_cache=False,
-                adarms_cond=[None, None],
-            )
-            
-            lm_head = self.paligemma_with_expert.paligemma.lm_head
-            logits = lm_head(prefix_out)
-            
-            T_high_level_task = high_level_task.size(1)
-            T_subtask = subtask_tokens.size(1)
-            start_index = total_T_images + T_high_level_task
-            end_index = start_index + T_subtask
-            logits_subtask = logits[:, start_index-1:end_index-1, :]
-            
-            targets = subtask_tokens
-            loss_fct = torch.nn.CrossEntropyLoss(reduction='none')
-            logits_flat = logits_subtask.reshape(-1, logits_subtask.size(-1))
-            targets_flat = targets.reshape(-1)
-            loss_per_token = loss_fct(logits_flat, targets_flat)
-            loss_per_token = loss_per_token.reshape(targets.shape)
-            masked_loss = loss_per_token * subtask_masks.float()
-            subtask_loss = masked_loss.sum() / subtask_masks.sum().clamp(min=1)
-
-        # ========== PASS 2: Full forward WITHOUT FAST tokens for flow matching ==========
-        # Embed prefix WITHOUT FAST tokens (images + high_level_task + subtask_tokens)
-        prefix_embs_no_fast, prefix_pad_masks_no_fast, prefix_att_masks_no_fast, _, _, _ = self.embed_prefix(
-            images, img_masks, high_level_task, subtask_tokens, high_level_task_masks, subtask_masks, 
-            fast_action_tokens=None, fast_action_masks=None
-        )
-        
+        prefix_embs, prefix_pad_masks, prefix_att_masks = self.embed_prefix(images, img_masks, tokens, masks)
        suffix_embs, suffix_pad_masks, suffix_att_masks, adarms_cond = self.embed_suffix(x_t, time)

-        # Convert embeddings to bfloat16 if needed for the model
        if (
            self.paligemma_with_expert.paligemma.language_model.layers[0].self_attn.q_proj.weight.dtype
            == torch.bfloat16
        ):
            suffix_embs = suffix_embs.to(dtype=torch.bfloat16)
-            prefix_embs_no_fast = prefix_embs_no_fast.to(dtype=torch.bfloat16)
+            prefix_embs = prefix_embs.to(dtype=torch.bfloat16)

-        # For the flow matching pass, we need custom attention where:
-        # - prefix follows the custom pattern (images+lang bidirectional, subtask causal, no cross-attention)
-        # - suffix attends to all prefix + causal to itself
-        # We'll construct this by extending prefix_att_masks_no_fast to include suffix
-        
-        # prefix_att_masks_no_fast is already a 2D boolean mask [B, prefix_len, prefix_len]
-        # We need to extend it to [B, prefix_len + suffix_len, prefix_len + suffix_len]
-        
-        bsize = prefix_pad_masks_no_fast.shape[0]
-        prefix_len = prefix_pad_masks_no_fast.shape[1]
-        suffix_len = suffix_pad_masks.shape[1]
-        total_len = prefix_len + suffix_len
-        device = prefix_pad_masks_no_fast.device
-        
-        # Create full attention mask
-        full_att_2d_masks = torch.zeros(bsize, total_len, total_len, dtype=torch.bool, device=device)
-        
-        # Copy prefix attention pattern
-        full_att_2d_masks[:, :prefix_len, :prefix_len] = prefix_att_masks_no_fast
-        
-        # Suffix attends to all prefix
-        full_att_2d_masks[:, prefix_len:, :prefix_len] = True
-        
-        # Suffix has causal attention among itself
-        suffix_causal_mask = torch.tril(torch.ones(suffix_len, suffix_len, dtype=torch.bool, device=device))
-        full_att_2d_masks[:, prefix_len:, prefix_len:] = suffix_causal_mask[None, :, :]
-        
-        # Apply padding masks
-        pad_masks = torch.cat([prefix_pad_masks_no_fast, suffix_pad_masks], dim=1)
-        pad_2d_masks = pad_masks[:, None, :] * pad_masks[:, :, None]
-        full_att_2d_masks = full_att_2d_masks & pad_2d_masks
-        
+        pad_masks = torch.cat([prefix_pad_masks, suffix_pad_masks], dim=1)
+        att_masks = torch.cat([prefix_att_masks, suffix_att_masks], dim=1)
+
+        att_2d_masks = make_att_2d_masks(pad_masks, att_masks)
        position_ids = torch.cumsum(pad_masks, dim=1) - 1
-        att_2d_masks_4d = self._prepare_attention_masks_4d(full_att_2d_masks, dtype=prefix_embs_no_fast.dtype)
+
+        att_2d_masks_4d = self._prepare_attention_masks_4d(att_2d_masks)

        def forward_func(prefix_embs, suffix_embs, att_2d_masks_4d, position_ids, adarms_cond):
            (_, suffix_out), _ = self.paligemma_with_expert.forward(
@@ -1194,7 +739,7 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
            return suffix_out

        suffix_out = self._apply_checkpoint(
-            forward_func, prefix_embs_no_fast, suffix_embs, att_2d_masks_4d, position_ids, adarms_cond
+            forward_func, prefix_embs, suffix_embs, att_2d_masks_4d, position_ids, adarms_cond
        )

        suffix_out = suffix_out[:, -self.config.chunk_size :]
@@ -1205,87 +750,7 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`

        v_t = self._apply_checkpoint(action_out_proj_func, suffix_out)

-        fm_loss = F.mse_loss(u_t, v_t, reduction="none")
-
-        return {
-            "flow_loss": fm_loss,
-            "subtask_loss": subtask_loss,
-            "fast_loss": fast_loss,
-            "loss": fm_loss.mean() + 0.1 * subtask_loss + 0.05 * fast_loss, # ref: b1k winner
-        }
-    
-    @torch.no_grad()
-    def _generate_subtask_tokens(
-        self, images, img_masks, tokens, masks, tokenizer, max_length, device
-    ):
-        bsize = tokens.shape[0]
-        lm_head = self.paligemma_with_expert.paligemma.lm_head
-        
-        prefix_embs, prefix_pad_masks, prefix_att_masks, total_T_images, _, _ = self.embed_prefix(
-            images, img_masks, tokens, subtask_tokens=None, masks=masks, subtask_masks=None, 
-            fast_action_tokens=None, fast_action_masks=None
-        )
-
-        generated_tokens = torch.zeros((bsize, max_length), dtype=torch.long, device=device)
-        
-        # tracking mask: False = still generating, True = finished
-        finished = torch.zeros(bsize, dtype=torch.bool, device=device)
-        
-        for t in range(max_length):
-            position_ids_prefix = torch.cumsum(prefix_pad_masks, dim=1) - 1
-            att_2d_prefix_4d = self._prepare_attention_masks_4d(prefix_att_masks, dtype=prefix_embs.dtype)
-
-            (prefix_out, _), _ = self.paligemma_with_expert.forward(
-                attention_mask=att_2d_prefix_4d, 
-                position_ids=position_ids_prefix,
-                inputs_embeds=[prefix_embs, None],
-                # ...
-            )
-            
-            logits = lm_head(prefix_out)
-            next_token_logits = logits[:, -1, :]
-            next_token = torch.argmax(next_token_logits, dim=-1) # (B,)
-            
-            # 1. if a row was already finished, force the next token to be PAD (0)
-            next_token = torch.where(finished, torch.tensor(0, device=device), next_token)
-            
-            # 2. store the token
-            generated_tokens[:, t] = next_token
-            
-            # 3. update the finished mask
-            if tokenizer.eos_token_id is not None:
-                finished |= (next_token == tokenizer.eos_token_id)
-            
-            # 4. break only if everyone is finished
-            if finished.all():
-                break
-                
-            next_token_unsqueezed = next_token.unsqueeze(1)
-            
-            def next_token_embed_func(next_token_unsqueezed):
-                next_emb = self.paligemma_with_expert.embed_language_tokens(next_token_unsqueezed)
-                return next_emb * math.sqrt(next_emb.shape[-1])
-            
-            next_emb = self._apply_checkpoint(next_token_embed_func, next_token_unsqueezed)
-            
-            # update embeddings
-            prefix_embs = torch.cat([prefix_embs, next_emb], dim=1)
-            
-            # update padding masks
-            prefix_pad_masks = torch.cat([
-                prefix_pad_masks,
-                torch.ones((bsize, 1), dtype=torch.bool, device=device)
-            ], dim=1)
-            
-            # update attention masks
-            old_seq_len = prefix_att_masks.shape[1]
-            new_seq_len = old_seq_len + 1
-            new_att_masks = torch.zeros((bsize, new_seq_len, new_seq_len), dtype=torch.bool, device=device)
-            new_att_masks[:, :old_seq_len, :old_seq_len] = prefix_att_masks
-            new_att_masks[:, -1, :] = prefix_pad_masks
-            prefix_att_masks = new_att_masks
-
-        return generated_tokens
+        return F.mse_loss(u_t, v_t, reduction="none")

    @torch.no_grad()  # see openpi `sample_actions` (slightly adapted)
    def sample_actions(
@@ -1296,8 +761,6 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        masks,
        noise=None,
        num_steps=None,
-        tokenizer=None,
-        max_subtask_tokens=50,
        **kwargs: Unpack[ActionSelectKwargs],
    ) -> Tensor:
        """Do a full inference forward and compute the action."""
@@ -1316,41 +779,11 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
            )  # Use config max_action_dim for internal processing
            noise = self.sample_noise(actions_shape, device)

-        # Generate subtask tokens autoregressively during inference
-        generated_subtask_tokens = None
-        if tokenizer is not None:
-            generated_subtask_tokens = self._generate_subtask_tokens(
-                images, img_masks, tokens, masks, tokenizer, max_subtask_tokens, device
-            )
-            
-            # Decode and print the generated subtask tokens
-            for i in range(bsize):
-                # Remove padding tokens (0) and special tokens
-                valid_tokens = generated_subtask_tokens[i][generated_subtask_tokens[i] != 0]
-                decoded_text = tokenizer.decode(valid_tokens, skip_special_tokens=True)
-                print(f"[Inference] Generated subtask {i}: {decoded_text}")
-
-        # Create mask for generated tokens (all valid)
-        subtask_masks = torch.ones_like(generated_subtask_tokens, dtype=torch.bool)
-
-        # During inference, we don't have subtask_tokens yet, so pass None
-        # Also no FAST tokens during inference
-        prefix_embs, prefix_pad_masks, prefix_att_masks, _, _, _ = self.embed_prefix(
-            images, img_masks, tokens, subtask_tokens=generated_subtask_tokens, masks=masks, subtask_masks=subtask_masks, 
-            fast_action_tokens=None, fast_action_masks=None
-        )
-        
-        # Convert embeddings to bfloat16 if needed for the model
-        if (
-            self.paligemma_with_expert.paligemma.language_model.layers[0].self_attn.q_proj.weight.dtype
-            == torch.bfloat16
-        ):
-            prefix_embs = prefix_embs.to(dtype=torch.bfloat16)
-        
-        # prefix_att_masks is already a 2D attention mask from embed_prefix
+        prefix_embs, prefix_pad_masks, prefix_att_masks = self.embed_prefix(images, img_masks, tokens, masks)
+        prefix_att_2d_masks = make_att_2d_masks(prefix_pad_masks, prefix_att_masks)
        prefix_position_ids = torch.cumsum(prefix_pad_masks, dim=1) - 1

-        prefix_att_2d_masks_4d = self._prepare_attention_masks_4d(prefix_att_masks, dtype=prefix_embs.dtype)
+        prefix_att_2d_masks_4d = self._prepare_attention_masks_4d(prefix_att_2d_masks)
        self.paligemma_with_expert.paligemma.language_model.config._attn_implementation = "eager"  # noqa: SLF001

        _, past_key_values = self.paligemma_with_expert.forward(
@@ -1362,16 +795,13 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        )

        dt = -1.0 / num_steps
-        dt = torch.tensor(dt, dtype=torch.float32, device=device)

        x_t = noise
-        time = torch.tensor(1.0, dtype=torch.float32, device=device)
-        while time >= -dt / 2:
-            expanded_time = time.expand(bsize)
+        for step in range(num_steps):
+            time = 1.0 + step * dt
+            time_tensor = torch.tensor(time, dtype=torch.float32, device=device).expand(bsize)

-            # Define a closure function to properly capture expanded_time
-            # This avoids the lambda expression (E731) and loop variable binding (B023) issues
-            def denoise_step_partial_call(input_x_t, current_timestep=expanded_time):
+            def denoise_step_partial_call(input_x_t, current_timestep=time_tensor):
                return self.denoise_step(
                    prefix_pad_masks=prefix_pad_masks,
                    past_key_values=past_key_values,
@@ -1395,15 +825,11 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
            else:
                v_t = denoise_step_partial_call(x_t)

-            # Euler step
-            x_t += dt * v_t
+            x_t = x_t + dt * v_t

-            # Record x_t and v_t after Euler step
            if self.rtc_processor is not None and self.rtc_processor.is_debug_enabled():
                self.rtc_processor.track(time=time, x_t=x_t, v_t=v_t)

-            time += dt
-
        return x_t

    def denoise_step(
@@ -1427,7 +853,7 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
        prefix_offsets = torch.sum(prefix_pad_masks, dim=-1)[:, None]
        position_ids = prefix_offsets + torch.cumsum(suffix_pad_masks, dim=1) - 1

-        full_att_2d_masks_4d = self._prepare_attention_masks_4d(full_att_2d_masks, dtype=suffix_embs.dtype)
+        full_att_2d_masks_4d = self._prepare_attention_masks_4d(full_att_2d_masks)
        self.paligemma_with_expert.gemma_expert.model.config._attn_implementation = "eager"  # noqa: SLF001

        outputs_embeds, _ = self.paligemma_with_expert.forward(
@@ -1454,6 +880,7 @@ class PI05Policy(PreTrainedPolicy):
    def __init__(
        self,
        config: PI05Config,
+        **kwargs,
    ):
        """
        Args:
@@ -1472,14 +899,6 @@ class PI05Policy(PreTrainedPolicy):
            self.model.gradient_checkpointing_enable()

        self.model.to(config.device)
-        
-        # Load tokenizer for subtask decoding
-        try:
-            from transformers import AutoTokenizer
-            self.tokenizer = AutoTokenizer.from_pretrained("google/paligemma-3b-pt-224")
-        except Exception as e:
-            logging.warning(f"Could not load tokenizer for subtask decoding: {e}")
-            self.tokenizer = None

        self.reset()

@@ -1575,7 +994,7 @@ class PI05Policy(PreTrainedPolicy):
                print(f"Remapped {remap_count} state dict keys")

            # Load the remapped state dict into the model
-            missing_keys, unexpected_keys = model.load_state_dict(remapped_state_dict, strict=False)
+            missing_keys, unexpected_keys = model.load_state_dict(remapped_state_dict, strict=strict)

            if missing_keys:
                print(f"Missing keys when loading state dict: {len(missing_keys)} keys")
@@ -1780,16 +1199,10 @@ class PI05Policy(PreTrainedPolicy):

        # Prepare inputs
        images, img_masks = self._preprocess_images(batch)
-        # Use high_level_task tokens (WITHOUT subtask) for inference - we'll generate the subtask
-        high_level_task = batch[f"{OBS_LANGUAGE_HIGH_LEVEL_TASK_TOKENS}"]
-        high_level_task_masks = batch[f"{OBS_LANGUAGE_HIGH_LEVEL_TASK_ATTENTION_MASK}"]
-        
+        tokens, masks = batch[f"{OBS_LANGUAGE_TOKENS}"], batch[f"{OBS_LANGUAGE_ATTENTION_MASK}"]
+
        # Sample actions using the model (pass through RTC kwargs, no separate state needed for PI05)
-        actions = self.model.sample_actions(
-            images, img_masks, high_level_task, high_level_task_masks, 
-            tokenizer=self.tokenizer,
-            **kwargs
-        )
+        actions = self.model.sample_actions(images, img_masks, tokens, masks, **kwargs)

        # Unpad actions to actual action dimension
        original_action_dim = self.config.output_features[ACTION].shape[0]
@@ -1797,47 +1210,39 @@ class PI05Policy(PreTrainedPolicy):

        return actions

-    def forward(self, batch: dict[str, Tensor]) -> tuple[Tensor, dict]:
-        """Run the batch through the model and compute the loss for training."""
+    def forward(self, batch: dict[str, Tensor], reduction: str = "mean") -> tuple[Tensor, dict]:
+        """Run the batch through the model and compute the loss for training.

+        Args:
+            batch: Training batch containing observations and actions.
+            reduction: How to reduce the loss. Options:
+                - "mean": Return scalar mean loss (default, backward compatible)
+                - "none": Return per-sample losses of shape (batch_size,) for RA-BC weighting
+        """
        # Prepare inputs
        images, img_masks = self._preprocess_images(batch)
-        high_level_task = batch[f"{OBS_LANGUAGE_HIGH_LEVEL_TASK_TOKENS}"]
-        high_level_task_masks = batch[f"{OBS_LANGUAGE_HIGH_LEVEL_TASK_ATTENTION_MASK}"]
-        subtask_tokens, subtask_masks = batch[f"{OBS_LANGUAGE_SUBTASK_ONLY_TOKENS}"], batch[f"{OBS_LANGUAGE_SUBTASK_ONLY_ATTENTION_MASK}"]
-        actions = self.prepare_action(batch)
-        
-        # Decode and print ground truth subtask tokens during training
-        # if self.tokenizer is not None and self.training:
-        #     bsize = subtask_tokens.shape[0]
-        #     for i in range(bsize):
-        #         # Remove padding tokens (0) and special tokens
-        #         valid_tokens = subtask_tokens[i][subtask_masks[i].bool()]
-        #         # if len(valid_tokens) > 0:
-        #             # decoded_text = self.tokenizer.decode(valid_tokens, skip_special_tokens=True)
-        #             # print(f"[Training] Ground truth subtask {i}: {decoded_text}")
-        
-        # Get FAST action tokens from batch
-        fast_action_tokens = batch.get("action.tokens", None)  # (B, max_action_tokens)
-        fast_action_masks = batch.get("action.token_mask", None)  # (B, max_action_tokens)
-        # Compute loss (no separate state needed for PI05)
-        # high_level_task = instruction tokens WITHOUT subtask (e.g., "High level task: X; State: Y; Subtask:")
-        # subtask_tokens = subtask tokens to predict (e.g., "pick up the cup")
-        # fast_action_tokens = discrete action token IDs to predict
-        loss_dict = self.model.forward(
-            images, img_masks, high_level_task, high_level_task_masks, subtask_tokens, subtask_masks, actions,
-            fast_action_tokens=fast_action_tokens, fast_action_masks=fast_action_masks
-        )
+        tokens, masks = batch[f"{OBS_LANGUAGE_TOKENS}"], batch[f"{OBS_LANGUAGE_ATTENTION_MASK}"]

-        # Extract the total loss
-        loss = loss_dict["loss"]
-        
-        # Prepare detailed loss dictionary for logging
-        detailed_loss_dict = {
-            "loss": loss.item(),
-            "flow_loss": loss_dict["flow_loss"].mean().item(),
-            "subtask_loss": loss_dict["subtask_loss"].item(),
-            "fast_loss": loss_dict["fast_loss"].item(),
+        actions = self.prepare_action(batch)
+
+        # Compute loss (no separate state needed for PI05)
+        losses = self.model.forward(images, img_masks, tokens, masks, actions)
+
+        # Truncate losses to actual action dimensions
+        original_action_dim = self.config.output_features[ACTION].shape[0]
+        losses = losses[:, :, :original_action_dim]
+
+        loss_dict = {
+            "loss_per_dim": losses.mean(dim=[0, 1]).detach().cpu().numpy().tolist(),
        }

-        return loss, detailed_loss_dict
+        if reduction == "none":
+            # Return per-sample losses (B,) by averaging over time and action dims
+            per_sample_loss = losses.mean(dim=(1, 2))
+            loss_dict["loss"] = per_sample_loss.mean().item()
+            return per_sample_loss, loss_dict
+        else:
+            # Default: return scalar mean loss
+            loss = losses.mean()
+            loss_dict["loss"] = loss.item()
+            return loss, loss_dict
@@ -33,7 +33,6 @@ from lerobot.processor import (
    ProcessorStep,
    ProcessorStepRegistry,
    RenameObservationsProcessorStep,
-    ActionTokenizerProcessorStep,
    TokenizerProcessorStep,
    UnnormalizerProcessorStep,
 )
@@ -48,15 +47,13 @@ from lerobot.utils.constants import (

@ProcessorStepRegistry.register(name="pi05_prepare_state_tokenizer_processor_step")
@dataclass
-class Pi05PrepareStateAndLanguageTokenizerProcessorStep(ProcessorStep):
+class Pi05PrepareStateTokenizerProcessorStep(ProcessorStep):
    """
    Processor step to prepare the state and tokenize the language input.
    """

    max_state_dim: int = 32
    task_key: str = "task"
-    high_level_task_key: str = "user_prompt"
-    subtask_only_key: str = "subtask"

    def __call__(self, transition: EnvTransition) -> EnvTransition:
        transition = transition.copy()
@@ -67,8 +64,6 @@ class Pi05PrepareStateAndLanguageTokenizerProcessorStep(ProcessorStep):
        tasks = transition.get(TransitionKey.COMPLEMENTARY_DATA, {}).get(self.task_key)
        if tasks is None:
            raise ValueError("No task found in complementary data")
-        
-        high_level_tasks = transition.get(TransitionKey.COMPLEMENTARY_DATA, {}).get(self.high_level_task_key)

        # TODO: check if this necessary
        state = deepcopy(state)
@@ -81,42 +76,16 @@ class Pi05PrepareStateAndLanguageTokenizerProcessorStep(ProcessorStep):
        state_np = state.cpu().numpy()
        discretized_states = np.digitize(state_np, bins=np.linspace(-1, 1, 256 + 1)[:-1]) - 1

-        # Clean high level tasks first (if available)
-        cleaned_high_level_tasks = []
-        if high_level_tasks is not None:
-            for high_level_task in high_level_tasks:
-                cleaned_high_level_tasks.append(high_level_task.strip().replace("_", " ").replace("\n", " "))
-        
-        # Process low level tasks with state information
-        low_level_prompts = []
-        subtask_only_prompts = []  # Store only the subtask text for prediction
+        full_prompts = []
        for i, task in enumerate(tasks):
            cleaned_text = task.strip().replace("_", " ").replace("\n", " ")
            state_str = " ".join(map(str, discretized_states[i]))
-            
-            # Store only the subtask text (used as prediction target)
-            subtask_only_prompts.append(cleaned_text)
-            
-            if cleaned_high_level_tasks:
-                cleaned_high_level_task = cleaned_high_level_tasks[i]
-                full_prompt = f"High level task: {cleaned_high_level_task}; State: {state_str}; Subtask: {cleaned_text}"
-            else:
-                full_prompt = f"Task: {cleaned_text}, State: {state_str};\nAction: "
-            
-            low_level_prompts.append(full_prompt)
+            full_prompt = f"Task: {cleaned_text}, State: {state_str};\nAction: "
+            full_prompts.append(full_prompt)

-        transition[TransitionKey.COMPLEMENTARY_DATA][self.task_key] = low_level_prompts
-        transition[TransitionKey.COMPLEMENTARY_DATA][self.subtask_only_key] = subtask_only_prompts
-        
-        # Process high level tasks without state information (if available)
-        if high_level_tasks is not None:
-            high_level_prompts = []
-            for i, cleaned_high_level_task in enumerate(cleaned_high_level_tasks):
-                state_str = " ".join(map(str, discretized_states[i]))
-                full_prompt = f"High level task: {cleaned_high_level_task}; State: {state_str}; Subtask:"
-                high_level_prompts.append(full_prompt)
-            
-            transition[TransitionKey.COMPLEMENTARY_DATA][self.high_level_task_key] = high_level_prompts
+        transition[TransitionKey.COMPLEMENTARY_DATA][self.task_key] = full_prompts
+        # Normalize state to [-1, 1] range if needed (assuming it's already normalized by normalizer processor step!!)
+        # Discretize into 256 bins (see openpi `PaligemmaTokenizer.tokenize()`)
        return transition

    def transform_features(
@@ -159,27 +128,25 @@ def make_pi05_pre_post_processors(
    Returns:
        A tuple containing the configured pre-processor and post-processor pipelines.
    """
+
    # Add remaining processors
    input_steps: list[ProcessorStep] = [
        RenameObservationsProcessorStep(rename_map={}),  # To mimic the same processor as pretrained one
        AddBatchDimensionProcessorStep(),
-        # NOTE: NormalizerProcessorStep MUST come before Pi05PrepareStateAndLanguageTokenizerProcessorStep
+        # NOTE: NormalizerProcessorStep MUST come before Pi05PrepareStateTokenizerProcessorStep
        # because the tokenizer step expects normalized state in [-1, 1] range for discretization
        NormalizerProcessorStep(
            features={**config.input_features, **config.output_features},
            norm_map=config.normalization_mapping,
            stats=dataset_stats,
        ),
-        Pi05PrepareStateAndLanguageTokenizerProcessorStep(max_state_dim=config.max_state_dim),
+        Pi05PrepareStateTokenizerProcessorStep(max_state_dim=config.max_state_dim),
        TokenizerProcessorStep(
            tokenizer_name="google/paligemma-3b-pt-224",
            max_length=config.tokenizer_max_length,
            padding_side="right",
            padding="max_length",
        ),
-        ActionTokenizerProcessorStep(
-            tokenizer_name="/fsx/jade_choghari/outputs/fast_tokenizer", # TODO: jade put the PI
-        ),
        DeviceProcessorStep(device=config.device),
    ]

@@ -189,7 +156,7 @@ def make_pi05_pre_post_processors(
        ),
        DeviceProcessorStep(device="cpu"),
    ]
-    
+
    return (
        PolicyProcessorPipeline[dict[str, Any], dict[str, Any]](
            steps=input_steps,
@@ -1,22 +0,0 @@
-export CUDA_LAUNCH_BLOCKING=1 
-lerobot-train \
-    --dataset.repo_id=local \
-    --dataset.root=/fsx/jade_choghari/outputs/collect-data-pgen \
-    --output_dir=/fsx/jade_choghari/outputs/pi0_fast_fruit1 \
-    --job_name=pi0_training \
-    --policy.repo_id=jade_choghari/pi0-base1 \
-    --policy.path=lerobot/pi05_base \
-    --policy.dtype=bfloat16 \
-    --steps=200000 \
-    --save_freq=5000 \
-    --rename_map='{
-        "observation.images.base": "observation.images.base_0_rgb",
-        "observation.images.left_wrist": "observation.images.left_wrist_0_rgb",
-        "observation.images.right_wrist": "observation.images.right_wrist_0_rgb",
-        }' \
-    --batch_size=4 \
-    --policy.device=cuda \
-    --wandb.enable=true \
-    --wandb.disable_artifact=true \
-    --wandb.project=pi05hi-training \
-# /fsx/jade_choghari/.cache/huggingface/lerobot/jadechoghari/collect-data
@@ -1,18 +0,0 @@
-rm -rf /fsx/jade_choghari/outputs/pi0_multi_training
-lerobot-train \
-    --dataset.repo_id=local\
-    --dataset.root=/fsx/jade_choghari/outputs/collect-data-pgen \
-    --output_dir=/fsx/jade_choghari/outputs/pi0_multi_training \
-    --job_name=pi0_multi_training \
-    --policy.repo_id=jadechoghari/pi0-base1 \
-    --policy.path=lerobot/pi05_base \
-    --policy.dtype=bfloat16 \
-    --steps=50000 \
-    --save_freq=5000 \
-    --rename_map='{
-        "observation.images.base": "observation.images.base_0_rgb",
-        "observation.images.left_wrist": "observation.images.left_wrist_0_rgb",
-        "observation.images.right_wrist": "observation.images.right_wrist_0_rgb",
-        }' \
-    --batch_size=32 \
-    --policy.device=cuda \
@@ -1,9 +0,0 @@
-python src/lerobot/policies/pi05/train_fast_tokenizer.py \
-    --repo_id "local" \
-    --root "/fsx/jade_choghari/outputs/collect-data-pgen" \
-    --action_horizon 16 \
-    --encoded_dims "0:15" \
-    --action_horizon 50 \
-    --vocab_size 1024 \
-    --scale 10.0 \
-    --output_dir "/fsx/jade_choghari/outputs/fast_tokenizer"
@@ -1,410 +0,0 @@
-"""Train FAST tokenizer for action encoding.
-
-This script:
-1. Loads action chunks from LeRobotDataset (with sampling)
-2. Applies delta transforms and per-timestamp normalization
-3. Trains FAST tokenizer on specified action dimensions
-4. Saves tokenizer to assets directory
-5. Reports compression statistics
-"""
-
-import json
-import numpy as np
-import tyro
-from pathlib import Path
-from transformers import AutoProcessor
-import torch
-
-from lerobot.datasets.lerobot_dataset import LeRobotDataset
-
-
-def apply_delta_transform(state: np.ndarray, actions: np.ndarray, delta_dims: list[int] | None) -> np.ndarray:
-    """Apply delta transform to specified dimensions.
-    
-    Args:
-        state: Current state [D]
-        actions: Future actions [D]
-        delta_dims: List of dimension indices to apply delta transform to
-    
-    Returns:
-        Transformed actions [D]
-    """
-    if delta_dims is None or len(delta_dims) == 0:
-        return actions
-    
-    delta_actions = actions.copy()
-    for dim in delta_dims:
-        delta_actions[dim] = actions[dim] - state[dim]
-    
-    return delta_actions
-
-
-def process_episode(args):
-    """Process single episode and return action chunks."""
-    dataset, ep_idx, action_horizon, delta_dims, sample_fraction, state_key, use_delta_transform = args
-    
-    try:
-        # Get episode info
-        ep_info = dataset.meta.episodes[ep_idx]
-        from_idx = ep_info["dataset_from_index"]
-        to_idx = ep_info["dataset_to_index"]
-        ep_length = to_idx - from_idx
-        
-        if ep_length < action_horizon:
-            return None
-        
-        # Load all frames in episode
-        # If dataset has episode filtering, we need to use the mapping
-        states = []
-        actions = []
-        
-        for abs_idx in range(from_idx, to_idx):
-            # Map absolute index to relative index if needed
-            if dataset._absolute_to_relative_idx is not None:
-                if abs_idx not in dataset._absolute_to_relative_idx:
-                    # This episode's frames aren't in the filtered dataset
-                    return None
-                rel_idx = dataset._absolute_to_relative_idx[abs_idx]
-            else:
-                rel_idx = abs_idx
-            
-            frame = dataset.hf_dataset[rel_idx]
-            
-            # Get state (could be from observation.state or other state key)
-            if state_key in frame:
-                state = frame[state_key].numpy() if torch.is_tensor(frame[state_key]) else np.array(frame[state_key])
-            else:
-                # If no state key, use zeros (no delta transform)
-                state = np.zeros_like(frame["action"].numpy() if torch.is_tensor(frame["action"]) else np.array(frame["action"]))
-            
-            action = frame["action"].numpy() if torch.is_tensor(frame["action"]) else np.array(frame["action"])
-            
-            states.append(state)
-            actions.append(action)
-        
-        states = np.array(states)
-        actions = np.array(actions)
-        
-        # Create action chunks (sliding window)
-        # All actions in a chunk are relative to the FIRST state in that chunk
-        action_chunks = []
-        
-        for i in range(len(states) - action_horizon + 1):
-            current_state = states[i]  # First state in chunk
-            future_absolute_actions = actions[i:i + action_horizon]
-            
-            if use_delta_transform:
-                # Relative actions
-                delta_chunk = np.zeros_like(future_absolute_actions)
-                for t in range(action_horizon):
-                    delta_chunk[t] = apply_delta_transform(
-                        current_state,
-                        future_absolute_actions[t],
-                        delta_dims,
-                    )
-                action_chunks.append(delta_chunk)
-            else:
-                # Absolute actions (NO delta)
-                action_chunks.append(future_absolute_actions)
-        
-        if len(action_chunks) == 0:
-            return None
-        
-        action_chunks = np.array(action_chunks)
-        
-        # Sample chunks
-        if sample_fraction < 1.0:
-            n_chunks = len(action_chunks)
-            n_samples = max(1, int(n_chunks * sample_fraction))
-            episode_seed = hash(ep_idx) % (2**31)
-            rng = np.random.RandomState(episode_seed)
-            indices = rng.choice(n_chunks, size=n_samples, replace=False)
-            action_chunks = action_chunks[indices]
-        
-        return action_chunks
-        
-    except Exception as e:
-        print(f"Error processing episode {ep_idx}: {e}")
-        import traceback
-        traceback.print_exc()
-        return None
-
-
-def train_fast_tokenizer(
-    action_chunks: np.ndarray,
-    vocab_size: int = 1024,
-    scale: float = 10.0,
-) -> AutoProcessor:
-    """
-    Train FAST tokenizer (BPE on DCT coefficients) on action chunks.
-    
-    Uses the .fit() method to train a new tokenizer on the provided data.
-    
-    Args:
-        action_chunks: Array of action chunks [N, H, D] where N=num_chunks, H=horizon, D=action_dim
-        vocab_size: BPE vocabulary size
-        scale: DCT scaling factor for quantization
-    
-    Returns:
-        Trained FAST tokenizer
-    """
-    print(f"Training FAST tokenizer on {len(action_chunks)} action chunks...")
-    print(f"Action chunk shape: {action_chunks.shape}")
-    print(f"Vocab size: {vocab_size}")
-    print(f"DCT scale: {scale}")
-    
-    # Download the tokenizer source code (not pretrained weights)
-    # We'll train a new tokenizer on our own data
-    base_tokenizer = AutoProcessor.from_pretrained(
-        "physical-intelligence/fast",
-        trust_remote_code=True
-    )
-    
-    # Convert action_chunks array to list of arrays (expected by .fit())
-    action_data_list = [action_chunks[i] for i in range(len(action_chunks))]
-    
-    # Train the new tokenizer on our action data using .fit()
-    # This trains the BPE tokenizer on DCT coefficients
-    print("Training new tokenizer (this may take a few minutes)...")
-    tokenizer = base_tokenizer.fit(
-        action_data_list,
-        scale=scale,
-        vocab_size=vocab_size,
-        time_horizon=action_chunks.shape[1],  # action_horizon
-        action_dim=action_chunks.shape[2],     # encoded dimensions
-    )
-    print("✓ Tokenizer training complete!")
-    
-    # Validate it works
-    sample_chunk = action_chunks[0]
-    encoded = tokenizer(sample_chunk[None])[0]
-    if isinstance(encoded, list):
-        encoded = np.array(encoded)
-    print(f"Sample encoding: {len(encoded)} tokens for chunk shape {sample_chunk.shape}")
-    
-    return tokenizer
-
-
-def compute_compression_stats(tokenizer, action_chunks: np.ndarray):
-    """Compute compression statistics."""
-    print("\nComputing compression statistics...")
-    
-    # Sample for stats (use max 1000 chunks for speed)
-    sample_size = min(1000, len(action_chunks))
-    sample_indices = np.random.RandomState(42).choice(len(action_chunks), size=sample_size, replace=False)
-    sample_chunks = action_chunks[sample_indices]
-    
-    token_lengths = []
-    for chunk in sample_chunks:
-        encoded = tokenizer(chunk[None])[0]
-        if isinstance(encoded, list):
-            token_lengths.append(len(encoded))
-        else:
-            token_lengths.append(encoded.shape[0] if hasattr(encoded, 'shape') else len(encoded))
-    
-    token_lengths = np.array(token_lengths)
-    
-    # Compression ratio: (H * D) / avg_tokens
-    input_size = action_chunks.shape[1] * action_chunks.shape[2]
-    avg_tokens = np.mean(token_lengths)
-    compression_ratio = input_size / avg_tokens
-    
-    stats = {
-        'compression_ratio': float(compression_ratio),
-        'mean_token_length': float(np.mean(token_lengths)),
-        'p99_token_length': float(np.percentile(token_lengths, 99)),
-        'min_token_length': float(np.min(token_lengths)),
-        'max_token_length': float(np.max(token_lengths)),
-    }
-    
-    print(f"Compression Statistics:")
-    print(f"  Average compression ratio: {stats['compression_ratio']:.2f}x")
-    print(f"  Mean token length: {stats['mean_token_length']:.1f}")
-    print(f"  P99 token length: {stats['p99_token_length']:.0f}")
-    print(f"  Min token length: {stats['min_token_length']:.0f}")
-    print(f"  Max token length: {stats['max_token_length']:.0f}")
-    
-    return stats
-
-
-def main(
-    repo_id: str,
-    root: str | None = None,
-    action_horizon: int = 10,
-    max_episodes: int | None = None,
-    sample_fraction: float = 0.1,
-    encoded_dims: str = "0:6,7:23",
-    delta_dims: str | None = None,
-    use_delta_transform: bool = False,
-    state_key: str = "observation.state",
-    vocab_size: int = 1024,
-    scale: float = 10.0,
-    output_dir: str | None = None,
-):
-    """
-    Train FAST tokenizer for action encoding.
-    
-    Args:
-        repo_id: LeRobot dataset repository ID
-        root: Root directory for dataset (default: ~/.cache/huggingface/lerobot)
-        action_horizon: Number of future actions in each chunk
-        max_episodes: Max episodes to use (None = all episodes in dataset)
-        sample_fraction: Fraction of chunks to sample per episode
-        encoded_dims: Comma-separated dimension ranges to encode (e.g., "0:6,7:23")
-        delta_dims: Comma-separated dimension indices for delta transform (e.g., "0,1,2,3,4,5")
-        use_delta_transform: Whether to apply delta transform (relative actions vs absolute actions)
-        state_key: Dataset key for state observations (default: "observation.state")
-        vocab_size: FAST vocabulary size (BPE vocab size)
-        scale: DCT scaling factor (default: 10.0)
-        output_dir: Directory to save tokenizer (default: ./fast_tokenizer_{repo_id})
-    """
-    # Load dataset
-    print(f"Loading dataset: {repo_id}")
-    dataset = LeRobotDataset(repo_id=repo_id, root=root)
-    print(f"Dataset loaded: {dataset.num_episodes} episodes, {dataset.num_frames} frames")
-    
-    # Parse encoded dimensions
-    encoded_dim_ranges = []
-    for range_str in encoded_dims.split(','):
-        start, end = map(int, range_str.strip().split(':'))
-        encoded_dim_ranges.append((start, end))
-    
-    total_encoded_dims = sum(end - start for start, end in encoded_dim_ranges)
-    print(f"Encoding {total_encoded_dims} dimensions: {encoded_dims}")
-    
-    # Parse delta dimensions
-    delta_dim_list = None
-    if delta_dims is not None and delta_dims.strip():
-        delta_dim_list = [int(d.strip()) for d in delta_dims.split(',')]
-        print(f"Delta dimensions: {delta_dim_list}")
-    else:
-        print("No delta dimensions specified")
-    
-    print(f"Use delta transform: {use_delta_transform}")
-    if use_delta_transform and (delta_dim_list is None or len(delta_dim_list) == 0):
-        print("Warning: use_delta_transform=True but no delta_dims specified. No delta will be applied.")
-    
-    print(f"Action horizon: {action_horizon}")
-    print(f"State key: {state_key}")
-    
-    # Determine episodes to process
-    num_episodes = dataset.num_episodes
-    if max_episodes is not None:
-        num_episodes = min(max_episodes, num_episodes)
-    
-    print(f"Processing {num_episodes} episodes...")
-    
-    # Process episodes sequentially (to avoid pickling issues with dataset)
-    all_chunks = []
-    for ep_idx in range(num_episodes):
-        if ep_idx % 10 == 0:
-            print(f"  Processing episode {ep_idx}/{num_episodes}...")
-        
-        chunks = process_episode(
-            (dataset, ep_idx, action_horizon, delta_dim_list, sample_fraction, state_key, use_delta_transform)
-        )
-        if chunks is not None:
-            all_chunks.append(chunks)
-    
-    # Concatenate all chunks
-    all_chunks = np.concatenate(all_chunks, axis=0)
-    print(f"Collected {len(all_chunks)} action chunks")
-    
-    # Extract only encoded dimensions FIRST (before normalization)
-    encoded_chunks = []
-    for start, end in encoded_dim_ranges:
-        encoded_chunks.append(all_chunks[:, :, start:end])
-    encoded_chunks = np.concatenate(encoded_chunks, axis=-1)  # [N, H, D_encoded]
-    print(f"Extracted {encoded_chunks.shape[-1]} encoded dimensions")
-    
-    # Apply normalization to encoded dimensions only
-    # NOTE: For FAST, we ALWAYS use QUANTILE normalization (no per-timestamp)
-    # This clips outliers and provides consistent [-1, 1] range for DCT compression
-    print(f"\nBefore normalization - overall stats:")
-    print(f"  Min: {np.min(encoded_chunks):.4f}, Max: {np.max(encoded_chunks):.4f}")
-    print(f"  Mean: {np.mean(encoded_chunks):.4f}, Std: {np.std(encoded_chunks):.4f}")
-    
-    norm_stats = dataset.meta.stats
-    if norm_stats is not None and "action" in norm_stats:
-        action_stats = norm_stats["action"]
-        
-        # Build encoded dimension indices
-        encoded_dim_indices = []
-        for start, end in encoded_dim_ranges:
-            encoded_dim_indices.extend(range(start, end))
-        encoded_dim_indices = np.array(encoded_dim_indices)
-        
-        # Use QUANTILE normalization: clip to [q01, q99] and map to [-1, 1]
-        if "q01" in action_stats and "q99" in action_stats:
-            q01 = np.array(action_stats["q01"])[encoded_dim_indices]  # [D_encoded]
-            q99 = np.array(action_stats["q99"])[encoded_dim_indices]  # [D_encoded]
-            
-            print(f"\nNormalization stats (q01, q99) for encoded dimensions:")
-            for i, dim_idx in enumerate(encoded_dim_indices):
-                print(f"  Orig dim {dim_idx}: q01={q01[i]:7.4f}, q99={q99[i]:7.4f}, range={q99[i]-q01[i]:7.4f}")
-            
-            # Clip to quantile range and normalize to [-1, 1]
-            encoded_chunks = np.clip(encoded_chunks, q01, q99)
-            encoded_chunks = 2.0 * (encoded_chunks - q01) / np.maximum(q99 - q01, 1e-6) - 1.0
-            print(f"\nApplied quantile normalization [q01, q99] → [-1, 1]")
-            
-            print(f"\nAfter normalization - overall stats:")
-            print(f"  Min: {np.min(encoded_chunks):.4f}, Max: {np.max(encoded_chunks):.4f}")
-            print(f"  Mean: {np.mean(encoded_chunks):.4f}, Std: {np.std(encoded_chunks):.4f}")
-            
-            print(f"\nPer-dimension stats (after normalization):")
-            for d in range(encoded_chunks.shape[-1]):
-                dim_data = encoded_chunks[:, :, d]
-                print(f"  Dim {d}: min={np.min(dim_data):7.4f}, max={np.max(dim_data):7.4f}, "
-                      f"mean={np.mean(dim_data):7.4f}, std={np.std(dim_data):7.4f}")
-        else:
-            print("Warning: q01/q99 stats not found, using raw actions")
-    else:
-        print("Warning: No normalization stats found, using raw actions")
-    
-    print(f"Encoded chunks shape: {encoded_chunks.shape}")
-    
-    # Train FAST tokenizer
-    tokenizer = train_fast_tokenizer(
-        encoded_chunks,
-        vocab_size=vocab_size,
-        scale=scale,
-    )
-    
-    # Compute compression statistics
-    compression_stats = compute_compression_stats(tokenizer, encoded_chunks)
-    
-    # Save tokenizer
-    if output_dir is None:
-        output_dir = f"fast_tokenizer_{repo_id.replace('/', '_')}"
-    output_path = Path(output_dir)
-    output_path.mkdir(parents=True, exist_ok=True)
-    
-    tokenizer.save_pretrained(output_path)
-    
-    # Save metadata
-    metadata = {
-        'repo_id': repo_id,
-        'vocab_size': vocab_size,
-        'scale': scale,
-        'encoded_dims': encoded_dims,
-        'encoded_dim_ranges': encoded_dim_ranges,
-        'total_encoded_dims': total_encoded_dims,
-        'delta_dims': delta_dims,
-        'delta_dim_list': delta_dim_list,
-        'use_delta_transform': use_delta_transform,
-        'state_key': state_key,
-        'action_horizon': action_horizon,
-        'num_training_chunks': len(encoded_chunks),
-        'compression_stats': compression_stats,
-    }
-    
-    with open(output_path / "metadata.json", 'w') as f:
-        json.dump(metadata, f, indent=2)
-    
-    print(f"\n✅ Saved FAST tokenizer to {output_path}")
-    print(f"Metadata: {json.dumps(metadata, indent=2)}")
-
-
-if __name__ == "__main__":
-    tyro.cli(main)
@@ -1,101 +0,0 @@
-# Train FAST Tokenizer - Usage Examples
-
-This script trains a FAST (Factorized Action Sequence Tokenizer) on LeRobotDataset action data.
-
-## Basic Usage
-
-```bash
-python src/lerobot/policies/pi05/train_fast_tokenizer.py \
-    --repo_id "lerobot/aloha_sim_insertion_human" \
-    --action_horizon 10 \
-    --encoded_dims "0:7" \
-    --vocab_size 1024 \
-    --scale 10.0
-```
-
-## Parameters
-
-### Required
- `--repo_id`: LeRobot dataset repository ID (e.g., "lerobot/aloha_sim_insertion_human")
-
-### Optional
- `--root`: Root directory for dataset (default: ~/.cache/huggingface/lerobot)
- `--action_horizon`: Number of future actions in each chunk (default: 10)
- `--max_episodes`: Maximum number of episodes to use (default: None = all)
- `--sample_fraction`: Fraction of chunks to sample per episode (default: 0.1)
- `--encoded_dims`: Comma-separated dimension ranges to encode (default: "0:6,7:23")
-  - Example: "0:7" encodes dimensions 0-6
-  - Example: "0:3,6:9" encodes dimensions 0-2 and 6-8
- `--delta_dims`: Comma-separated dimension indices for delta transform (default: None)
-  - Example: "0,1,2,3,4,5" applies delta transform to first 6 dimensions
-  - Delta transform: action[i] - state[i] for specified dimensions
- `--state_key`: Dataset key for state observations (default: "observation.state")
- `--vocab_size`: FAST vocabulary size / BPE vocab size (default: 1024)
- `--scale`: DCT scaling factor (default: 10.0)
- `--output_dir`: Directory to save tokenizer (default: ./fast_tokenizer_{repo_id})
-
-## Examples
-
-### Example 1: Train on full action space
-
-```bash
-python src/lerobot/policies/pi05/train_fast_tokenizer.py \
-    --repo_id "lerobot/pusht" \
-    --action_horizon 16 \
-    --encoded_dims "0:2" \
-    --vocab_size 512 \
-    --max_episodes 100
-```
-
-### Example 2: Train with delta transform
-
-```bash
-python src/lerobot/policies/pi05/train_fast_tokenizer.py \
-    --repo_id "lerobot/aloha_sim_insertion_human" \
-    --action_horizon 10 \
-    --encoded_dims "0:14" \
-    --delta_dims "0,1,2,3,4,5,6,7,8,9,10,11,12,13" \
-    --state_key "observation.state" \
-    --vocab_size 1024 \
-    --scale 10.0 \
-    --sample_fraction 0.2
-```
-
-### Example 3: Train on subset of dimensions
-
-```bash
-python src/lerobot/policies/pi05/train_fast_tokenizer.py \
-    --repo_id "lerobot/aloha_sim_insertion_human" \
-    --action_horizon 10 \
-    --encoded_dims "0:7" \
-    --vocab_size 1024 \
-    --output_dir "./my_tokenizer"
-```
-
-## Output
-
-The script saves:
-1. **Tokenizer files**: Trained FAST tokenizer (can be loaded with `AutoProcessor.from_pretrained()`)
-2. **metadata.json**: Contains:
-   - Configuration parameters
-   - Compression statistics (compression ratio, token lengths)
-   - Training dataset information
-
-## Understanding the Process
-
-1. **Load Dataset**: Loads the LeRobotDataset from HuggingFace
-2. **Extract Action Chunks**: Creates sliding windows of actions with specified horizon
-3. **Apply Delta Transform**: (Optional) Computes action deltas relative to current state
-4. **Select Encoded Dimensions**: Extracts only the dimensions to be encoded
-5. **Normalize**: Applies quantile normalization ([q01, q99] → [-1, 1])
-6. **Train Tokenizer**: Trains BPE tokenizer on DCT coefficients
-7. **Compute Stats**: Reports compression ratio and token length statistics
-8. **Save**: Saves tokenizer and metadata
-
-## Notes
-
- **Normalization**: The script uses quantile normalization (q01, q99) from the dataset's statistics
- **Sampling**: To speed up training, you can sample a fraction of chunks per episode
- **Delta Transform**: Applied per-dimension to make actions relative to current state
- **Compression**: FAST uses DCT + BPE to compress action sequences efficiently
-
@@ -1,23 +0,0 @@
-rm -rf /fsx/jade_choghari/outputs/pi0_multi_training
-accelerate launch --multi_gpu --num_processes=2 \
-    $(which lerobot-train) \
-    --dataset.repo_id=local \
-    --dataset.root=/fsx/jade_choghari/outputs/collect-data-pgen \
-    --output_dir=/fsx/jade_choghari/outputs/pi0_multi_training \
-    --job_name=pi0_multi_training \
-    --policy.repo_id=jadechoghari/pi0-base1 \
-    --policy.path=lerobot/pi05_base \
-    --policy.dtype=bfloat16 \
-    --steps=50000 \
-    --save_freq=5000 \
-    --rename_map='{
-        "observation.images.base": "observation.images.base_0_rgb",
-        "observation.images.left_wrist": "observation.images.left_wrist_0_rgb",
-        "observation.images.right_wrist": "observation.images.right_wrist_0_rgb",
-        }' \
-    --policy.gradient_checkpointing=true \
-    --batch_size=1 \
-    --policy.device=cpu
-    # --wandb.enable=true \
-    # --wandb.disable_artifact=true \
-    # --wandb.project=pi05hi-training \
@@ -206,6 +206,7 @@ class PreTrainedPolicy(nn.Module, HubMixin, abc.ABC):
    def push_model_to_hub(
        self,
        cfg: TrainPipelineConfig,
+        peft_model=None,
    ):
        api = HfApi()
        repo_id = api.create_repo(
@@ -216,7 +217,14 @@ class PreTrainedPolicy(nn.Module, HubMixin, abc.ABC):
        with TemporaryDirectory(ignore_cleanup_errors=True) as tmp:
            saved_path = Path(tmp) / repo_id

-            self.save_pretrained(saved_path)  # Calls _save_pretrained and stores model tensors
+            if peft_model is not None:
+                # Since PEFT just forwards calls to `push_model_to_hub`, `self` is not the PeftModel wrapper
+                # but the actual policy which is why we need the PEFT model passed to us to save the adapter.
+                # That also means that we need to store the policy config ourselves since PEFT can't.
+                peft_model.save_pretrained(saved_path)
+                self.config.save_pretrained(saved_path)
+            else:
+                self.save_pretrained(saved_path)  # Calls _save_pretrained and stores model tensors

            card = self.generate_model_card(
                cfg.dataset.repo_id, self.config.type, self.config.license, self.config.tags
@@ -0,0 +1,14 @@
+## Paper
+
+https://arxiv.org/abs/2509.25358
+
+## Citation
+
+```bibtex
+@article{chen2025sarm,
+  title={SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulation},
+  author={Chen, Qianzhong and Yu, Justin and Schwager, Mac and Abbeel, Pieter and Shentu, Yide and Wu, Philipp},
+  journal={arXiv preprint arXiv:2509.25358},
+  year={2025}
+}
+```
@@ -0,0 +1,870 @@
+#!/usr/bin/env python
+
+# Copyright 2024 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Compute SARM progress values for RA-BC (Reward-Aware Behavior Cloning) weighting.
+
+This script processes all frames in a dataset with SARM to compute progress values [0, 1].
+The results are saved as a parquet file that can be loaded during training for RA-BC weighting.
+
+Uses multi-output extraction: each SARM query returns progress for 9 frames, so we only
+need ~num_frames/30 queries instead of one per frame (~30x speedup).
+
+Usage:
+    # Full RA-BC computation with visualizations
+    python src/lerobot/policies/sarm/compute_rabc_weights.py \\
+        --dataset-repo-id lerobot/aloha_sim_insertion_human \\
+        --reward-model-path pepijn223/sarm_single_uni4
+
+    # Faster computation with stride (compute every 5 frames, interpolate the rest)
+    python src/lerobot/policies/sarm/compute_rabc_weights.py \\
+        --dataset-repo-id lerobot/aloha_sim_insertion_human \\
+        --reward-model-path pepijn223/sarm_single_uni4 \\
+        --stride 5
+
+    # Visualize predictions only (no RA-BC computation)
+    python src/lerobot/policies/sarm/compute_rabc_weights.py \\
+        --dataset-repo-id lerobot/aloha_sim_insertion_human \\
+        --reward-model-path pepijn223/sarm_single_uni4 \\
+        --visualize-only \\
+        --num-visualizations 5
+
+The output is saved to the dataset's local cache directory as 'sarm_progress.parquet'.
+"""
+
+import argparse
+import logging
+from pathlib import Path
+
+import matplotlib.gridspec as gridspec
+import matplotlib.pyplot as plt
+import numpy as np
+import pyarrow as pa
+import pyarrow.parquet as pq
+import torch
+from tqdm import tqdm
+
+from lerobot.datasets.lerobot_dataset import LeRobotDataset
+from lerobot.policies.sarm.modeling_sarm import SARMRewardModel
+from lerobot.policies.sarm.processor_sarm import make_sarm_pre_post_processors
+from lerobot.policies.sarm.sarm_utils import normalize_stage_tau
+
+
+def get_reward_model_path_from_parquet(parquet_path: Path) -> str | None:
+    """Read reward_model_path from parquet metadata if available."""
+    if not parquet_path.exists():
+        return None
+    try:
+        metadata = pq.read_metadata(parquet_path).schema.to_arrow_schema().metadata
+        if metadata and b"reward_model_path" in metadata:
+            return metadata[b"reward_model_path"].decode()
+    except Exception:  # nosec B110
+        return None
+    return None
+
+
+def load_sarm_resources(
+    dataset_repo_id: str,
+    reward_model_path: str,
+    device: str = "cuda",
+) -> tuple[LeRobotDataset, SARMRewardModel, any]:
+    """
+    Load SARM model, dataset, and preprocessor.
+
+    Returns:
+        Tuple of (dataset, reward_model, preprocessor)
+    """
+    logging.info(f"Loading model: {reward_model_path}")
+    reward_model = SARMRewardModel.from_pretrained(reward_model_path)
+    reward_model.config.device = device
+    reward_model.to(device).eval()
+
+    image_key = reward_model.config.image_key
+    state_key = reward_model.config.state_key
+    delta_indices = reward_model.config.observation_delta_indices
+
+    logging.info(f"Loading dataset: {dataset_repo_id}")
+    temp_dataset = LeRobotDataset(dataset_repo_id, download_videos=True)
+    fps = temp_dataset.fps
+
+    delta_timestamps = {
+        image_key: [idx / fps for idx in delta_indices],
+        state_key: [idx / fps for idx in delta_indices],
+    }
+    dataset = LeRobotDataset(dataset_repo_id, delta_timestamps=delta_timestamps)
+    logging.info(f"Dataset: {dataset.num_episodes} episodes, {dataset.num_frames} frames")
+
+    preprocess, _ = make_sarm_pre_post_processors(
+        config=reward_model.config,
+        dataset_stats=dataset.meta.stats,
+        dataset_meta=dataset.meta,
+    )
+
+    return dataset, reward_model, preprocess
+
+
+def to_numpy_image(img) -> np.ndarray:
+    """Convert image tensor to numpy uint8 (H, W, C)."""
+    if isinstance(img, torch.Tensor):
+        img = img.cpu().numpy()
+    if img.ndim == 4:
+        # Take center frame for bidirectional sampling
+        img = img[img.shape[0] // 2]
+    if img.shape[0] in [1, 3]:
+        img = np.transpose(img, (1, 2, 0))
+    if img.dtype != np.uint8:
+        # Handle normalized images (may have negative values or values > 1)
+        img = img.astype(np.float32)
+        img = (img - img.min()) / (img.max() - img.min() + 1e-8)  # Normalize to [0, 1]
+        img = (img * 255).astype(np.uint8)
+    return img
+
+
+def visualize_episode(
+    frames, progress_preds, stage_preds, title, output_path, stage_labels, gt_progress=None, gt_stages=None
+):
+    """Create visualization with progress plot, stage probabilities, and sample frames.
+
+    Same as sarm_inference_visualization.py
+    """
+    num_stages = stage_preds.shape[1]
+    colors = plt.cm.tab10(np.linspace(0, 1, num_stages))
+    frame_indices = np.arange(len(progress_preds))
+
+    fig = plt.figure(figsize=(14, 12))
+    gs = gridspec.GridSpec(3, 1, height_ratios=[2, 1, 1], hspace=0.3)
+    ax_progress, ax_stages, ax_frames = fig.add_subplot(gs[0]), fig.add_subplot(gs[1]), fig.add_subplot(gs[2])
+
+    # Progress plot
+    ax_progress.plot(frame_indices, progress_preds, linewidth=2, color="#2E86AB", label="Predicted")
+    ax_progress.fill_between(frame_indices, 0, progress_preds, alpha=0.3, color="#2E86AB")
+    if gt_progress is not None:
+        ax_progress.plot(
+            frame_indices, gt_progress, linewidth=2, color="#28A745", linestyle="--", label="Ground Truth"
+        )
+    ax_progress.axhline(y=1.0, color="gray", linestyle="--", alpha=0.5)
+    ax_progress.set_ylabel("Progress")
+    ax_progress.set_title(f'Task: "{title}"', fontweight="bold")
+    ax_progress.set_ylim(-0.05, 1.1)
+    ax_progress.legend(loc="upper left")
+    ax_progress.grid(True, alpha=0.3)
+
+    # Stage predictions
+    ax_stages.stackplot(
+        frame_indices,
+        *[stage_preds[:, i] for i in range(num_stages)],
+        colors=colors,
+        alpha=0.8,
+        labels=stage_labels,
+    )
+    if gt_stages is not None:
+        for change_idx in np.where(np.diff(gt_stages) != 0)[0] + 1:
+            ax_stages.axvline(x=change_idx, color="black", linestyle="-", alpha=0.7, linewidth=1.5)
+    ax_stages.set_xlabel("Frame")
+    ax_stages.set_ylabel("Stage Probability")
+    ax_stages.set_ylim(0, 1)
+    ax_stages.legend(loc="upper left", ncol=min(num_stages, 5), fontsize=8)
+    ax_stages.grid(True, alpha=0.3)
+
+    # Sample frames
+    ax_frames.axis("off")
+    num_sample = 8
+    sample_indices = np.linspace(0, len(frames) - 1, num_sample, dtype=int)
+    h, w = frames[0].shape[:2]
+    combined = np.zeros((h, w * num_sample, 3), dtype=np.uint8)
+    for i, idx in enumerate(sample_indices):
+        frame = frames[idx]
+        if frame.shape[-1] == 1:
+            frame = np.repeat(frame, 3, axis=-1)
+        combined[:, i * w : (i + 1) * w] = frame
+        stage_name = stage_labels[np.argmax(stage_preds[idx])][:12]
+        ax_frames.text(
+            i * w + w / 2,
+            -10,
+            f"Frame {idx}\n{progress_preds[idx]:.2f}\n{stage_name}",
+            ha="center",
+            va="top",
+            fontsize=7,
+        )
+    ax_frames.imshow(combined)
+    ax_frames.set_title("Sample Frames", pad=20)
+
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    plt.savefig(output_path, dpi=150, bbox_inches="tight")
+    plt.close()
+    print(f"Saved: {output_path}")
+
+
+def visualize_sarm_predictions(
+    dataset: LeRobotDataset,
+    reward_model: SARMRewardModel,
+    preprocess,
+    episode_indices: list[int],
+    head_mode: str,
+    output_dir: Path,
+    num_display_frames: int = 5,
+    stride: int = 1,
+):
+    """
+    Visualize SARM predictions for multiple episodes.
+
+    Computes predictions for every frame by default. With stride > 1, computes predictions
+    every N frames and interpolates (progress + stage probabilities) for visualization.
+
+    Args:
+        dataset: LeRobotDataset with delta_timestamps configured
+        reward_model: Loaded SARM model
+        preprocess: Preprocessor from make_sarm_pre_post_processors
+        episode_indices: List of episode indices to visualize
+        head_mode: "sparse", "dense", or "both"
+        output_dir: Directory to save visualizations
+        num_display_frames: Number of frames to display in thumbnail strip (default: 5)
+        stride: Compute predictions every N frames, interpolate the rest (default: 1)
+    """
+    output_dir = Path(output_dir)
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    image_key = reward_model.config.image_key
+    state_key = reward_model.config.state_key
+    dual_mode = reward_model.config.uses_dual_heads
+    device = reward_model.device
+
+    # Center frame index for bidirectional sampling
+    target_idx = reward_model.config.n_obs_steps // 2
+
+    # Determine which heads to visualize
+    schemes_to_viz = []
+    if head_mode in ("sparse", "both") or not dual_mode:
+        schemes_to_viz.append("sparse")
+    if head_mode in ("dense", "both") and dual_mode:
+        schemes_to_viz.append("dense")
+
+    # Set preprocessor to eval mode to disable augmentations
+    if hasattr(preprocess, "eval"):
+        preprocess.eval()
+    for step in preprocess.steps:
+        if hasattr(step, "eval"):
+            step.eval()
+
+    for episode_idx in episode_indices:
+        ep = dataset.meta.episodes[episode_idx]
+        ep_start = ep["dataset_from_index"]
+        ep_end = ep["dataset_to_index"]
+        task = dataset[ep_start].get("task", "perform the task")
+        num_frames = ep_end - ep_start
+
+        # Select frames for display thumbnails (evenly sampled from begin to end)
+        display_indices = set(
+            [
+                ep_start + int(i * (num_frames - 1) / (num_display_frames - 1))
+                for i in range(num_display_frames)
+            ]
+            if num_frames >= num_display_frames
+            else list(range(ep_start, ep_end))
+        )
+        viz_frames = {}
+
+        # Load display frames up-front (stride mode might skip them otherwise).
+        for frame_idx in display_indices:
+            sample = dataset[frame_idx]
+            viz_frames[frame_idx] = to_numpy_image(sample[image_key])
+
+        # Initialize storage for each scheme
+        scheme_data = {}
+        for scheme in schemes_to_viz:
+            num_stages = getattr(reward_model.config, f"num_{scheme}_stages")
+            scheme_data[scheme] = {
+                "viz_progress": np.full(num_frames, np.nan),
+                "viz_stages": np.full((num_frames, num_stages), np.nan),
+                "viz_gt_progress": np.full(num_frames, np.nan),
+                "viz_gt_stages": np.full(num_frames, np.nan),
+                "target_key": f"{scheme}_targets",
+                "num_stages": num_stages,
+                "temporal_props": getattr(reward_model.config, f"{scheme}_temporal_proportions"),
+                "subtask_names": getattr(reward_model.config, f"{scheme}_subtask_names"),
+            }
+
+        if stride > 1:
+            logging.info(f"Visualization stride={stride}: inferring every {stride} frames and interpolating")
+
+        # Process frames one at a time to avoid memory buildup
+        frame_indices = list(range(ep_start, ep_end, stride))
+        if (ep_end - 1) not in frame_indices:
+            frame_indices.append(ep_end - 1)
+        frame_indices = sorted(set(frame_indices))
+
+        for frame_idx in tqdm(frame_indices, desc=f"Episode {episode_idx}", leave=False):
+            local_idx = frame_idx - ep_start
+            sample = dataset[frame_idx]
+
+            batch = {
+                image_key: sample[image_key],
+                "task": task,
+                "index": frame_idx,
+                "episode_index": episode_idx,
+            }
+            if state_key in sample:
+                batch[state_key] = sample[state_key]
+
+            with torch.no_grad():
+                processed = preprocess(batch)
+                video_features = processed["video_features"].to(device)
+                text_features = processed["text_features"].to(device)
+                state_features = processed.get("state_features")
+                if state_features is not None:
+                    state_features = state_features.to(device)
+                lengths = processed.get("lengths")
+
+                for scheme in schemes_to_viz:
+                    sd = scheme_data[scheme]
+
+                    # Ground truth
+                    # In stride visualization mode, ground-truth plots can be misleading
+                    # (only sparse points are available), so we skip GT.
+                    if stride == 1 and sd["target_key"] in processed:
+                        gt_target = processed[sd["target_key"]][0, target_idx].cpu().item()
+                        sd["viz_gt_stages"][local_idx] = int(gt_target)
+                        sd["viz_gt_progress"][local_idx] = normalize_stage_tau(
+                            gt_target,
+                            num_stages=sd["num_stages"],
+                            temporal_proportions=sd["temporal_props"],
+                            subtask_names=sd["subtask_names"],
+                        )
+
+                    # Predictions
+                    reward, stage_probs = reward_model.calculate_rewards(
+                        text_embeddings=text_features,
+                        video_embeddings=video_features,
+                        state_features=state_features,
+                        lengths=lengths,
+                        return_all_frames=True,
+                        return_stages=True,
+                        head_mode=scheme,
+                    )
+
+                    # Handle both tensor and numpy outputs
+                    if isinstance(reward, torch.Tensor):
+                        reward = reward.cpu().numpy()
+                        stage_probs = stage_probs.cpu().numpy()
+
+                    if reward.ndim == 2:
+                        sd["viz_progress"][local_idx] = reward[0, target_idx]
+                        sd["viz_stages"][local_idx] = stage_probs[0, target_idx, :]
+                    else:
+                        sd["viz_progress"][local_idx] = reward[target_idx]
+                        sd["viz_stages"][local_idx] = stage_probs[target_idx, :]
+
+                # Clear GPU memory after each frame
+                del processed, video_features, text_features
+                if state_features is not None:
+                    del state_features
+
+            torch.cuda.empty_cache()
+
+        # Interpolate predictions back to per-frame arrays for smooth visualization.
+        if stride > 1:
+            all_local = np.arange(num_frames)
+            for scheme in schemes_to_viz:
+                sd = scheme_data[scheme]
+
+                valid = np.isfinite(sd["viz_progress"])
+                valid_idx = np.where(valid)[0]
+                if valid_idx.size >= 1:
+                    sd["viz_progress"] = interpolate_progress(
+                        valid_idx, sd["viz_progress"][valid_idx], all_local
+                    )
+
+                    stage_interp = np.zeros_like(sd["viz_stages"], dtype=np.float32)
+                    for s in range(sd["num_stages"]):
+                        stage_interp[:, s] = interpolate_progress(
+                            valid_idx, sd["viz_stages"][valid_idx, s], all_local
+                        )
+
+                    stage_interp = np.clip(stage_interp, 0.0, 1.0)
+                    row_sums = stage_interp.sum(axis=1, keepdims=True)
+                    nz = row_sums.squeeze(-1) > 0
+                    stage_interp[nz] = stage_interp[nz] / row_sums[nz]
+                    sd["viz_stages"] = stage_interp
+                else:
+                    # No valid points: keep NaNs/zeros; visualization will be empty.
+                    sd["viz_stages"] = np.nan_to_num(sd["viz_stages"], nan=0.0)
+
+        # Generate visualization for each head
+        ordered_viz_frames = [viz_frames[idx] for idx in sorted(display_indices)]
+        for scheme in schemes_to_viz:
+            sd = scheme_data[scheme]
+            stage_labels = sd["subtask_names"] or [f"Stage {i + 1}" for i in range(sd["num_stages"])]
+            viz_path = output_dir / f"sarm_prediction_ep{episode_idx}_{scheme}.png"
+
+            visualize_episode(
+                frames=np.array(ordered_viz_frames),
+                progress_preds=sd["viz_progress"],
+                stage_preds=sd["viz_stages"],
+                title=f"{task} (Episode {episode_idx})",
+                output_path=viz_path,
+                stage_labels=stage_labels,
+                gt_progress=sd["viz_gt_progress"] if not np.all(np.isnan(sd["viz_gt_progress"])) else None,
+                gt_stages=sd["viz_gt_stages"] if not np.all(np.isnan(sd["viz_gt_stages"])) else None,
+            )
+
+        # Clear memory between episodes
+        torch.cuda.empty_cache()
+
+    logging.info(f"Visualizations saved to: {output_dir.absolute()}")
+
+
+def generate_all_frame_indices(ep_start: int, ep_end: int, frame_gap: int = 30) -> list[int]:
+    """Generate all frame indices, ordered by offset for cache-friendly access.
+
+    Orders frames as: [0, 30, 60...], [1, 31, 61...], ..., [29, 59, 89...]
+    This groups frames that share similar temporal windows together.
+    """
+    num_frames = ep_end - ep_start
+    indices = []
+    for offset in range(frame_gap):
+        for frame_rel in range(offset, num_frames, frame_gap):
+            indices.append(ep_start + frame_rel)
+    return indices
+
+
+def interpolate_progress(
+    computed_indices: np.ndarray,
+    computed_values: np.ndarray,
+    all_indices: np.ndarray,
+) -> np.ndarray:
+    """Linearly interpolate values to fill in gaps (robust to NaNs / edge cases)."""
+    computed_indices = np.asarray(computed_indices)
+    computed_values = np.asarray(computed_values)
+    all_indices = np.asarray(all_indices)
+
+    mask = np.isfinite(computed_values)
+    if mask.sum() == 0:
+        return np.full(all_indices.shape, np.nan, dtype=np.float32)
+    if mask.sum() == 1:
+        return np.full(all_indices.shape, float(computed_values[mask][0]), dtype=np.float32)
+
+    out = np.interp(all_indices, computed_indices[mask], computed_values[mask])
+    return out.astype(np.float32)
+
+
+def compute_sarm_progress(
+    dataset_repo_id: str,
+    reward_model_path: str,
+    output_path: str | None = None,
+    head_mode: str = "sparse",
+    device: str = "cuda",
+    num_visualizations: int = 5,
+    output_dir: str = "./sarm_viz",
+    stride: int = 1,
+):
+    """
+    Compute SARM progress predictions for all frames in a dataset.
+
+    Args:
+        dataset_repo_id: HuggingFace dataset repo ID or local path
+        reward_model_path: Path to pretrained SARM model
+        output_path: Path to save results. If None, saves to dataset's cache directory
+        head_mode: SARM head to use ("sparse", "dense", or "both")
+        device: Device to use for inference
+        num_visualizations: Number of episodes to visualize (0 to skip)
+        output_dir: Directory to save visualizations
+        stride: Compute progress every N frames, interpolate the rest (default: 1 = every frame)
+    """
+    dataset, reward_model, preprocess = load_sarm_resources(dataset_repo_id, reward_model_path, device)
+
+    # Set preprocessor to eval mode to disable augmentations
+    if hasattr(preprocess, "eval"):
+        preprocess.eval()
+    for step in preprocess.steps:
+        if hasattr(step, "eval"):
+            step.eval()
+
+    image_key = reward_model.config.image_key
+    state_key = reward_model.config.state_key
+    frame_gap = reward_model.config.frame_gap
+    num_episodes = dataset.num_episodes
+    total_frames = dataset.num_frames
+    logging.info(f"Processing {total_frames} frames across {num_episodes} episodes")
+
+    # Determine which heads to compute
+    dual_mode = reward_model.config.uses_dual_heads
+    compute_sparse = head_mode in ("sparse", "both") or not dual_mode
+    compute_dense = head_mode in ("dense", "both") and dual_mode
+
+    # Storage arrays
+    all_indices = []
+    all_episode_indices = []
+    all_frame_indices = []
+    all_progress_sparse = [] if compute_sparse else None
+    all_progress_dense = [] if compute_dense else None
+
+    if stride > 1:
+        logging.info(f"Using stride={stride}: computing every {stride} frames, interpolating the rest")
+
+    # Process all episodes
+    for episode_idx in tqdm(range(num_episodes), desc="Episodes"):
+        ep = dataset.meta.episodes[episode_idx]
+        ep_start = ep["dataset_from_index"]
+        ep_end = ep["dataset_to_index"]
+
+        # Get task description
+        task = dataset[ep_start].get("task", "perform the task")
+
+        # Generate frames to compute (with stride applied)
+        all_ep_indices = generate_all_frame_indices(ep_start, ep_end, frame_gap)
+        if stride > 1:
+            # Only compute every stride-th frame (relative to episode start)
+            compute_indices = [idx for idx in all_ep_indices if (idx - ep_start) % stride == 0]
+            # Always include last frame for better interpolation at episode end
+            last_frame = ep_end - 1
+            if last_frame not in compute_indices:
+                compute_indices.append(last_frame)
+            compute_indices = sorted(set(compute_indices))
+        else:
+            compute_indices = all_ep_indices
+
+        center_idx = reward_model.config.n_obs_steps // 2  # Center of bidirectional window
+
+        # Dictionary to collect results
+        frame_results = {}
+
+        for query_idx in tqdm(compute_indices, desc=f"  Ep {episode_idx}", leave=False):
+            try:
+                sample = dataset[query_idx]
+
+                batch = {
+                    image_key: sample[image_key],
+                    "task": task,
+                    "index": query_idx,
+                    "episode_index": episode_idx,
+                }
+                if state_key in sample:
+                    batch[state_key] = sample[state_key]
+
+                with torch.no_grad():
+                    processed = preprocess(batch)
+                    video_features = processed["video_features"].to(device)
+                    text_features = processed["text_features"].to(device)
+                    state_features = processed.get("state_features")
+                    if state_features is not None:
+                        state_features = state_features.to(device)
+                    lengths = processed.get("lengths")
+
+                    sparse_val = np.nan
+                    dense_val = np.nan
+
+                    # Compute sparse prediction for center frame
+                    if compute_sparse:
+                        sparse_progress = reward_model.calculate_rewards(
+                            text_embeddings=text_features,
+                            video_embeddings=video_features,
+                            state_features=state_features,
+                            lengths=lengths,
+                            return_all_frames=True,
+                            head_mode="sparse",
+                        )
+                        sparse_val = float(
+                            sparse_progress[0, center_idx]
+                            if sparse_progress.ndim == 2
+                            else sparse_progress[center_idx]
+                        )
+
+                    # Compute dense prediction for center frame
+                    if compute_dense:
+                        dense_progress = reward_model.calculate_rewards(
+                            text_embeddings=text_features,
+                            video_embeddings=video_features,
+                            state_features=state_features,
+                            lengths=lengths,
+                            return_all_frames=True,
+                            head_mode="dense",
+                        )
+                        dense_val = float(
+                            dense_progress[0, center_idx]
+                            if dense_progress.ndim == 2
+                            else dense_progress[center_idx]
+                        )
+
+                    frame_results[query_idx] = (sparse_val, dense_val)
+
+            except Exception as e:
+                logging.warning(f"Failed to process frame {query_idx}: {e}")
+
+        # Interpolate to get values for all frames
+        computed_indices = np.array(sorted(frame_results.keys()))
+        computed_sparse = (
+            np.array([frame_results[i][0] for i in computed_indices]) if compute_sparse else None
+        )
+        computed_dense = np.array([frame_results[i][1] for i in computed_indices]) if compute_dense else None
+
+        # All frame indices for this episode
+        all_frame_idx_array = np.arange(ep_start, ep_end)
+
+        if stride > 1 and len(computed_indices) > 1:
+            # Interpolate progress values
+            if compute_sparse:
+                interp_sparse = interpolate_progress(computed_indices, computed_sparse, all_frame_idx_array)
+            if compute_dense:
+                interp_dense = interpolate_progress(computed_indices, computed_dense, all_frame_idx_array)
+        else:
+            # No interpolation needed
+            interp_sparse = computed_sparse if compute_sparse else None
+            interp_dense = computed_dense if compute_dense else None
+
+        # Store results for all frames
+        for i, frame_idx in enumerate(all_frame_idx_array):
+            local_idx = frame_idx - ep_start
+            all_indices.append(frame_idx)
+            all_episode_indices.append(episode_idx)
+            all_frame_indices.append(local_idx)
+            if compute_sparse:
+                if stride > 1 and len(computed_indices) > 1:
+                    all_progress_sparse.append(float(interp_sparse[i]))
+                elif frame_idx in frame_results:
+                    all_progress_sparse.append(frame_results[frame_idx][0])
+                else:
+                    all_progress_sparse.append(np.nan)
+            if compute_dense:
+                if stride > 1 and len(computed_indices) > 1:
+                    all_progress_dense.append(float(interp_dense[i]))
+                elif frame_idx in frame_results:
+                    all_progress_dense.append(frame_results[frame_idx][1])
+                else:
+                    all_progress_dense.append(np.nan)
+
+    # Create output table
+    table_data = {
+        "index": np.array(all_indices, dtype=np.int64),
+        "episode_index": np.array(all_episode_indices, dtype=np.int64),
+        "frame_index": np.array(all_frame_indices, dtype=np.int64),
+    }
+    if compute_sparse:
+        table_data["progress_sparse"] = np.array(all_progress_sparse, dtype=np.float32)
+    if compute_dense:
+        table_data["progress_dense"] = np.array(all_progress_dense, dtype=np.float32)
+
+    # Sort by index
+    df = pa.table(table_data).to_pandas()
+    df = df.sort_values("index").reset_index(drop=True)
+    final_table = pa.Table.from_pandas(df, preserve_index=False)
+
+    # Add metadata with reward model path
+    metadata = {b"reward_model_path": reward_model_path.encode()}
+    final_table = final_table.replace_schema_metadata(metadata)
+
+    # Determine output path
+    output_path = Path(dataset.root) / "sarm_progress.parquet" if output_path is None else Path(output_path)
+
+    # Save
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+    pq.write_table(final_table, output_path)
+    logging.info(f"Saved {len(final_table)} frame progress values to {output_path}")
+
+    # Print statistics
+    if "progress_sparse" in df.columns:
+        valid = df["progress_sparse"].dropna()
+        logging.info(
+            f"Sparse progress: mean={valid.mean():.4f}, std={valid.std():.4f}, "
+            f"min={valid.min():.4f}, max={valid.max():.4f}"
+        )
+
+    if "progress_dense" in df.columns:
+        valid = df["progress_dense"].dropna()
+        logging.info(
+            f"Dense progress: mean={valid.mean():.4f}, std={valid.std():.4f}, "
+            f"min={valid.min():.4f}, max={valid.max():.4f}"
+        )
+
+    # Visualize episodes after processing
+    if num_visualizations > 0:
+        viz_episodes = list(range(min(num_visualizations, num_episodes)))
+        logging.info(f"Generating {len(viz_episodes)} visualizations...")
+        visualize_sarm_predictions(
+            dataset=dataset,
+            reward_model=reward_model,
+            preprocess=preprocess,
+            episode_indices=viz_episodes,
+            head_mode=head_mode,
+            output_dir=Path(output_dir),
+            stride=stride,
+        )
+
+    return output_path
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Compute SARM progress values for RA-BC weighting or visualize SARM predictions",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Examples:
+    # Full RA-BC computation with visualizations
+    python src/lerobot/policies/sarm/compute_rabc_weights.py \\
+        --dataset-repo-id lerobot/aloha_sim_insertion_human \\
+        --reward-model-path pepijn223/sarm_single_uni4
+
+    # Visualize predictions only (no RA-BC computation)
+    python src/lerobot/policies/sarm/compute_rabc_weights.py \\
+        --dataset-repo-id lerobot/aloha_sim_insertion_human \\
+        --reward-model-path pepijn223/sarm_single_uni4 \\
+        --visualize-only \\
+        --num-visualizations 10
+        """,
+    )
+    parser.add_argument(
+        "--dataset-repo-id",
+        type=str,
+        required=True,
+        help="HuggingFace dataset repo ID or local path",
+    )
+    parser.add_argument(
+        "--reward-model-path",
+        type=str,
+        default=None,
+        help="Path to pretrained SARM model (reads from existing parquet metadata if not provided)",
+    )
+    parser.add_argument(
+        "--output-path",
+        type=str,
+        default=None,
+        help="Output path for parquet. If not set, saves to dataset's cache directory",
+    )
+    parser.add_argument(
+        "--head-mode",
+        type=str,
+        default="sparse",
+        choices=["sparse", "dense", "both"],
+        help="SARM head to use (default: sparse)",
+    )
+    parser.add_argument(
+        "--device",
+        type=str,
+        default="cuda",
+        help="Device to use (default: cuda)",
+    )
+    # Visualization options
+    parser.add_argument(
+        "--visualize-only",
+        action="store_true",
+        help="Only visualize SARM predictions (no RA-BC computation)",
+    )
+    parser.add_argument(
+        "--num-visualizations",
+        type=int,
+        default=5,
+        help="Number of episodes to visualize (default: 5, set to 0 to skip)",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default="./sarm_viz",
+        help="Output directory for visualizations (default: ./sarm_viz)",
+    )
+    parser.add_argument(
+        "--push-to-hub",
+        action="store_true",
+        help="Upload progress file to the dataset repo on HuggingFace Hub",
+        default=True,
+    )
+    parser.add_argument(
+        "--stride",
+        type=int,
+        default=1,
+        help="Compute progress every N frames, interpolate the rest (default: 1 = every frame)",
+    )
+
+    args = parser.parse_args()
+
+    logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
+
+    # Try to get reward_model_path from parquet metadata if not provided
+    reward_model_path = args.reward_model_path
+    if reward_model_path is None:
+        # Load dataset to find parquet path
+        temp_dataset = LeRobotDataset(args.dataset_repo_id, download_videos=False)
+        parquet_path = Path(temp_dataset.root) / "sarm_progress.parquet"
+        reward_model_path = get_reward_model_path_from_parquet(parquet_path)
+        if reward_model_path:
+            logging.info(f"Using reward model from parquet metadata: {reward_model_path}")
+        else:
+            raise ValueError(
+                "--reward-model-path is required (no existing parquet with model metadata found)"
+            )
+
+    # Handle visualize-only mode
+    if args.visualize_only:
+        dataset, reward_model, preprocess = load_sarm_resources(
+            args.dataset_repo_id, reward_model_path, args.device
+        )
+        logging.info(f"Visualization-only mode: visualizing {args.num_visualizations} episodes")
+        viz_episodes = list(range(min(args.num_visualizations, dataset.num_episodes)))
+        visualize_sarm_predictions(
+            dataset=dataset,
+            reward_model=reward_model,
+            preprocess=preprocess,
+            episode_indices=viz_episodes,
+            head_mode=args.head_mode,
+            output_dir=Path(args.output_dir),
+            stride=args.stride,
+        )
+        print(f"\nVisualizations saved to: {Path(args.output_dir).absolute()}")
+        return
+
+    # Full RABC computation (compute_sarm_progress loads model/dataset itself)
+    output_path = compute_sarm_progress(
+        dataset_repo_id=args.dataset_repo_id,
+        reward_model_path=reward_model_path,
+        output_path=args.output_path,
+        head_mode=args.head_mode,
+        device=args.device,
+        num_visualizations=args.num_visualizations,
+        output_dir=args.output_dir,
+        stride=args.stride,
+    )
+
+    print(f"\nSARM progress values saved to: {output_path}")
+
+    # Upload to Hub if requested
+    if args.push_to_hub:
+        from huggingface_hub import HfApi
+
+        api = HfApi()
+        hub_path = "sarm_progress.parquet"
+
+        print(f"\nUploading to Hub: {args.dataset_repo_id}/{hub_path}")
+        api.upload_file(
+            path_or_fileobj=str(output_path),
+            path_in_repo=hub_path,
+            repo_id=args.dataset_repo_id,
+            repo_type="dataset",
+        )
+        print(
+            f"Successfully uploaded to: https://huggingface.co/datasets/{args.dataset_repo_id}/blob/main/{hub_path}"
+        )
+
+        print("\nTo use in training, add to your config:")
+        print("  use_rabc: true")
+        print(f"  rabc_progress_path: hf://datasets/{args.dataset_repo_id}/{hub_path}")
+        print("  rabc_head_mode: sparse  # or dense")
+    else:
+        print("\nTo use in training, add to your config:")
+        print("  use_rabc: true")
+        print(f"  rabc_progress_path: {output_path}")
+        print("  rabc_head_mode: sparse  # or dense")
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,248 @@
+#!/usr/bin/env python
+
+# Copyright 2025 Qianzhong Chen, Justin Yu, Mac Schwager, Pieter Abbeel, Yide Shentu, Philipp Wu
+# and The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulation.
+Paper: https://arxiv.org/abs/2509.25358
+"""
+
+from dataclasses import dataclass, field
+
+from lerobot.configs.policies import PreTrainedConfig
+from lerobot.configs.types import FeatureType, NormalizationMode, PolicyFeature
+from lerobot.optim.optimizers import AdamWConfig
+from lerobot.optim.schedulers import CosineDecayWithWarmupSchedulerConfig
+
+
+@PreTrainedConfig.register_subclass("sarm")
+@dataclass
+class SARMConfig(PreTrainedConfig):
+    """Configuration class for SARM (Stage-Aware Reward Modeling).
+
+    Supports three annotation modes:
+
+    1. single_stage (default): No annotations needed. Uses the episode's task description
+       as a single stage covering the entire episode.
+
+    2. dense_only: Uses dense (fine-grained) annotations from VLM, with an auto-generated
+       single sparse "task" stage covering the full episode. The dense head learns detailed
+       subtask progression while sparse provides overall task completion.
+
+    3. dual: Full dual-head mode with both sparse (high-level) and dense (fine-grained)
+       annotations from VLM. Both heads are trained on their respective annotations.
+
+    The annotation_mode determines how sparse_temporal_proportions and dense_temporal_proportions
+    are loaded/generated during model initialization.
+    """
+
+    annotation_mode: str = "single_stage"  # "single_stage", "dense_only", or "dual"
+    n_obs_steps: int = 8  # Number of observation history steps
+    frame_gap: int = 30  # Frame gap between frames (at 30 fps = 1 second)
+    max_rewind_steps: int = 4  # Maximum rewind steps for temporal augmentation
+
+    # Total frames = 1 + n_obs_steps + max_rewind_steps (computed in property)
+    # During training with rewind: [obs_frames] + [rewind_frames]
+    # During inference: [obs_frames] only
+
+    # Architecture params
+    image_dim: int = 512
+    text_dim: int = 512
+    hidden_dim: int = 768
+    num_heads: int = 12
+    num_layers: int = 8
+    max_state_dim: int = 32
+    drop_n_last_frames: int = 1
+    batch_size: int = 64
+    clip_batch_size: int = 64
+    dropout: float = 0.1
+    stage_loss_weight: float = 1.0  # Weight for stage classification loss when using subtask annotations
+
+    rewind_probability: float = 0.8
+    language_perturbation_probability: float = 0.2
+
+    # Sparse annotations (high-level stages)
+    num_sparse_stages: int = 1
+    sparse_subtask_names: list | None = None
+    sparse_temporal_proportions: list | None = None
+
+    # Dense annotations (fine-grained stages)
+    num_dense_stages: int | None = None
+    dense_subtask_names: list | None = None
+    dense_temporal_proportions: list | None = None
+
+    pretrained_model_path: str | None = None
+    device: str | None = None
+    image_key: str = "observation.images.top"  # Key for image used from the dataset
+    state_key: str = "observation.state"
+
+    # Populated by the processor (video_features, state_features, text_features)
+    input_features: dict = field(default_factory=lambda: {})
+
+    # Output features (updated in __post_init__)
+    output_features: dict = field(
+        default_factory=lambda: {
+            "stage": PolicyFeature(shape=(9, 5), type=FeatureType.REWARD),
+            "progress": PolicyFeature(shape=(9, 1), type=FeatureType.REWARD),
+        }
+    )
+
+    normalization_mapping: dict[str, NormalizationMode] = field(
+        default_factory=lambda: {
+            "VISUAL": NormalizationMode.IDENTITY,
+            "STATE": NormalizationMode.MEAN_STD,
+            "LANGUAGE": NormalizationMode.IDENTITY,
+            "REWARD": NormalizationMode.IDENTITY,
+        }
+    )
+
+    def __post_init__(self):
+        super().__post_init__()
+
+        if self.annotation_mode not in ["single_stage", "dense_only", "dual"]:
+            raise ValueError(
+                f"annotation_mode must be 'single_stage', 'dense_only', or 'dual', got {self.annotation_mode}"
+            )
+
+        if self.annotation_mode == "single_stage":
+            # Use task description as stage name, full episode as one stage
+            self.num_sparse_stages = 1
+            self.sparse_subtask_names = ["task"]
+            self.sparse_temporal_proportions = [1.0]
+            self.num_dense_stages = None
+            self.dense_subtask_names = None
+            self.dense_temporal_proportions = None
+
+        elif self.annotation_mode == "dense_only":
+            self.num_sparse_stages = 1
+            self.sparse_subtask_names = ["task"]
+            self.sparse_temporal_proportions = [1.0]
+
+        self.input_features = {}
+        self.output_features = {}
+
+        if self.image_key:
+            self.input_features[self.image_key] = PolicyFeature(shape=(480, 640, 3), type=FeatureType.VISUAL)
+
+        self.input_features[self.state_key] = PolicyFeature(
+            shape=(self.max_state_dim,),
+            type=FeatureType.STATE,
+        )
+
+        # Update output features based on annotation_mode
+        if self.annotation_mode in ["dense_only", "dual"]:
+            self.output_features["sparse_stage"] = PolicyFeature(
+                shape=(self.num_frames, self.num_sparse_stages), type=FeatureType.REWARD
+            )
+            self.output_features["sparse_progress"] = PolicyFeature(
+                shape=(self.num_frames, 1), type=FeatureType.REWARD
+            )
+            dense_stages = self.num_dense_stages or self.num_sparse_stages
+            self.output_features["dense_stage"] = PolicyFeature(
+                shape=(self.num_frames, dense_stages), type=FeatureType.REWARD
+            )
+            self.output_features["dense_progress"] = PolicyFeature(
+                shape=(self.num_frames, 1), type=FeatureType.REWARD
+            )
+        else:
+            self.output_features["sparse_stage"] = PolicyFeature(
+                shape=(self.num_frames, self.num_sparse_stages), type=FeatureType.REWARD
+            )
+            self.output_features["sparse_progress"] = PolicyFeature(
+                shape=(self.num_frames, 1), type=FeatureType.REWARD
+            )
+
+        if self.max_rewind_steps >= self.n_obs_steps:
+            raise ValueError(
+                f"max_rewind_steps ({self.max_rewind_steps}) must be less than n_obs_steps ({self.n_obs_steps})"
+            )
+        if self.num_sparse_stages < 1:
+            raise ValueError(f"num_sparse_stages must be at least 1, got {self.num_sparse_stages}")
+        if (
+            self.annotation_mode in ["dense_only", "dual"]
+            and self.num_dense_stages is not None
+            and self.num_dense_stages < 2
+        ):
+            raise ValueError(f"num_dense_stages must be at least 2, got {self.num_dense_stages}")
+
+    def get_optimizer_preset(self) -> AdamWConfig:
+        """Get default optimizer configuration for SARM training."""
+        return AdamWConfig(
+            lr=5e-5,
+            weight_decay=1e-3,
+            betas=(0.9, 0.999),
+            eps=1e-8,
+        )
+
+    def get_scheduler_preset(self) -> CosineDecayWithWarmupSchedulerConfig:
+        """Get default learning rate scheduler configuration."""
+        return CosineDecayWithWarmupSchedulerConfig(
+            peak_lr=5e-5,
+            decay_lr=5e-6,
+            num_warmup_steps=500,
+            num_decay_steps=50000,
+        )
+
+    def validate_features(self) -> None:
+        pass
+
+    @property
+    def uses_dual_heads(self) -> bool:
+        """Whether the model uses dual heads (dense_only or dual annotation modes)."""
+        return self.annotation_mode in ["dense_only", "dual"]
+
+    @property
+    def num_frames(self) -> int:
+        """Total number of frames in sequence.
+
+        For training: 1 + n_obs_steps + max_rewind_steps
+        The sequence is: [obs_frames (n_obs_steps + 1)] + [rewind_frames (max_rewind_steps)]
+        """
+        return 1 + self.n_obs_steps + self.max_rewind_steps
+
+    @property
+    def max_length(self) -> int:
+        return self.num_frames
+
+    @property
+    def observation_delta_indices(self) -> list[int]:
+        """Bidirectional frame sampling centered on target frame.
+
+        Example with n_obs_steps=8, gap=30:
+        Before: [-120, -90, -60, -30]  (4 frames)
+        Current: [0]                   (1 frame)
+        After:  [30, 60, 90, 120]      (4 frames)
+        Total: 9 frames
+        """
+        half_steps = self.n_obs_steps // 2
+
+        past_deltas = [-self.frame_gap * i for i in range(half_steps, 0, -1)]
+        future_deltas = [self.frame_gap * i for i in range(1, half_steps + 1)]
+        obs_deltas = past_deltas + [0] + future_deltas
+
+        # Rewind placeholders
+        rewind_deltas = [-self.frame_gap * (i + 1) for i in range(self.max_rewind_steps)]
+
+        return obs_deltas + rewind_deltas
+
+    @property
+    def action_delta_indices(self) -> None:
+        """SARM is a reward model, not an action policy."""
+        return None
+
+    @property
+    def reward_delta_indices(self) -> None:
+        return None
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Pepijn	56fd0ef378	run precommit and fix issues	2025-12-22 14:50:58 +01:00
Pepijn	486c52c14b	Merge branch 'main' into feature/basic-peft-support	2025-12-22 14:29:51 +01:00
Tong Wu	17c5a0774f	feat: support wallx model (#2593 ) * support wallx * fix bugs in flow * incorporate wallx model into lerobot * update the policy methods * reduce to least config and params & pass lerobot basic test * fixed dtype bugs * add wallx dependencies * update * remove flash-attn requirement && fix bug in inference and fast mode * fix bug for inference * add some small modifications * fix pre-commit errors * remove lerobot[wallx] * fix ci * fix precommit issues * fix: exclude wallx extra properly in CI workflows * fix: add uv conflicts for wallx transformers version * fix: peft test import * pre-commit * only export WallXConfig from wall_x package to avoid peft import in CI * remove torch dep * precommit * add import --------- Co-authored-by: vincentchen <chenlufang@x2robot.com> Co-authored-by: Geoffrey19 <sympathischmann35@gmail.com> Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com> Co-authored-by: Pepijn <pepijn@huggingface.co>	2025-12-22 10:12:39 +01:00
Pepijn	0071b1ff6e	Add readme (#2698 ) * Add readme * change ref	2025-12-22 10:04:33 +01:00
nemo	ad4a82b77e	Add peft as extra dependency, mark tests Fast tests currently fail because of the missing dependency.	2025-12-20 19:20:12 +01:00
nemo	4bc75776f7	Formatting	2025-12-20 19:18:56 +01:00
Clément Verrier	00b5f65752	fix(optim): enable and resolve mypy type errors (#2683 ) * fix(optim): enable and resolve mypy type errors Resolves #1729 build(deps): add mypy as dependency and update pre-commit hook * change build's type annotation	2025-12-20 17:19:42 +01:00
Francesco Capuano	2f6c870c4b	Fixes ReadMe Policies Classification (#2682 ) * fix: tdmpc is a model-based RL method, does not learn from expert demonstrations so no IL there * fix: typo * remove trailing space Signed-off-by: Francesco Capuano <74058581+fracapuano@users.noreply.github.com> * fix: minor --------- Signed-off-by: Francesco Capuano <74058581+fracapuano@users.noreply.github.com>	2025-12-20 17:11:02 +01:00
nemo	43ca1dc216	Merge remote-tracking branch 'hf/main' into feature/basic-peft-support	2025-12-19 12:23:37 +01:00
nemo	6a9dac7c20	Add basic documentation	2025-12-19 12:20:50 +01:00
nemo	54c25a4400	Disallow PEFT training on non-pretrained policies At first I thought it would make sense to have this feature in case you want to fine-tune a pre-trained section but in the end it makes more trouble than it's worth. It's still possible to allow this in the future when a concrete need arises.	2025-12-19 11:50:29 +01:00
Steven Palma	0bd1969d0a	feat(docs): modernize readme (#2660 )	2025-12-18 19:45:13 +01:00
Pepijn	f04958527e	Add sarm (#2639 ) * add initial modeling * make rewind pretrained policy * add annotation * small fix * add sarm * subtasks * fix spawn * fix rewind discrepancies * Add script to generate embedding for dataset (#2138) * Add generate and validate script * fix precommit * Improve generate embeddings function by using dataset tools (#2206) --------- Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co> * cleanup * change order train log * print batch size * update sarm processor * add reward output * change expected features * add image validation * change validation * get state input from dataset stats * raise if no state key is found * pass stats * cleanup and refactor * add episode inddex to complementary data * add subtask init and detection * revert lerobot_train changes * pass dataset metadata to policy * change loadig subtasks * add small logging * fix progress conversion and adding initial frame * use large offset for initial frame (ugly) * Remove rewind, use clip tokenizer * add tests, implement formula 1,2 correctly and cleanup * use task from dataset, cleanup visualizer * simplify * simplify and cleanup code and move compute_temporal_proportions to utils * fix normalization in visualization * Fix visualization and change prompt * fix formatting * add visualize subtask annotations * use qwen thinking * try different prompt * format * update prompt * higher temp, long output * different settings * use instruct * show full resp * split message * Temp: increase tolerance dataset * Fix RA-BC (#2572) * Add next observation loading for RA-BC progress deltas * Compute weights based on temporal progress deltas instead of static rewards * Add hard-masking for negative progress deltas in weight computation * Feat/add dual head (#2582) * Add dual dense sparse head and annotation * Add docs * add dual to procesor * cleanup * change sampling in visualize and cleanup * remove validation * remove compile * Feat/test uniform (#2587) * test uniform * add different string for misaligned * Fix rewind and add tests * uncomment text implementation * run precommit * Add head mode for ra-bc * fix visalization of single task * add * return per sample loss * Fix RA_BC (#2602) * update rabc implementation * compute rabc beforehand * fix import * add only progress calulation * use precomputed progress * multi gpu processing * import * fix dataset meta data extraction * add logging * logging * log * progress per episode * split differently * move clip to gpu * pre decode frames for an episode * fix cuda initalization * fix import * multi processing * rename * fix import * fix * fix rabc * use last known progress if oob * use last known progress if oob * add misalignment loss with random embeddings * discard previous changes * add selection of models to docs for ra_bc * add transformers dep * extend tolerance * initial commit with new codebase * add tests * fix * remove temporal sampler * drop last frame for sampler * use original ref * some fixes * fix visualization * remove smoothing and fix order subtasks * add stride rabc computation * add push to hub * add explanation * add kappa expllaination * better rabc logging * feedback pr * remove dataset tolerance * revert dataset tool * revert dataset changes * add credit * run precommit * change path for generate ra_bc * fix type * include sarm in all in pyproject * fix precommit * lazy import matplotlib * lazy import qwen * remove rich console * skip if transformers is not installed? * run only when we have faker * place transformer lazy loading * Dont test if low transformer version * fix * increase transformer * increase as 4.57.0 is yanked * remove pi from all * go back --------- Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co> Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com>	2025-12-18 12:50:32 +01:00
Steven Palma	4a151a9682	chore(ci): minor improvement bug-report template & pr auto label (#2676 ) * chore(ci): minor improvement bug-report template * chore(ci): change triggers for PR auto label	2025-12-18 00:23:23 +01:00
Steven Palma	8667b9ef08	chore(ci): minor improvements auto labeling (#2675 )	2025-12-17 22:54:47 +01:00
Steven Palma	86eee5c1e2	fix(ci): close bracket pattern (#2674 )	2025-12-17 22:40:33 +01:00
Steven Palma	469b855e42	fix(ci): better heuristic + issue type template fix (#2672 ) * fix(ci): better heuristic + issue type template fix * chore(ci): remove keywords in performance tag	2025-12-17 22:31:22 +01:00
Steven Palma	292333cafc	chore(ci): update issue template (#2666 )	2025-12-17 18:02:20 +01:00
Steven Palma	f0c98e23f1	feat(ci): simple automatic labelling (#2667 ) * ci: add pr labeler * ci: add issue labeler * ci: minor fixes for labelers * fix(ci): add explicit path for pr labeler	2025-12-17 17:52:45 +01:00
Steven Palma	7621af5acd	chore(ci): update PR template (#2665 ) * chore: update code of conduct to transformers one * chore: update PR template	2025-12-17 17:10:04 +01:00
Steven Palma	92fdbe9bbf	docs(dataset): add visualization section (#2664 )	2025-12-17 14:14:31 +01:00
Steven Palma	b303d1ab38	feat(scripts): add more info to lerobot-info (#2663 )	2025-12-17 14:14:23 +01:00
Steven Palma	b1d162f333	fix(policies): add device back to smolvlm expert (#2662 )	2025-12-17 12:12:03 +01:00
nemo	e01927f641	Merge remote-tracking branch 'hf/main' into feature/basic-peft-support	2025-12-16 18:35:11 +01:00
nemo	3bc557552b	Adapt state/action projections instead of full-finetuning There doesn't seem to be a benefit to fully fine-tune these layers over just adapting them, so we do that instead.	2025-12-16 18:31:25 +01:00
nemo	51876e7f55	Add test checking that PEFT actually reduces params	2025-12-16 17:58:27 +01:00
nemo	9649f140ca	Make it possible to use PeftModels in eval	2025-12-16 17:57:03 +01:00
Steven Palma	2b304eeb84	feat(dataset): expose tolerance_s argument to training config (#2653 )	2025-12-16 00:53:19 +01:00
Sota Nakamura	4e6048a221	finalize the dataset after recording (#2496 ) Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>	2025-12-15 17:57:04 +01:00
./c²	81ebcac8d7	docs: update IL robots API example and add OpenCV workaround (#2648 ) * docs: update IL robots API example and add OpenCV workaround - Fix import path from lerobot.record to lerobot.scripts.lerobot_record - Add required processor parameters to record_loop calls - Document fourcc="MJPG" workaround for OpenCV async errors - Improve code formatting in robot configuration examples Fixes compatibility issues for users following the tutorial on embedded systems and ensures API examples match current codebase requirements. * Update il_robots.mdx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: ./c² <cagataycali@icloud.com> --------- Signed-off-by: ./c² <cagataycali@icloud.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>	2025-12-15 17:56:33 +01:00
Martino Russi	a6c3a0fa09	Feat/add mj env (#2613 ) * add sim support * close fix threading issues	2025-12-15 16:22:27 +01:00
Woojin Wie	c2fb644613	feat(robot): Add support for OMX robot (#2614 ) * upload * feat(omx): simplify motor initialization and remove default calibration files * feat(omx): read motor positions without normalization for improved accuracy * update calibration method for return factory value Signed-off-by: Junha Cha <ckwnsgk1@gachon.ac.kr> * change the drive mode * refactor: clean up code by removing unnecessary blank lines in omx_follower and omx_leader modules * feat(omx): update calibration method to set drive modes for motors * feat(pyproject): add 'ROBOTIS' to extend-ignore-identifiers-re list * feat(omx): enhance calibration method to write default drive modes to motors * Update src/lerobot/robots/omx_follower/__init__.py Add informations about the robot Co-authored-by: Steven Palma <imstevenpmwork@ieee.org> Signed-off-by: Woojin Wie <dnldnwls1123@gmail.com> --------- Signed-off-by: Junha Cha <ckwnsgk1@gachon.ac.kr> Signed-off-by: Woojin Wie <dnldnwls1123@gmail.com> Co-authored-by: Junha02 <chajunha2023@naver.com> Co-authored-by: Junha Cha <ckwnsgk1@gachon.ac.kr> Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>	2025-12-15 15:50:29 +01:00
Jade Choghari	1d07a4aefd	add auto in docs (#2645 ) Signed-off-by: Jade Choghari <chogharijade@gmail.com>	2025-12-13 17:11:19 +01:00
Michel Aractingi	ce348a3460	enable variable image sizes to pi0/pi0.5 (#2609 ) * enable variable image sizes to pi0/pi0.5 * add square image assertion	2025-12-10 19:41:11 +01:00
Jade Choghari	cb920235c4	docs: update X-VLA training strategies/commands (#2611 )	2025-12-09 19:08:09 +01:00
Jade Choghari	7f40b3bf82	feat(dataset): add tool to convert images to video datasets (#2560 ) * add video encoding tool * style * make it work * more fixes	2025-12-08 18:50:21 +01:00
Michel Aractingi	2e9c9fd832	Replay while loop in sample actions with for loops (#2600 )	2025-12-08 14:47:54 +01:00
Steven Palma	f9cb5e659c	chore(ci): skip workflows if not lerobot repository (#2601 ) Co-authored-by: Alex Tyshka <atyshka15@gmail.com>	2025-12-08 12:44:36 +01:00
nemo	76e64521c2	Use correct loading for PEFT in RTC example	2025-12-05 14:26:30 +01:00
nemo	f5bf6bb028	Make it possible to unset policy features This is necessary to train pre-trained policies on new datasets so that the features are inferred from the new dataset and not from the pretrained policy.	2025-12-05 12:48:07 +01:00
nemo	75266860aa	Revamp pretrained model loading There were quite a few factors that convinced me that the status quo is able to load pretrained models from the PEFT adapter config but in fact that didn't work. This commit fixes the following things: - policies wrapped in PEFT will now have a `name_or_path` attribute containing the name or path of the pretrained model we're fine-tuning - we further assume that SmolVLA without `pretrained_path` and `load_vlm_weights==False` must be an user-side error - we assume that using PEFT on from-scratch-policies must be an user-side-error	2025-12-01 18:30:45 +01:00
nemo	e0b6aca97a	Merge remote-tracking branch 'hf/main' into feature/basic-peft-support	2025-11-28 19:00:16 +01:00
nemo	abfed126ff	Warn when encountering from-scratch-training	2025-11-28 18:18:00 +01:00
nemo	e544562c39	formatting	2025-11-24 18:52:18 +01:00
nemo	83aac1b42e	Make sure push_to_hub works Since PEFT only wraps `push_to_hub` and not `push_model_to_hub`, the reference to `self` in `policy.push_model_to_hub` is the unwrapped policy which, of course, doesn't know anything about PEFT. To make the upload process aware of PEFT, we pass the unwrapped policy down to `push_model_to_hub` as a kwarg. This is not ideal but I think it is the best way for now.	2025-11-24 18:50:02 +01:00
nemo	e9b3889bd2	Clean up loading code - Centralized instantiation of the PEFT wrapper in `make_policy` for inference (e.g. in `lerobot-record`) - Training a PEFT policy also sets `cfg.use_peft` so that all inference code loading the policy can rely on that attribute to identify if PEFT loading is needed - Modified RTC example to also include PEFT policies. Mostly because this is an example I'm currently exploring.	2025-11-24 15:30:26 +01:00
nemo	841c76c7d3	Merge remote-tracking branch 'hf/main' into feature/basic-peft-support	2025-11-21 17:37:15 +01:00
nemo	356f2a6150	Merge remote-tracking branch 'origin/main' into feature/basic-peft-support	2025-11-21 11:29:33 +01:00
nemo	de34e876cc	Update default targets Removed ACT since it doesn't make sense to fine-tune ACT without having it pretrained beforehand. SmolVLA and Pi0/0.5 are much more senseful targets.	2025-10-16 16:13:41 +02:00
nemo	7f42338ae9	Add CLI end-to-end tests Currently there don't seem to be any way to test the CLI commands. Since this change mostly happens in those I thought it best to add a way to test these commands end-to-end. More integrated commands like `lerobot-record` need patching but standalone commands like training seem to work fine.	2025-10-16 16:08:20 +02:00
nemo	6c2fd8970c	Correct way of identifying when to save config	2025-10-16 16:07:22 +02:00
nemo	6fb8539159	Don't unload & merge the PEFT model This can make things hard when using quantized layers (user expects quantized base layers with unquantized adapters for example, merging defaults to upcast the layers leading to higher memory).	2025-10-16 16:06:23 +02:00
nemo	e263a1de13	Better documentation for CLI arguments	2025-10-16 14:39:43 +02:00
nemo	d39e7f0bc4	Log adapter config to WandB	2025-10-14 15:36:06 +02:00
nemo	20d4b5d347	Remove `use_peft` parameter from training script Instead we make the PEFT config optional which has the same effect.	2025-10-14 15:35:38 +02:00
nemo	02408fb244	Remove PEFT compatibility changes in config We'll wait for the PEFT release that fixes this for good.	2025-10-13 16:32:04 +02:00
nemo	390bfffa92	Merge remote-tracking branch 'hf/main' into feature/basic-peft-support	2025-10-09 11:30:55 +02:00
nemo	8a9be85306	Fix failing tests	2025-07-14 16:45:11 +02:00
nemo	57014e5bb1	Merge branch 'main' of github.com:huggingface/lerobot into feature/basic-peft-support	2025-07-08 11:23:40 +02:00
pre-commit-ci[bot]	354d118407	[pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci	2025-06-30 13:40:19 +00:00
nemo	72d02feef4	Formatting	2025-06-30 15:33:51 +02:00
nemo	4788434d9f	Support targets like `all-linear`	2025-06-30 15:33:13 +02:00
nemo	56e7979cfd	Add default config for ACT	2025-06-27 19:27:08 +02:00
nemo	44245e714e	Merge branch 'main' of github.com:huggingface/lerobot into feature/basic-peft-support	2025-06-27 13:55:07 +02:00
nemo	dc67b2ff3f	Store policy config alongside PEFT checkpoint Before this change the PEFT-wrapped policy did not save the policy's config alongside the adapter config / weights which prevented us from changing the policy config. Now the policy config is saved both in full training and PEFT training. This change makes loading the PEFT policy adapter much easier as well.	2025-06-22 19:54:10 +02:00
nemo	7fd8b4c773	Implement loading of PEFT adapters Loading a PEFT adapter is currently done by initializing a policy with default config and then applying the adapter on the resulting model. This has the obvious drawback that any configurations done during training are not applied in the adapted model. Currently the `use_peft` attribute of `PreTrainedConfig` is only set during loading to signal the following code that it has to deal with a PEFT adapter. However we could imagine a scenario where this is already set at training time and stored alongside the adapter.	2025-06-22 19:10:10 +02:00
nemo	98856662c1	Add basic support for PEFT adapter methods This changes adds support for training policies with much less parameters by applying adapter methods such as LoRA on specific parts of the policies and therefore possibly higher learning rates / batch sizes. To make this as accessible as possible I thought it useful to provide defaults for `target_modules` and `modules_to_save`. Currently only SmolVLA has such defaults but when we agree that this change is useful I will set out to generate more such defaults. While the user can override these settings, they are expected to only change the peft_method, rank and init_type parameters.	2025-06-22 13:45:07 +02:00
				`@@ -1 +0,0 @@`
				`srun --time 12:00:00 --qos=high --gres=gpu:1 --mem=24G --partition=hopper-prod --container-image /fsx/michel_aractingi/docker_images/huggingface+lerobot-gpu+dev.sqsh --container-mounts /fsx/jade_choghari`