Merge branch 'main' into feat/language-annotation-pipeline

This commit is contained in:
Pepijn
2026-06-03 15:46:13 +02:00
committed by GitHub
11 changed files with 59 additions and 26 deletions
+2
View File
@@ -9,6 +9,8 @@
- sections: - sections:
- local: il_robots - local: il_robots
title: Imitation Learning for Robots title: Imitation Learning for Robots
- local: lelab
title: LeLab - Lerobot GUI
- local: bring_your_own_policies - local: bring_your_own_policies
title: Adding a Policy title: Adding a Policy
- local: integrate_hardware - local: integrate_hardware
+29
View File
@@ -0,0 +1,29 @@
# LeLab - LeRobot Guide
LeLab is a graphical user interface built on top of the LeRobot library, designed to make robotics accessible without needing to memorize CLI commands. From a single app you can configure your robot, teleoperate it, collect datasets, train policies locally or on cloud GPUs via HF Jobs, and deploy trained models back onto your robot. It's the easiest way to go from an unboxed SO-101 to a working policy, and a great companion for anyone learning the LeRobot workflow. Source code and issues live on GitHub: [huggingface/leLab](https://github.com/huggingface/leLab).
> [!TIP]
> For now LeLab is compatible only with SO-ARM101
<Youtube id="VqyKUuW9V1g" />
### Installation
Requires [`uv`](https://docs.astral.sh/uv/getting-started/installation/). Install and launch in one command:
```
uv tool install git+https://github.com/huggingface/leLab.git && lelab
```
After install, run `lelab` from your terminal anytime to start the app.
### Features
- **Add robots** — Select arm type (leader/follower), calibrate each joint from the middle position, and attach cameras.
- **Teleoperation** — Control the follower arm with the leader and see a live 3D visualization of the arms.
- **Dataset recording** — Define a task description, number of episodes, and episode/reset durations. Press spacebar to advance between episodes. 30+ episodes recommended.
- **Local training** — Train a policy directly on your own machine with a selected dataset, policy type, batch size, and step count.
- **Cloud training with HF Jobs** — Train on powerful GPUs via [HF Jobs](https://huggingface.co/docs/huggingface_hub/en/guides/jobs) with transparent pricing. Run `hf auth login` first. See the [Compute HW Guide](hardware_guide) for hardware/batch size tips.
- **Training visualization** — Watch progress live in the app, with checkpoints saved automatically.
- **Run trained policies** — Pick any model from your jobs list and run inference on your robot with one click.
- **Use community datasets** — Provide any Hugging Face dataset ID to train on datasets you didn't record yourself.
+1 -1
View File
@@ -275,7 +275,7 @@ A converter aggregates perepisode files into larger shards and writes episode
pip install "https://github.com/huggingface/lerobot/archive/33cad37054c2b594ceba57463e8f11ee374fa93c.zip" pip install "https://github.com/huggingface/lerobot/archive/33cad37054c2b594ceba57463e8f11ee374fa93c.zip"
# Convert an existing v2.1 dataset hosted on the Hub: # Convert an existing v2.1 dataset hosted on the Hub:
python -m lerobot.datasets.v30.convert_dataset_v21_to_v30 --repo-id=<HF_USER/DATASET_ID> python -m lerobot.scripts.convert_dataset_v21_to_v30 --repo-id=<HF_USER/DATASET_ID>
``` ```
**What it does** **What it does**
+1 -1
View File
@@ -238,7 +238,7 @@ your dataset has not been converted with quantile statistics, you can add them
with: with:
```bash ```bash
python src/lerobot/datasets/v30/augment_dataset_quantile_stats.py \ python src/lerobot/scripts/augment_dataset_quantile_stats.py \
--repo-id=your_dataset --repo-id=your_dataset
``` ```
+1 -1
View File
@@ -91,7 +91,7 @@ lerobot-train \
If your dataset is not converted with `quantiles`, you can convert it with the following command: If your dataset is not converted with `quantiles`, you can convert it with the following command:
```bash ```bash
python src/lerobot/datasets/v30/augment_dataset_quantile_stats.py \ python src/lerobot/scripts/augment_dataset_quantile_stats.py \
--repo-id=your_dataset \ --repo-id=your_dataset \
``` ```
+1 -1
View File
@@ -300,7 +300,7 @@ This replaces the old episode-per-file structure with efficient, optimally-sized
If you have existing datasets in v2.1 format, use the migration tool: If you have existing datasets in v2.1 format, use the migration tool:
```bash ```bash
python src/lerobot/datasets/v30/convert_dataset_v21_to_v30.py \ python src/lerobot/scripts/convert_dataset_v21_to_v30.py \
--repo-id your_id/existing_dataset --repo-id your_id/existing_dataset
``` ```
+2 -2
View File
@@ -41,8 +41,8 @@ class DatasetRecordConfig:
video: bool = True video: bool = True
# Upload dataset to Hugging Face hub. # Upload dataset to Hugging Face hub.
push_to_hub: bool = True push_to_hub: bool = True
# Upload on private repository on the Hugging Face hub. # If True, upload as private; if None, defer to the org default on the Hub (only affects orgs).
private: bool = False private: bool | None = None
# Add tags to your dataset on the hub. # Add tags to your dataset on the hub.
tags: list[str] | None = None tags: list[str] | None = None
# Number of subprocesses handling the saving of frames as PNG. Set to 0 to use threads only; # Number of subprocesses handling the saving of frames as PNG. Set to 0 to use threads only;
+6
View File
@@ -177,6 +177,12 @@ class TrainPipelineConfig(HubMixin):
) )
active_cfg = self.trainable_config active_cfg = self.trainable_config
if self.rename_map and active_cfg.pretrained_path is None:
raise ValueError(
"`rename_map` requires a pretrained policy checkpoint. "
"Fresh initialization derives feature names from the current dataset, so no rename is applied."
)
if not self.job_name: if not self.job_name:
if self.env is None: if self.env is None:
self.job_name = f"{active_cfg.type}" self.job_name = f"{active_cfg.type}"
+3 -2
View File
@@ -524,7 +524,7 @@ class LeRobotDataset(torch.utils.data.Dataset):
license: str | None = "apache-2.0", license: str | None = "apache-2.0",
tag_version: bool = True, tag_version: bool = True,
push_videos: bool = True, push_videos: bool = True,
private: bool = False, private: bool | None = None,
allow_patterns: list[str] | str | None = None, allow_patterns: list[str] | str | None = None,
upload_large_folder: bool = False, upload_large_folder: bool = False,
**card_kwargs, **card_kwargs,
@@ -543,7 +543,8 @@ class LeRobotDataset(torch.utils.data.Dataset):
tag_version: If ``True``, create a Git tag for the current codebase tag_version: If ``True``, create a Git tag for the current codebase
version. version.
push_videos: If ``False``, skip uploading the ``videos/`` directory. push_videos: If ``False``, skip uploading the ``videos/`` directory.
private: If ``True``, create a private repository. private: If ``True``, create a private repository. If ``None``
(default), defer to the org default on the Hub (only affects orgs).
allow_patterns: Glob pattern(s) restricting which files to upload. allow_patterns: Glob pattern(s) restricting which files to upload.
upload_large_folder: If ``True``, use ``upload_large_folder`` instead upload_large_folder: If ``True``, use ``upload_large_folder`` instead
of ``upload_folder`` for very large datasets. of ``upload_folder`` for very large datasets.
@@ -81,7 +81,7 @@ def to_absolute_actions(actions: Tensor, state: Tensor, mask: Sequence[bool]) ->
return actions return actions
@ProcessorStepRegistry.register("delta_actions_processor") @ProcessorStepRegistry.register("relative_actions_processor")
@dataclass @dataclass
class RelativeActionsProcessorStep(ProcessorStep): class RelativeActionsProcessorStep(ProcessorStep):
"""Converts absolute actions to relative actions (action -= state) for masked dimensions. """Converts absolute actions to relative actions (action -= state) for masked dimensions.
+12 -17
View File
@@ -292,19 +292,8 @@ def train(cfg: TrainPipelineConfig, accelerator: "Accelerator | None" = None):
active_cfg = cfg.trainable_config active_cfg = cfg.trainable_config
processor_pretrained_path = active_cfg.pretrained_path processor_pretrained_path = active_cfg.pretrained_path
if (
getattr(active_cfg, "use_relative_actions", False)
and processor_pretrained_path is not None
and not cfg.resume
):
logging.warning(
"use_relative_actions=true with pretrained processors can skip relative transforms if "
"the checkpoint processors do not define them. Building processors from current policy config."
)
processor_pretrained_path = None
processor_kwargs = {} processor_kwargs = {}
postprocessor_kwargs = {}
if (processor_pretrained_path and not cfg.resume) or not processor_pretrained_path: if (processor_pretrained_path and not cfg.resume) or not processor_pretrained_path:
processor_kwargs["dataset_stats"] = dataset.meta.stats processor_kwargs["dataset_stats"] = dataset.meta.stats
@@ -312,24 +301,31 @@ def train(cfg: TrainPipelineConfig, accelerator: "Accelerator | None" = None):
processor_kwargs["dataset_meta"] = dataset.meta processor_kwargs["dataset_meta"] = dataset.meta
if not cfg.is_reward_model_training and processor_pretrained_path is not None: if not cfg.is_reward_model_training and processor_pretrained_path is not None:
processor_kwargs["preprocessor_overrides"] = { preprocessor_overrides = {
"device_processor": {"device": device.type}, "device_processor": {"device": device.type},
"normalizer_processor": { "normalizer_processor": {
"stats": dataset.meta.stats, "stats": dataset.meta.stats,
"features": {**policy.config.input_features, **policy.config.output_features}, "features": {**policy.config.input_features, **policy.config.output_features},
"norm_map": policy.config.normalization_mapping, "norm_map": policy.config.normalization_mapping,
}, },
"rename_observations_processor": {"rename_map": cfg.rename_map},
} }
processor_kwargs["preprocessor_overrides"]["rename_observations_processor"] = { postprocessor_overrides = {
"rename_map": cfg.rename_map
}
postprocessor_kwargs["postprocessor_overrides"] = {
"unnormalizer_processor": { "unnormalizer_processor": {
"stats": dataset.meta.stats, "stats": dataset.meta.stats,
"features": policy.config.output_features, "features": policy.config.output_features,
"norm_map": policy.config.normalization_mapping, "norm_map": policy.config.normalization_mapping,
}, },
} }
if getattr(active_cfg, "use_relative_actions", False):
preprocessor_overrides["relative_actions_processor"] = {
"enabled": True,
"exclude_joints": getattr(active_cfg, "relative_exclude_joints", []),
"action_names": getattr(active_cfg, "action_feature_names", None),
}
postprocessor_overrides["absolute_actions_processor"] = {"enabled": True}
processor_kwargs["preprocessor_overrides"] = preprocessor_overrides
processor_kwargs["postprocessor_overrides"] = postprocessor_overrides
if cfg.is_reward_model_training: if cfg.is_reward_model_training:
preprocessor, postprocessor = make_reward_pre_post_processors( preprocessor, postprocessor = make_reward_pre_post_processors(
@@ -341,7 +337,6 @@ def train(cfg: TrainPipelineConfig, accelerator: "Accelerator | None" = None):
policy_cfg=cfg.policy, policy_cfg=cfg.policy,
pretrained_path=processor_pretrained_path, pretrained_path=processor_pretrained_path,
**processor_kwargs, **processor_kwargs,
**postprocessor_kwargs,
) )
if is_main_process: if is_main_process: