{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 🤗 LeRobot Quickstart\n", "\n", "Calibration → teleoperation → data collection → training → evaluation.\n", "\n", "Install the required dependencies: `pip install -e .[notebook,dataset,training,viz,hardware]`.\n", "\n", "**How to use:**\n", "1. Edit the **Configuration** cell with your settings.\n", "2. Run all cells (`Run All`).\n", "3. Each section prints a ready-to-paste terminal command - copy it and run it.\n", "\n", "Each setup is different, please refer to the [LeRobot documentation](https://huggingface.co/docs/lerobot/il_robots) for more details on each step and available options.
\n", "Feel free to make this notebook your own and adapt it to your needs!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## Utils" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def _cameras_arg(cameras: dict) -> str:\n", " if not cameras:\n", " return \"\"\n", " entries = [f\"{n}: {{{', '.join(f'{k}: {v}' for k, v in cfg.items())}}}\" for n, cfg in cameras.items()]\n", " return \"{ \" + \", \".join(entries) + \" }\"\n", "\n", "\n", "def print_cmd(*parts: str) -> None:\n", " \"\"\"Print a shell command with line continuations, skipping empty parts.\"\"\"\n", " non_empty = [p for p in parts if p]\n", " print(\" \\\\\\n \".join(non_empty))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## Configuration\n", "\n", "Edit this cell, then **Run All** to generate all commands below." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Robot (follower) - run `lerobot-find-port` to discover the port\n", "ROBOT_TYPE = \"so101_follower\"\n", "ROBOT_PORT = \"/dev/ttyACM0\"\n", "ROBOT_ID = \"my_follower_arm\"\n", "\n", "# Teleop (leader) - run `lerobot-find-port` to discover the port\n", "TELEOP_TYPE = \"so101_leader\"\n", "TELEOP_PORT = \"/dev/ttyACM1\"\n", "TELEOP_ID = \"my_leader_arm\"\n", "\n", "# Cameras - set to {} to disable\n", "# Run `lerobot-find-cameras opencv` to list available cameras and their indices\n", "CAMERAS = {\n", " \"top\": {\"type\": \"opencv\", \"index_or_path\": 2, \"width\": 640, \"height\": 480, \"fps\": 30},\n", " \"wrist\": {\"type\": \"opencv\", \"index_or_path\": 4, \"width\": 640, \"height\": 480, \"fps\": 30},\n", "}\n", "\n", "# Dataset\n", "HF_USER = \"your_hf_username\" # `huggingface-cli whoami` to find your username\n", "DATASET_NAME = \"my_so101_dataset\"\n", "TASK_DESCRIPTION = \"pick and place the block\"\n", "NUM_EPISODES = 10\n", "\n", "# Training\n", "POLICY_TYPE = \"act\" # act, diffusion, smolvla, ...\n", "POLICY_DEVICE = \"cuda\" # cuda / cpu / mps\n", "TRAIN_STEPS = 10_000\n", "SAVE_FREQ = 2_000\n", "OUTPUT_DIR = f\"outputs/train/{DATASET_NAME}\"\n", "\n", "# Inference - Hub repo ID or local checkpoint path\n", "# e.g. set to f\"{OUTPUT_DIR}/checkpoints/last\" to use a local checkpoint\n", "POLICY_PATH = f\"{HF_USER}/{DATASET_NAME}_{POLICY_TYPE}\"\n", "LAST_CHECKPOINT_PATH = f\"{OUTPUT_DIR}/checkpoints/last\"\n", "\n", "# Derived\n", "DATASET_REPO_ID = f\"{HF_USER}/{DATASET_NAME}\"\n", "DATASET_ROOT = f\"data/{DATASET_NAME}\"\n", "POLICY_REPO_ID = f\"{HF_USER}/{DATASET_NAME}_{POLICY_TYPE}\"\n", "EVAL_REPO_ID = f\"{HF_USER}/eval_{DATASET_NAME}\"\n", "CAMERAS_ARG = _cameras_arg(CAMERAS)\n", "CAMERAS_FLAG = f'--robot.cameras=\"{CAMERAS_ARG}\"' if CAMERAS_ARG else \"\"\n", "\n", "print(f\"Robot : {ROBOT_TYPE} @ {ROBOT_PORT}\")\n", "print(f\"Teleop : {TELEOP_TYPE} @ {TELEOP_PORT}\")\n", "print(f\"Cameras: {list(CAMERAS) or 'none'}\")\n", "print(f\"Dataset: {DATASET_REPO_ID} ({NUM_EPISODES} episodes) saved to {DATASET_ROOT}\")\n", "print(f\"Policy : {POLICY_TYPE} -> {POLICY_REPO_ID}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## 1. Calibration\n", "\n", "Run once per arm before first use." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Follower\n", "print_cmd(\n", " \"lerobot-calibrate\",\n", " f\"--robot.type={ROBOT_TYPE}\",\n", " f\"--robot.port={ROBOT_PORT}\",\n", " f\"--robot.id={ROBOT_ID}\",\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Leader\n", "print_cmd(\n", " \"lerobot-calibrate\",\n", " f\"--teleop.type={TELEOP_TYPE}\",\n", " f\"--teleop.port={TELEOP_PORT}\",\n", " f\"--teleop.id={TELEOP_ID}\",\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## 2. Teleoperation\n", "\n", "See the [teleoperation docs](https://huggingface.co/docs/lerobot/il_robots#teleoperate) and the [cameras guide](https://huggingface.co/docs/lerobot/cameras) for more options." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print_cmd(\n", " \"lerobot-teleoperate\",\n", " f\"--robot.type={ROBOT_TYPE}\",\n", " f\"--robot.port={ROBOT_PORT}\",\n", " f\"--robot.id={ROBOT_ID}\",\n", " CAMERAS_FLAG,\n", " f\"--teleop.type={TELEOP_TYPE}\",\n", " f\"--teleop.port={TELEOP_PORT}\",\n", " f\"--teleop.id={TELEOP_ID}\",\n", " \"--display_data=true\",\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## 3. Record Dataset\n", "\n", "See the [recording docs](https://huggingface.co/docs/lerobot/il_robots#record-a-dataset) for tips on gathering good data." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print_cmd(\n", " \"lerobot-record\",\n", " f\"--robot.type={ROBOT_TYPE}\",\n", " f\"--robot.port={ROBOT_PORT}\",\n", " f\"--robot.id={ROBOT_ID}\",\n", " CAMERAS_FLAG,\n", " f\"--teleop.type={TELEOP_TYPE}\",\n", " f\"--teleop.port={TELEOP_PORT}\",\n", " f\"--teleop.id={TELEOP_ID}\",\n", " f\"--dataset.repo_id={DATASET_REPO_ID}\",\n", " f\"--dataset.num_episodes={NUM_EPISODES}\",\n", " f'--dataset.single_task=\"{TASK_DESCRIPTION}\"',\n", " \"--dataset.streaming_encoding=true\",\n", " \"--display_data=true\",\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Resume a previously interrupted recording session\n", "print_cmd(\n", " \"lerobot-record\",\n", " f\"--robot.type={ROBOT_TYPE}\",\n", " f\"--robot.port={ROBOT_PORT}\",\n", " f\"--robot.id={ROBOT_ID}\",\n", " CAMERAS_FLAG,\n", " f\"--teleop.type={TELEOP_TYPE}\",\n", " f\"--teleop.port={TELEOP_PORT}\",\n", " f\"--teleop.id={TELEOP_ID}\",\n", " f\"--dataset.repo_id={DATASET_REPO_ID}\",\n", " f\"--dataset.root={DATASET_ROOT}\",\n", " f\"--dataset.num_episodes={NUM_EPISODES}\",\n", " f'--dataset.single_task=\"{TASK_DESCRIPTION}\"',\n", " \"--dataset.streaming_encoding=true\",\n", " \"--display_data=true\",\n", " \"--resume=true\",\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## 4. Train Policy\n", "\n", "See the [training docs](https://huggingface.co/docs/lerobot/il_robots#train-a-policy) for configuration options and tips." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print_cmd(\n", " \"lerobot-train\",\n", " f\"--dataset.repo_id={DATASET_REPO_ID}\",\n", " f\"--policy.type={POLICY_TYPE}\",\n", " f\"--policy.device={POLICY_DEVICE}\",\n", " f\"--policy.repo_id={POLICY_REPO_ID}\",\n", " f\"--output_dir={OUTPUT_DIR}\",\n", " f\"--steps={TRAIN_STEPS}\",\n", " f\"--save_freq={SAVE_FREQ}\",\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Resume a previously interrupted training session\n", "print_cmd(\n", " \"lerobot-train\",\n", " f\"--config_path={LAST_CHECKPOINT_PATH}/pretrained_model/train_config.json\",\n", " \"--resume=true\",\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## 5. Inference\n", "\n", "Uses `POLICY_PATH` from the Configuration cell (defaults to the Hub repo ID). You can also put there the `LAST_CHECKPOINT_PATH`.\n", "\n", "See the [inference docs](https://huggingface.co/docs/lerobot/il_robots#run-inference-and-evaluate-your-policy) for details." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print_cmd(\n", " \"lerobot-record\",\n", " f\"--policy.path={POLICY_PATH}\",\n", " f\"--robot.type={ROBOT_TYPE}\",\n", " f\"--robot.port={ROBOT_PORT}\",\n", " f\"--robot.id={ROBOT_ID}\",\n", " CAMERAS_FLAG,\n", " f\"--teleop.type={TELEOP_TYPE}\",\n", " f\"--teleop.port={TELEOP_PORT}\",\n", " f\"--teleop.id={TELEOP_ID}\",\n", " f\"--dataset.repo_id={EVAL_REPO_ID}\",\n", " f\"--dataset.num_episodes={NUM_EPISODES}\",\n", " f'--dataset.single_task=\"{TASK_DESCRIPTION}\"',\n", " \"--dataset.streaming_encoding=true\",\n", ")" ] } ], "metadata": { "kernelspec": { "display_name": "lerobot (3.12.3)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 4 }