lerobot/examples/notebooks/quickstart.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 🤗 LeRobot Quickstart\n",
    "\n",
    "Calibration → teleoperation → data collection → training → evaluation.\n",
    "\n",
    "Install the required dependencies: `pip install -e .[notebook,dataset,training,viz,hardware]`.\n",
    "\n",
    "**How to use:**\n",
    "1. Edit the **Configuration** cell with your settings.\n",
    "2. Run all cells (`Run All`).\n",
    "3. Each section prints a ready-to-paste terminal command - copy it and run it.\n",
    "\n",
    "Each setup is different, please refer to the [LeRobot documentation](https://huggingface.co/docs/lerobot/il_robots) for more details on each step and available options. <br>\n",
    "Feel free to make this notebook your own and adapt it to your needs!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Utils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def _cameras_arg(cameras: dict) -> str:\n",
    "    if not cameras:\n",
    "        return \"\"\n",
    "    entries = [f\"{n}: {{{', '.join(f'{k}: {v}' for k, v in cfg.items())}}}\" for n, cfg in cameras.items()]\n",
    "    return \"{ \" + \", \".join(entries) + \" }\"\n",
    "\n",
    "\n",
    "def print_cmd(*parts: str) -> None:\n",
    "    \"\"\"Print a shell command with line continuations, skipping empty parts.\"\"\"\n",
    "    non_empty = [p for p in parts if p]\n",
    "    print(\" \\\\\\n    \".join(non_empty))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Configuration\n",
    "\n",
    "Edit this cell, then **Run All** to generate all commands below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Robot (follower) - run `lerobot-find-port` to discover the port\n",
    "ROBOT_TYPE = \"so101_follower\"\n",
    "ROBOT_PORT = \"/dev/ttyACM0\"\n",
    "ROBOT_ID = \"my_follower_arm\"\n",
    "\n",
    "# Teleop (leader) - run `lerobot-find-port` to discover the port\n",
    "TELEOP_TYPE = \"so101_leader\"\n",
    "TELEOP_PORT = \"/dev/ttyACM1\"\n",
    "TELEOP_ID = \"my_leader_arm\"\n",
    "\n",
    "# Cameras - set to {} to disable\n",
    "# Run `lerobot-find-cameras opencv` to list available cameras and their indices\n",
    "CAMERAS = {\n",
    "    \"top\": {\"type\": \"opencv\", \"index_or_path\": 2, \"width\": 640, \"height\": 480, \"fps\": 30},\n",
    "    \"wrist\": {\"type\": \"opencv\", \"index_or_path\": 4, \"width\": 640, \"height\": 480, \"fps\": 30},\n",
    "}\n",
    "\n",
    "# Dataset\n",
    "HF_USER = \"your_hf_username\"  # `huggingface-cli whoami` to find your username\n",
    "DATASET_NAME = \"my_so101_dataset\"\n",
    "TASK_DESCRIPTION = \"pick and place the block\"\n",
    "NUM_EPISODES = 10\n",
    "\n",
    "# Training\n",
    "POLICY_TYPE = \"act\"  # act, diffusion, smolvla, ...\n",
    "POLICY_DEVICE = \"cuda\"  # cuda / cpu / mps\n",
    "TRAIN_STEPS = 10_000\n",
    "SAVE_FREQ = 2_000\n",
    "OUTPUT_DIR = f\"outputs/train/{DATASET_NAME}\"\n",
    "\n",
    "# Inference - Hub repo ID or local checkpoint path\n",
    "# e.g. set to f\"{OUTPUT_DIR}/checkpoints/last\" to use a local checkpoint\n",
    "POLICY_PATH = f\"{HF_USER}/{DATASET_NAME}_{POLICY_TYPE}\"\n",
    "LAST_CHECKPOINT_PATH = f\"{OUTPUT_DIR}/checkpoints/last\"\n",
    "\n",
    "# Derived\n",
    "DATASET_REPO_ID = f\"{HF_USER}/{DATASET_NAME}\"\n",
    "DATASET_ROOT = f\"data/{DATASET_NAME}\"\n",
    "POLICY_REPO_ID = f\"{HF_USER}/{DATASET_NAME}_{POLICY_TYPE}\"\n",
    "EVAL_REPO_ID = f\"{HF_USER}/eval_{DATASET_NAME}\"\n",
    "CAMERAS_ARG = _cameras_arg(CAMERAS)\n",
    "CAMERAS_FLAG = f'--robot.cameras=\"{CAMERAS_ARG}\"' if CAMERAS_ARG else \"\"\n",
    "\n",
    "print(f\"Robot  : {ROBOT_TYPE} @ {ROBOT_PORT}\")\n",
    "print(f\"Teleop : {TELEOP_TYPE} @ {TELEOP_PORT}\")\n",
    "print(f\"Cameras: {list(CAMERAS) or 'none'}\")\n",
    "print(f\"Dataset: {DATASET_REPO_ID} ({NUM_EPISODES} episodes) saved to {DATASET_ROOT}\")\n",
    "print(f\"Policy : {POLICY_TYPE} -> {POLICY_REPO_ID}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 1. Calibration\n",
    "\n",
    "Run once per arm before first use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Follower\n",
    "print_cmd(\n",
    "    \"lerobot-calibrate\",\n",
    "    f\"--robot.type={ROBOT_TYPE}\",\n",
    "    f\"--robot.port={ROBOT_PORT}\",\n",
    "    f\"--robot.id={ROBOT_ID}\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Leader\n",
    "print_cmd(\n",
    "    \"lerobot-calibrate\",\n",
    "    f\"--teleop.type={TELEOP_TYPE}\",\n",
    "    f\"--teleop.port={TELEOP_PORT}\",\n",
    "    f\"--teleop.id={TELEOP_ID}\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 2. Teleoperation\n",
    "\n",
    "See the [teleoperation docs](https://huggingface.co/docs/lerobot/il_robots#teleoperate) and the [cameras guide](https://huggingface.co/docs/lerobot/cameras) for more options."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print_cmd(\n",
    "    \"lerobot-teleoperate\",\n",
    "    f\"--robot.type={ROBOT_TYPE}\",\n",
    "    f\"--robot.port={ROBOT_PORT}\",\n",
    "    f\"--robot.id={ROBOT_ID}\",\n",
    "    CAMERAS_FLAG,\n",
    "    f\"--teleop.type={TELEOP_TYPE}\",\n",
    "    f\"--teleop.port={TELEOP_PORT}\",\n",
    "    f\"--teleop.id={TELEOP_ID}\",\n",
    "    \"--display_data=true\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 3. Record Dataset\n",
    "\n",
    "See the [recording docs](https://huggingface.co/docs/lerobot/il_robots#record-a-dataset) for tips on gathering good data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print_cmd(\n",
    "    \"lerobot-record\",\n",
    "    f\"--robot.type={ROBOT_TYPE}\",\n",
    "    f\"--robot.port={ROBOT_PORT}\",\n",
    "    f\"--robot.id={ROBOT_ID}\",\n",
    "    CAMERAS_FLAG,\n",
    "    f\"--teleop.type={TELEOP_TYPE}\",\n",
    "    f\"--teleop.port={TELEOP_PORT}\",\n",
    "    f\"--teleop.id={TELEOP_ID}\",\n",
    "    f\"--dataset.repo_id={DATASET_REPO_ID}\",\n",
    "    f\"--dataset.num_episodes={NUM_EPISODES}\",\n",
    "    f'--dataset.single_task=\"{TASK_DESCRIPTION}\"',\n",
    "    \"--dataset.streaming_encoding=true\",\n",
    "    \"--display_data=true\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Resume a previously interrupted recording session\n",
    "print_cmd(\n",
    "    \"lerobot-record\",\n",
    "    f\"--robot.type={ROBOT_TYPE}\",\n",
    "    f\"--robot.port={ROBOT_PORT}\",\n",
    "    f\"--robot.id={ROBOT_ID}\",\n",
    "    CAMERAS_FLAG,\n",
    "    f\"--teleop.type={TELEOP_TYPE}\",\n",
    "    f\"--teleop.port={TELEOP_PORT}\",\n",
    "    f\"--teleop.id={TELEOP_ID}\",\n",
    "    f\"--dataset.repo_id={DATASET_REPO_ID}\",\n",
    "    f\"--dataset.root={DATASET_ROOT}\",\n",
    "    f\"--dataset.num_episodes={NUM_EPISODES}\",\n",
    "    f'--dataset.single_task=\"{TASK_DESCRIPTION}\"',\n",
    "    \"--dataset.streaming_encoding=true\",\n",
    "    \"--display_data=true\",\n",
    "    \"--resume=true\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 4. Train Policy\n",
    "\n",
    "See the [training docs](https://huggingface.co/docs/lerobot/il_robots#train-a-policy) for configuration options and tips."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print_cmd(\n",
    "    \"lerobot-train\",\n",
    "    f\"--dataset.repo_id={DATASET_REPO_ID}\",\n",
    "    f\"--policy.type={POLICY_TYPE}\",\n",
    "    f\"--policy.device={POLICY_DEVICE}\",\n",
    "    f\"--policy.repo_id={POLICY_REPO_ID}\",\n",
    "    f\"--output_dir={OUTPUT_DIR}\",\n",
    "    f\"--steps={TRAIN_STEPS}\",\n",
    "    f\"--save_freq={SAVE_FREQ}\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Resume a previously interrupted training session\n",
    "print_cmd(\n",
    "    \"lerobot-train\",\n",
    "    f\"--config_path={LAST_CHECKPOINT_PATH}/pretrained_model/train_config.json\",\n",
    "    \"--resume=true\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 5. Inference\n",
    "\n",
    "Uses `POLICY_PATH` from the Configuration cell (defaults to the Hub repo ID). You can also put there the `LAST_CHECKPOINT_PATH`.\n",
    "\n",
    "See the [inference docs](https://huggingface.co/docs/lerobot/il_robots#run-inference-and-evaluate-your-policy) for details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print_cmd(\n",
    "    \"lerobot-record\",\n",
    "    f\"--policy.path={POLICY_PATH}\",\n",
    "    f\"--robot.type={ROBOT_TYPE}\",\n",
    "    f\"--robot.port={ROBOT_PORT}\",\n",
    "    f\"--robot.id={ROBOT_ID}\",\n",
    "    CAMERAS_FLAG,\n",
    "    f\"--teleop.type={TELEOP_TYPE}\",\n",
    "    f\"--teleop.port={TELEOP_PORT}\",\n",
    "    f\"--teleop.id={TELEOP_ID}\",\n",
    "    f\"--dataset.repo_id={EVAL_REPO_ID}\",\n",
    "    f\"--dataset.num_episodes={NUM_EPISODES}\",\n",
    "    f'--dataset.single_task=\"{TASK_DESCRIPTION}\"',\n",
    "    \"--dataset.streaming_encoding=true\",\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "lerobot (3.12.3)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}