From d3bed0feeeccbb3a22ac88f2d18589caa1436aac Mon Sep 17 00:00:00 2001 From: Maxime Ellerbach Date: Fri, 24 Jul 2026 14:58:43 +0200 Subject: [PATCH] chore(agents): adding additional infos to AGENTS.md and bring-your-own-policies.mdx (#3904) * chore(agents): adding additional infos to AGENTS.md * adding `lerobot-train` requirement inside PR checklist * prefer using code already implemented from transformers / diffusers instead of re-implementing in tree --------- Signed-off-by: Maxime Ellerbach --- AGENTS.md | 3 ++- docs/source/bring_your_own_policies.mdx | 7 ++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index bd1bf0af1..e4f80ccbb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -51,6 +51,7 @@ pre-commit run --all-files # Lint + format (ruff, typo ## Notes - **Mypy is gradual**: strict only for `lerobot.envs`, `lerobot.configs`, `lerobot.optim`, `lerobot.model`, `lerobot.cameras`, `lerobot.motors`, `lerobot.transport`. Add type annotations when modifying these modules. -- **Optional dependencies**: many policies, envs, and robots are behind extras (e.g., `lerobot[aloha]`). New imports for optional packages must be guarded or lazy. See `pyproject.toml [project.optional-dependencies]`. +- **Imports**: prefer top-level imports; relative (`from .sibling import X`) across sibling files within a module, absolute (`from lerobot.module import X`) across modules. +- **Optional dependencies**: many policies, envs, and robots are behind extras (e.g., `lerobot[aloha]`, see `pyproject.toml`). Guard optional imports with `TYPE_CHECKING or _foo_available` at module top + a `require_package(...)` check at use time. Reuse the `_foo_available` flags in `utils/import_utils.py`; don't call `is_package_available`. - **Video decoding**: datasets can store observations as video files. `LeRobotDataset` handles frame extraction, but tests need ffmpeg installed. - **Prioritize use of `uv run`** to execute Python commands (not raw `python` or `pip`). diff --git a/docs/source/bring_your_own_policies.mdx b/docs/source/bring_your_own_policies.mdx index bf71efb7e..697a07691 100644 --- a/docs/source/bring_your_own_policies.mdx +++ b/docs/source/bring_your_own_policies.mdx @@ -165,6 +165,8 @@ Batches are flat dictionaries keyed by the constants in [`lerobot.utils.constant LeRobot uses `PolicyProcessorPipeline`s to normalize inputs and de-normalize outputs around your policy. For a concrete reference, see [`processor_act.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/policies/act/processor_act.py) or [`processor_diffusion.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/policies/diffusion/processor_diffusion.py). +Pay close attention here: processors are the most common reproducibility pain point. A mismatch in normalization mode (`IDENTITY` vs `MEAN_STD` vs `MIN_MAX` vs `QUANTILES`/`QUANTILE10`) or in which features get normalized will train and eval without erroring, yet silently wreck results. Make sure the modes match how the checkpoint was trained, that the required stats exist (e.g. `QUANTILES` needs `q01`/`q99`), and that the pre- and post-processors stay consistent. + ```python # processor_my_policy.py from typing import Any @@ -304,7 +306,9 @@ Mirror an existing policy that's structurally similar to yours; the diff is smal ### Heavy / optional dependencies -Most policies need a heavy backbone (transformers, diffusers, a specific VLM SDK). The convention is **two-step gating**: a `TYPE_CHECKING`-guarded import at module top, and a `require_package` runtime check in the constructor. [`modeling_diffusion.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/policies/diffusion/modeling_diffusion.py) is the canonical reference: +Most policies need a heavy backbone (transformers, diffusers, a specific VLM SDK). Wherever one exists, prefer loading it e.g from `transformers` or `diffusers` rather than re-implementing the architecture in-tree. + +The convention is **two-step gating**: a `TYPE_CHECKING`-guarded import at module top, and a `require_package` runtime check in the constructor. [`modeling_diffusion.py`](https://github.com/huggingface/lerobot/blob/main/src/lerobot/policies/diffusion/modeling_diffusion.py) is the canonical reference: ```python from typing import TYPE_CHECKING @@ -374,6 +378,7 @@ The general expectations are in [`CONTRIBUTING.md`](https://github.com/huggingfa - [ ] Optional deps live behind a `[project.optional-dependencies]` extra and the `TYPE_CHECKING + require_package` guard. - [ ] `tests/policies/` updated; backward-compat artifact committed & policy-specific tests. - [ ] `src/lerobot/policies//README.md` symlinked into `docs/source/policy__README.md`; user-facing `docs/source/.mdx` written and added to `_toctree.yml`. +- [ ] `lerobot-train --policy.type my_policy ...` runs end-to-end for at least a few steps + save a checkpoint that can be loaded and run by `lerobot-eval` or `lerobot-rollout`. - [ ] `templates/lerobot_modelcard_template.md` has a description entry and a `policy_docs` link for your policy. - [ ] The models table in the root `README.md` lists your policy in the right category, linking to your doc page. - [ ] At least one reproducible benchmark eval in the policy MDX with a published checkpoint (sim benchmark, or real-robot dataset + checkpoint).