admin/lerobot
1
0
Fork 0
mirror of https://github.com/huggingface/lerobot.git synced 2026-08-08 17:39:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
cc5759c00379283e2525fbf0989a0f9659b30d5f
lerobot/Makefile
T
Pepijn 2e8345a5cc docs: add API documentation infrastructure 
LeRobot's documentation build passes `--not_python_module`, which tells
doc-builder there is no importable Python package and disables `[[autodoc]]`
entirely. The result is that all 90+ pages are hand-written guides and there is
no generated API reference at all.

This is the machinery to change that. It deliberately contains no docstring
changes of its own — every docstring edit lives in the follow-up PR, so this
one can be reviewed as tooling and configuration alone.

**The standard.** `docs/source/writing_docstrings.mdx` is the contract: Google
section headers with Hugging Face type formatting, the machine-checked argument
line, `**Attributes**:`, doc-builder cross-references, fenced doctest examples.
It also records three behaviours that are not discoverable from the source and
were verified against a local build: `[[autodoc]]` silently skips members with
no docstring; doc-builder does not inherit docstrings from base classes, so a
registered config shim whose body is `pass` renders every field with no
description; and module-level aliases resolve to the canonical class.

**Autodoc turned on**, with two changes that are not obvious:

- `--version main` on the main-docs job. Without `--not_python_module`,
  doc-builder resolves the version from `lerobot.__version__` and only maps it
  to the default branch when it contains "dev". transformers relies on that;
  our main carries 0.6.2. Verified by building both ways — dropping the flag
  alone would publish the main docs to /lerobot/v0.6.2/ instead of
  /lerobot/main/ and disable notebook building.
- `pre_command` on both jobs. doc-builder ships a mock-deps registry entry for
  lerobot, so the reusable workflow takes its light-install path, which cannot
  import the package. The heavy dependencies cannot be mocked either: draccus
  runs `register_subclass` at import time and `processor/converters.py` calls
  `functools.singledispatch.register(torch.Tensor)`, which needs a real class.
  `[dataset]` is the only extra required.

Workflow triggers gain `src/**`, since the reference is now generated from
docstrings. `docs/source/api/` is excluded from the prettier hook, which reads
`[[autodoc]]` member lists as lazy paragraph continuations and joins a ten-entry
list onto one line.

Nine API reference pages, scaffolded with each module's base class.

**Doctests.** `LeRobotDocTestParser` is mandatory rather than optional here:
ruff's `docstring-code-format = true` drops the blank line before a closing
fence, after which stdlib's `_EXAMPLE_RE` reads the fence as expected output and
every example with output fails. It is written against the installed pytest
rather than copied from transformers, whose version predates pytest 9's
`import_path` signature and its own fix for the `@property` line-number bug.
`preprocess_string` also diverges: the upstream fenced-block split puts a
single-line example's code in a chunk with no `>>>` in it, so neither the CUDA
skip nor the `+IGNORE_RESULT` injection fires for it.

**Checkers.** `utils/check_docstrings.py` is the ~300-line core of the
2203-line transformers original; the `@auto_docstring` system, modular
propagation, GitPython and `checkers.py` are not ported.
`utils/check_config_docstrings.py` checks that every registered robot config
documents its port and calibration semantics.

**Gates**, all set to values that pass today: ruff `D` with per-file-ignores
per unconverted module, `interrogate` at `fail-under = 52` against a measured
52.1%, and Makefile targets wired into the quality workflow. The doctest
allowlist ships empty and the `doctest` target handles that, because the files
carrying runnable examples arrive with the docstring PR.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 20:57:26 +02:00

213 lines
6.1 KiB
Makefile
Raw Blame History

# Copyright 2024 The HuggingFace Inc. team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
.PHONY: tests
PYTHON_PATH := $(shell which python)
# If uv is installed and a virtual environment exists, use it
UV_CHECK := $(shell command -v uv)
ifneq ($(UV_CHECK),)
PYTHON_PATH := $(shell .venv/bin/python)
endif
export PATH := $(dir $(PYTHON_PATH)):$(PATH)
DEVICE ?= cpu
build-user:
docker build -f docker/Dockerfile.user -t lerobot-user .
build-internal:
docker build -f docker/Dockerfile.internal -t lerobot-internal .
test-end-to-end:
${MAKE} DEVICE=$(DEVICE) test-act-ete-train
${MAKE} DEVICE=$(DEVICE) test-act-ete-train-resume
${MAKE} DEVICE=$(DEVICE) test-act-ete-eval
${MAKE} DEVICE=$(DEVICE) test-diffusion-ete-train
${MAKE} DEVICE=$(DEVICE) test-diffusion-ete-eval
${MAKE} DEVICE=$(DEVICE) test-tdmpc-ete-train
${MAKE} DEVICE=$(DEVICE) test-tdmpc-ete-eval
${MAKE} DEVICE=$(DEVICE) test-smolvla-ete-train
${MAKE} DEVICE=$(DEVICE) test-smolvla-ete-eval
test-act-ete-train:
lerobot-train \
--policy.type=act \
--policy.dim_model=64 \
--policy.n_action_steps=20 \
--policy.chunk_size=20 \
--policy.device=$(DEVICE) \
--policy.push_to_hub=false \
--env.type=aloha \
--env.episode_length=5 \
--dataset.repo_id=lerobot/aloha_sim_transfer_cube_human \
--dataset.image_transforms.enable=true \
--dataset.episodes="[0]" \
--batch_size=2 \
--steps=4 \
--env_eval_freq=2 \
--eval.n_episodes=1 \
--eval.batch_size=1 \
--save_freq=2 \
--save_checkpoint=true \
--log_freq=1 \
--wandb.enable=false \
--output_dir=tests/outputs/act/
test-act-ete-train-resume:
lerobot-train \
--config_path=tests/outputs/act/checkpoints/000002/pretrained_model/train_config.json \
--resume=true
test-act-ete-eval:
lerobot-eval \
--policy.path=tests/outputs/act/checkpoints/000004/pretrained_model \
--policy.device=$(DEVICE) \
--env.type=aloha \
--env.episode_length=5 \
--eval.n_episodes=1 \
--eval.batch_size=1
test-diffusion-ete-train:
lerobot-train \
--policy.type=diffusion \
--policy.down_dims='[64,128,256]' \
--policy.diffusion_step_embed_dim=32 \
--policy.num_inference_steps=10 \
--policy.device=$(DEVICE) \
--policy.push_to_hub=false \
--env.type=pusht \
--env.episode_length=5 \
--dataset.repo_id=lerobot/pusht \
--dataset.image_transforms.enable=true \
--dataset.episodes="[0]" \
--batch_size=2 \
--steps=2 \
--env_eval_freq=2 \
--eval.n_episodes=1 \
--eval.batch_size=1 \
--save_checkpoint=true \
--save_freq=2 \
--log_freq=1 \
--wandb.enable=false \
--output_dir=tests/outputs/diffusion/
test-diffusion-ete-eval:
lerobot-eval \
--policy.path=tests/outputs/diffusion/checkpoints/000002/pretrained_model \
--policy.device=$(DEVICE) \
--env.type=pusht \
--env.episode_length=5 \
--eval.n_episodes=1 \
--eval.batch_size=1
test-tdmpc-ete-train:
lerobot-train \
--policy.type=tdmpc \
--policy.device=$(DEVICE) \
--policy.push_to_hub=false \
--env.type=pusht \
--env.episode_length=5 \
--dataset.repo_id=lerobot/pusht_image \
--dataset.image_transforms.enable=true \
--dataset.episodes="[0]" \
--batch_size=2 \
--steps=2 \
--env_eval_freq=2 \
--eval.n_episodes=1 \
--eval.batch_size=1 \
--save_checkpoint=true \
--save_freq=2 \
--log_freq=1 \
--wandb.enable=false \
--output_dir=tests/outputs/tdmpc/
test-tdmpc-ete-eval:
lerobot-eval \
--policy.path=tests/outputs/tdmpc/checkpoints/000002/pretrained_model \
--policy.device=$(DEVICE) \
--env.type=pusht \
--env.episode_length=5 \
--env.observation_height=96 \
--env.observation_width=96 \
--eval.n_episodes=1 \
--eval.batch_size=1
test-smolvla-ete-train:
lerobot-train \
--policy.type=smolvla \
--policy.n_action_steps=20 \
--policy.chunk_size=20 \
--policy.device=$(DEVICE) \
--policy.push_to_hub=false \
--env.type=aloha \
--env.episode_length=5 \
--dataset.repo_id=lerobot/aloha_sim_transfer_cube_human \
--dataset.image_transforms.enable=true \
--dataset.episodes="[0]" \
--batch_size=2 \
--steps=4 \
--env_eval_freq=2 \
--eval.n_episodes=1 \
--eval.batch_size=1 \
--save_freq=2 \
--save_checkpoint=true \
--log_freq=1 \
--wandb.enable=false \
--output_dir=tests/outputs/smolvla/
test-smolvla-ete-eval:
lerobot-eval \
--policy.path=tests/outputs/smolvla/checkpoints/000004/pretrained_model \
--policy.device=$(DEVICE) \
--env.type=aloha \
--env.episode_length=5 \
--eval.n_episodes=1 \
--eval.batch_size=1
# E2E annotation pipeline smoke test against a tiny in-memory fixture
# dataset. Opt-in (not part of `make test-end-to-end`) and uses a stub VLM
# backend, so it does not require a real model checkpoint or GPU.
annotation-e2e:
uv run python -m tests.annotations.run_e2e_smoke
# Docstring & doctest checks. See docs/source/writing_docstrings.mdx for the standard these enforce.
# Run the examples in the docstrings listed in utils/documentation_tests.txt. Hardware and GPU examples are
# skipped by content (see src/lerobot/utils/doctest_utils.py); CI sets both flags.
doctest:
@files=$$(grep -v '^\s*#' utils/documentation_tests.txt | grep -v '^\s*$$'); \
if [ -z "$$files" ]; then \
echo "utils/documentation_tests.txt lists no files; nothing to run."; \
else \
SKIP_HARDWARE_DOCTEST=1 uv run pytest --doctest-modules --no-header -q $$files; \
fi
check-doctest-list:
uv run python utils/check_doctest_list.py
fix-doctest-list:
uv run python utils/check_doctest_list.py --fix_and_overwrite
check-docstrings:
uv run python utils/check_docstrings.py
uv run python utils/check_config_docstrings.py
fix-docstrings:
uv run python utils/check_docstrings.py --fix_and_overwrite
uv run python utils/check_doctest_list.py --fix_and_overwrite
Reference in New Issue View Git Blame Copy Permalink