From 2e8345a5cce305493dc5827e07d21b559b796681 Mon Sep 17 00:00:00 2001 From: Pepijn Date: Thu, 6 Aug 2026 20:57:26 +0200 Subject: [PATCH] docs: add API documentation infrastructure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit LeRobot's documentation build passes `--not_python_module`, which tells doc-builder there is no importable Python package and disables `[[autodoc]]` entirely. The result is that all 90+ pages are hand-written guides and there is no generated API reference at all. This is the machinery to change that. It deliberately contains no docstring changes of its own — every docstring edit lives in the follow-up PR, so this one can be reviewed as tooling and configuration alone. **The standard.** `docs/source/writing_docstrings.mdx` is the contract: Google section headers with Hugging Face type formatting, the machine-checked argument line, `**Attributes**:`, doc-builder cross-references, fenced doctest examples. It also records three behaviours that are not discoverable from the source and were verified against a local build: `[[autodoc]]` silently skips members with no docstring; doc-builder does not inherit docstrings from base classes, so a registered config shim whose body is `pass` renders every field with no description; and module-level aliases resolve to the canonical class. **Autodoc turned on**, with two changes that are not obvious: - `--version main` on the main-docs job. Without `--not_python_module`, doc-builder resolves the version from `lerobot.__version__` and only maps it to the default branch when it contains "dev". transformers relies on that; our main carries 0.6.2. Verified by building both ways — dropping the flag alone would publish the main docs to /lerobot/v0.6.2/ instead of /lerobot/main/ and disable notebook building. - `pre_command` on both jobs. doc-builder ships a mock-deps registry entry for lerobot, so the reusable workflow takes its light-install path, which cannot import the package. The heavy dependencies cannot be mocked either: draccus runs `register_subclass` at import time and `processor/converters.py` calls `functools.singledispatch.register(torch.Tensor)`, which needs a real class. `[dataset]` is the only extra required. Workflow triggers gain `src/**`, since the reference is now generated from docstrings. `docs/source/api/` is excluded from the prettier hook, which reads `[[autodoc]]` member lists as lazy paragraph continuations and joins a ten-entry list onto one line. Nine API reference pages, scaffolded with each module's base class. **Doctests.** `LeRobotDocTestParser` is mandatory rather than optional here: ruff's `docstring-code-format = true` drops the blank line before a closing fence, after which stdlib's `_EXAMPLE_RE` reads the fence as expected output and every example with output fails. It is written against the installed pytest rather than copied from transformers, whose version predates pytest 9's `import_path` signature and its own fix for the `@property` line-number bug. `preprocess_string` also diverges: the upstream fenced-block split puts a single-line example's code in a chunk with no `>>>` in it, so neither the CUDA skip nor the `+IGNORE_RESULT` injection fires for it. **Checkers.** `utils/check_docstrings.py` is the ~300-line core of the 2203-line transformers original; the `@auto_docstring` system, modular propagation, GitPython and `checkers.py` are not ported. `utils/check_config_docstrings.py` checks that every registered robot config documents its port and calibration semantics. **Gates**, all set to values that pass today: ruff `D` with per-file-ignores per unconverted module, `interrogate` at `fail-under = 52` against a measured 52.1%, and Makefile targets wired into the quality workflow. The doctest allowlist ships empty and the `doctest` target handles that, because the files carrying runnable examples arrive with the docstring PR. Co-Authored-By: Claude Opus 5 --- .github/workflows/documentation.yml | 26 +- .github/workflows/quality.yml | 38 ++ .pre-commit-config.yaml | 13 +- CONTRIBUTING.md | 4 + Makefile | 26 ++ conftest.py | 60 +++ docs/source/_toctree.yml | 22 ++ docs/source/api/cameras.mdx | 24 ++ docs/source/api/configs.mdx | 27 ++ docs/source/api/datasets.mdx | 23 ++ docs/source/api/envs.mdx | 19 + docs/source/api/motors.mdx | 23 ++ docs/source/api/policies.mdx | 20 + docs/source/api/processor.mdx | 20 + docs/source/api/robots.mdx | 147 ++++++++ docs/source/api/teleoperators.mdx | 30 ++ docs/source/writing_docstrings.mdx | 287 ++++++++++++++ pyproject.toml | 81 +++- src/lerobot/utils/doctest_utils.py | 205 ++++++++++ tests/utils/test_doctest_utils.py | 111 ++++++ utils/check_config_docstrings.py | 153 ++++++++ utils/check_docstrings.py | 566 ++++++++++++++++++++++++++++ utils/check_doctest_list.py | 109 ++++++ utils/documentation_tests.txt | 12 + 24 files changed, 2022 insertions(+), 24 deletions(-) create mode 100644 conftest.py create mode 100644 docs/source/api/cameras.mdx create mode 100644 docs/source/api/configs.mdx create mode 100644 docs/source/api/datasets.mdx create mode 100644 docs/source/api/envs.mdx create mode 100644 docs/source/api/motors.mdx create mode 100644 docs/source/api/policies.mdx create mode 100644 docs/source/api/processor.mdx create mode 100644 docs/source/api/robots.mdx create mode 100644 docs/source/api/teleoperators.mdx create mode 100644 docs/source/writing_docstrings.mdx create mode 100644 src/lerobot/utils/doctest_utils.py create mode 100644 tests/utils/test_doctest_utils.py create mode 100644 utils/check_config_docstrings.py create mode 100644 utils/check_docstrings.py create mode 100644 utils/check_doctest_list.py create mode 100644 utils/documentation_tests.txt diff --git a/.github/workflows/documentation.yml b/.github/workflows/documentation.yml index f423d48ee..02d7dd2bf 100644 --- a/.github/workflows/documentation.yml +++ b/.github/workflows/documentation.yml @@ -24,19 +24,24 @@ on: required: false type: string - # Triggers the workflow on push events to main for the docs folder + # Triggers on pushes to main that touch the docs or the sources the API reference is generated from. + # `src/**` is included because the API reference is built from docstrings via `[[autodoc]]`: without it, + # published API pages would go stale as soon as a docstring changed. push: branches: - main paths: - "docs/**" + - "src/**" - # Triggers the workflow on pull request events targeting main for the docs folder + # Same for pull requests, so a docstring change gets a preview build and a broken `[[autodoc]]` path + # fails the PR rather than main. pull_request: branches: - main paths: - "docs/**" + - "src/**" release: types: [published] @@ -59,12 +64,21 @@ jobs: with: commit_sha: ${{ github.sha }} package: lerobot + # doc-builder ships a mock-deps registry entry for lerobot, so the reusable workflow takes its + # "light install" path: `pip install ./lerobot --no-deps` plus a handful of real dependencies. + # That is not enough to import lerobot — draccus runs `register_subclass` at import time and + # `processor/converters.py` calls `functools.singledispatch.register(torch.Tensor)`, neither of + # which works against a mock. Install the package for real before the build. + pre_command: uv pip install "./lerobot[dataset]" + # `--version main` is load-bearing: without `--not_python_module`, doc-builder falls back to + # `lerobot.__version__` and only maps that to the default branch when it contains "dev". Our main + # branch carries a release version (0.6.2), so omitting this would publish the main docs to + # /lerobot/v0.6.2/ instead of /lerobot/main/ and disable notebook building. additional_args: >- - --not_python_module ${{ (github.event_name == 'release' && format('--version {0}', github.event.release.tag_name)) || (inputs.version != '' && format('--version {0}', inputs.version)) || - '' + '--version main' }} secrets: token: ${{ secrets.HUGGINGFACE_PUSH }} @@ -83,4 +97,6 @@ jobs: commit_sha: ${{ github.event.pull_request.head.sha }} pr_number: ${{ github.event.number }} package: lerobot - additional_args: --not_python_module + # See the comment on build_main_docs. The PR workflow passes its own `--version pr_`, so no + # additional_args are needed here. + pre_command: uv pip install "./lerobot[dataset]" diff --git a/.github/workflows/quality.yml b/.github/workflows/quality.yml index d66734491..797fe49fb 100644 --- a/.github/workflows/quality.yml +++ b/.github/workflows/quality.yml @@ -56,3 +56,41 @@ jobs: uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1 with: extra_args: --all-files --show-diff-on-failure --color=always + + # This job runs the examples in our docstrings and validates the doctest allowlist. + # See docs/source/writing_docstrings.mdx for the standard these enforce. + doc-checks: + name: Run Documentation Checks (Doctests) + runs-on: ubuntu-latest + env: + # Examples that need a physical robot, a serial port or a Hub download are skipped by content. + # Everything else has to actually run. See src/lerobot/utils/doctest_utils.py. + SKIP_HARDWARE_DOCTEST: "1" + SKIP_CUDA_DOCTEST: "1" + steps: + - name: Checkout code + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Setup uv and Python + uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 + with: + enable-cache: true + version: "0.11.30" + python-version: "3.12" + + - name: Install dependencies + run: uv sync --locked --extra test --extra dataset + + - name: Check the doctest list is sorted and its paths exist + run: make check-doctest-list + + - name: Check documented arguments match their signatures + run: make check-docstrings + + - name: Check docstring coverage has not regressed + run: uv run --with interrogate interrogate --config=pyproject.toml + + - name: Run doctests + run: make doctest diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 8ae913e4e..6f61ae9c1 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -67,7 +67,11 @@ repos: args: [--prose-wrap=preserve] # Jinja2 model-card templates use a .md extension but contain {% ... %} / # {{ ... }} tags that prettier's Markdown formatter mangles (e.g. table loops). - exclude: ^src/lerobot/templates/.*\.md$ + # + # docs/source/api/ holds the generated API reference. Its `[[autodoc]]` blocks restrict output + # to an indented `- member` list, which prettier reads as a lazy paragraph continuation and + # joins onto one line — silently turning a member list into part of the directive. + exclude: ^(src/lerobot/templates/.*\.md|docs/source/api/.*\.mdx)$ ##### Security ##### - repo: https://github.com/gitleaks/gitleaks @@ -104,8 +108,13 @@ repos: # args: ["--docstring-style", "google", "-v", "2"] # exclude: ^tests/.*$ + # interrogate runs in CI (quality.yml, doc-checks job) rather than here. Its 1.7.0 release still imports + # the deprecated `py` package, which resolves against whatever `py` happens to be importable in + # pre-commit's isolated env — on a machine with miniconda on the path that is a stray `py.py` and the + # hook dies before it reads any config. The gate is the same either way; the CI step is just reliable. # - repo: https://github.com/econchick/interrogate # rev: 1.7.0 # hooks: # - id: interrogate - # args: ["-vv", "--config=pyproject.toml"] + # args: ["--config=pyproject.toml"] + # pass_filenames: false diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 315be3b5d..71aed1dff 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -50,6 +50,10 @@ To run checks manually on all files: pre-commit run --all-files ``` +### Docstrings + +The API reference is generated from the docstrings in `src/lerobot/`. If you add or change anything public, follow the [docstring standard](https://huggingface.co/docs/lerobot/writing_docstrings) — the format is parsed by the renderer and checked in CI. + ### Running Tests We use `pytest`. First, ensure you have test artifacts by installing **git-lfs**: diff --git a/Makefile b/Makefile index ea3b6e261..50dfa0386 100644 --- a/Makefile +++ b/Makefile @@ -184,3 +184,29 @@ test-smolvla-ete-eval: # backend, so it does not require a real model checkpoint or GPU. annotation-e2e: uv run python -m tests.annotations.run_e2e_smoke + +# Docstring & doctest checks. See docs/source/writing_docstrings.mdx for the standard these enforce. + +# Run the examples in the docstrings listed in utils/documentation_tests.txt. Hardware and GPU examples are +# skipped by content (see src/lerobot/utils/doctest_utils.py); CI sets both flags. +doctest: + @files=$$(grep -v '^\s*#' utils/documentation_tests.txt | grep -v '^\s*$$'); \ + if [ -z "$$files" ]; then \ + echo "utils/documentation_tests.txt lists no files; nothing to run."; \ + else \ + SKIP_HARDWARE_DOCTEST=1 uv run pytest --doctest-modules --no-header -q $$files; \ + fi + +check-doctest-list: + uv run python utils/check_doctest_list.py + +fix-doctest-list: + uv run python utils/check_doctest_list.py --fix_and_overwrite + +check-docstrings: + uv run python utils/check_docstrings.py + uv run python utils/check_config_docstrings.py + +fix-docstrings: + uv run python utils/check_docstrings.py --fix_and_overwrite + uv run python utils/check_doctest_list.py --fix_and_overwrite diff --git a/conftest.py b/conftest.py new file mode 100644 index 000000000..f4d9870ac --- /dev/null +++ b/conftest.py @@ -0,0 +1,60 @@ +# Copyright 2026 The HuggingFace Inc. team. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Root conftest: makes doctest collection use LeRobot's parser. + +This only affects `--doctest-modules` runs (see `make doctest`). The test suite itself is configured by +`tests/conftest.py`. +""" + +import doctest + +import _pytest.doctest + +from lerobot.utils.doctest_utils import LeRobotDoctestModule, LeRobotDocTestParser + +# Lets an example opt out of output comparison with `# doctest: +IGNORE_RESULT`, for calls whose output is +# a progress bar or otherwise not reproducible. +IGNORE_RESULT = doctest.register_optionflag("IGNORE_RESULT") + +OutputChecker = doctest.OutputChecker + + +class CustomOutputChecker(OutputChecker): + """An output checker that honours the `IGNORE_RESULT` flag.""" + + def check_output(self, want, got, optionflags): + """Return `True` when `IGNORE_RESULT` is set, otherwise defer to stdlib. + + Args: + want (`str`): + The expected output. + got (`str`): + The actual output. + optionflags (`int`): + Bitmask of active doctest option flags. + + Returns: + `bool`: Whether the output is considered a match. + """ + if IGNORE_RESULT & optionflags: + return True + return OutputChecker.check_output(self, want, got, optionflags) + + +# Reassigning these module attributes is how doctest behaviour is customised; mypy sees it as assigning to +# a type, which is exactly what is intended here. +doctest.OutputChecker = CustomOutputChecker # type: ignore[misc] +_pytest.doctest.DoctestModule = LeRobotDoctestModule +doctest.DocTestParser = LeRobotDocTestParser # type: ignore[misc] diff --git a/docs/source/_toctree.yml b/docs/source/_toctree.yml index f33c57d65..eb2400816 100644 --- a/docs/source/_toctree.yml +++ b/docs/source/_toctree.yml @@ -191,6 +191,28 @@ - sections: - local: contributing title: Contribute to LeRobot + - local: writing_docstrings + title: Writing docstrings - local: backwardcomp title: Backward compatibility title: "About" +- sections: + - local: api/robots + title: Robots + - local: api/teleoperators + title: Teleoperators + - local: api/cameras + title: Cameras + - local: api/motors + title: Motors + - local: api/datasets + title: Datasets + - local: api/policies + title: Policies + - local: api/processor + title: Processors + - local: api/envs + title: Environments + - local: api/configs + title: Configuration + title: "API Reference" diff --git a/docs/source/api/cameras.mdx b/docs/source/api/cameras.mdx new file mode 100644 index 000000000..2b8c83b96 --- /dev/null +++ b/docs/source/api/cameras.mdx @@ -0,0 +1,24 @@ +# Cameras + +Cameras supply the image observations a policy sees. Every backend — OpenCV, Intel RealSense, Reachy 2 — +implements the [`Camera`] interface, so swapping hardware does not change the code that reads frames. + +See the [Cameras guide](../cameras) for choosing and configuring a camera, and +[Third-Party Cameras & Sensors](../third_party_sensors) for devices outside the core set. + +## Camera + +[[autodoc]] lerobot.cameras.Camera + - connect + - disconnect + - read + - async_read + - find_cameras + +## CameraConfig + +[[autodoc]] lerobot.cameras.CameraConfig + +## make_cameras_from_configs + +[[autodoc]] lerobot.cameras.make_cameras_from_configs diff --git a/docs/source/api/configs.mdx b/docs/source/api/configs.mdx new file mode 100644 index 000000000..4a4a9a9f5 --- /dev/null +++ b/docs/source/api/configs.mdx @@ -0,0 +1,27 @@ +# Configuration + +LeRobot configuration is plain dataclasses parsed by [draccus](https://github.com/dlwh/draccus), so every +field is settable from the CLI. [`TrainPipelineConfig`] is the top-level object for `lerobot-train`. + +Polymorphic configs (policies, robots, environments) use `draccus.ChoiceRegistry`: a subclass registers +itself with `@register_subclass("name")` and is then selectable by that name on the command line. + +## TrainPipelineConfig + +[[autodoc]] lerobot.configs.train.TrainPipelineConfig + +## PreTrainedConfig + +[[autodoc]] lerobot.configs.PreTrainedConfig + +## DatasetConfig + +[[autodoc]] lerobot.configs.DatasetConfig + +## EvalConfig + +[[autodoc]] lerobot.configs.EvalConfig + +## WandBConfig + +[[autodoc]] lerobot.configs.WandBConfig diff --git a/docs/source/api/datasets.mdx b/docs/source/api/datasets.mdx new file mode 100644 index 000000000..5dcba7545 --- /dev/null +++ b/docs/source/api/datasets.mdx @@ -0,0 +1,23 @@ +# Datasets + +[`LeRobotDataset`] is the format every LeRobot script reads and writes. It is episode-aware, decodes video +observations on the fly, and round-trips to the Hugging Face Hub. + +See [Using LeRobotDataset](../lerobot-dataset-v3) for the format and the common operations, +[Porting Large Datasets](../porting_datasets_v3) for migration, and [Tools](../tools) for the CLI. + +## LeRobotDataset + +[[autodoc]] lerobot.datasets.LeRobotDataset + +## LeRobotDatasetMetadata + +[[autodoc]] lerobot.datasets.LeRobotDatasetMetadata + +## MultiLeRobotDataset + +[[autodoc]] lerobot.datasets.MultiLeRobotDataset + +## StreamingLeRobotDataset + +[[autodoc]] lerobot.datasets.StreamingLeRobotDataset diff --git a/docs/source/api/envs.mdx b/docs/source/api/envs.mdx new file mode 100644 index 000000000..1552e55b3 --- /dev/null +++ b/docs/source/api/envs.mdx @@ -0,0 +1,19 @@ +# Environments + +Simulation environments are configured through [`EnvConfig`] and built by [`make_env`]. Each subclass +declares its `gym_kwargs` and how to construct the vectorised environments. + +See [Environments from the Hub](../envhub) for using published environments and +[Adding a New Benchmark](../adding_benchmarks) for contributing one. + +## EnvConfig + +[[autodoc]] lerobot.envs.EnvConfig + +## make_env + +[[autodoc]] lerobot.envs.make_env + +## make_env_config + +[[autodoc]] lerobot.envs.make_env_config diff --git a/docs/source/api/motors.mdx b/docs/source/api/motors.mdx new file mode 100644 index 000000000..22ff61566 --- /dev/null +++ b/docs/source/api/motors.mdx @@ -0,0 +1,23 @@ +# Motors + +`MotorsBus` is the low-level interface to a chain of servos on a serial bus. Robots use it to read positions +and write goal positions; you rarely touch it directly unless you are adding hardware. + +See [Bring Your Own Hardware](../integrate_hardware) for adding a new bus, and +[Updating Feetech Firmware](../feetech) and [Damiao Motors and CAN Bus](../damiao) for device-specific notes. + +## MotorsBus + +[[autodoc]] lerobot.motors.motors_bus.MotorsBus + +## Motor + +[[autodoc]] lerobot.motors.Motor + +## MotorCalibration + +[[autodoc]] lerobot.motors.MotorCalibration + +## MotorNormMode + +[[autodoc]] lerobot.motors.MotorNormMode diff --git a/docs/source/api/policies.mdx b/docs/source/api/policies.mdx new file mode 100644 index 000000000..e21a5cc9e --- /dev/null +++ b/docs/source/api/policies.mdx @@ -0,0 +1,20 @@ +# Policies + +Every policy inherits [`PreTrainedPolicy`], which combines a `torch.nn.Module` with the Hub mixin, so any +policy can be pushed to and loaded from the Hugging Face Hub with the same two calls. + +Each policy has its own guide with training recipes and results — [ACT](../act), [SmolVLA](../smolvla), +[π₀](../pi0), [π₀.₅](../pi05) and the rest are listed under Policies. To add one, see +[Adding a Policy](../bring_your_own_policies). + +## PreTrainedPolicy + +[[autodoc]] lerobot.policies.pretrained.PreTrainedPolicy + +## PreTrainedConfig + +[[autodoc]] lerobot.configs.PreTrainedConfig + +## make_policy + +[[autodoc]] lerobot.policies.factory.make_policy diff --git a/docs/source/api/processor.mdx b/docs/source/api/processor.mdx new file mode 100644 index 000000000..daacc877d --- /dev/null +++ b/docs/source/api/processor.mdx @@ -0,0 +1,20 @@ +# Processors + +Processors are the data transformation layer between a robot, a dataset and a policy. A pipeline is a chain +of [`ProcessorStep`]s; each step declares how it transforms both the data and the feature contract. + +See [Introduction to Robot Processors](../introduction_processors) for the concepts, +[Implement your own processor](../implement_your_own_processor) to write a step, and +[Debug your processor pipeline](../debug_processor_pipeline) when a pipeline misbehaves. + +## ProcessorStep + +[[autodoc]] lerobot.processor.pipeline.ProcessorStep + +## DataProcessorPipeline + +[[autodoc]] lerobot.processor.pipeline.DataProcessorPipeline + +## PolicyProcessorPipeline + +[[autodoc]] lerobot.processor.pipeline.PolicyProcessorPipeline diff --git a/docs/source/api/robots.mdx b/docs/source/api/robots.mdx new file mode 100644 index 000000000..84a051a61 --- /dev/null +++ b/docs/source/api/robots.mdx @@ -0,0 +1,147 @@ +# Robots + +Every robot in LeRobot implements the [`Robot`] interface: connect, read an observation, send an action, +disconnect. Writing a policy or a recording script against that interface means it works with any supported +arm without change. + +This page is the generated reference. For wiring, calibration and first-run instructions, start with the +hardware guides — [SO-101](../so101), [LeKiwi](../lekiwi), [Hope Jr](../hope_jr), [Reachy 2](../reachy2), +[OpenArm](../openarm) — or [Imitation Learning for Robots](../il_robots) for the end-to-end workflow. To add +a robot of your own, see [Bring Your Own Hardware](../integrate_hardware). + +## Robot + +The abstract base class. Subclasses implement every method below; the contract described here is what a +policy or recording loop can rely on. + +[[autodoc]] lerobot.robots.Robot + - connect + - disconnect + - configure + - calibrate + - get_observation + - send_action + - observation_features + - action_features + - is_connected + - is_calibrated + +## RobotConfig + +[[autodoc]] lerobot.robots.RobotConfig + +## make_robot_from_config + +[[autodoc]] lerobot.robots.make_robot_from_config + +## SO-100 and SO-101 followers + +`SO100Follower` and `SO101Follower` are aliases of the same `SOFollower` class; the two arms differ in their +configuration, not their control code. `SO100FollowerConfig` and `SO101FollowerConfig` are likewise aliases +of `SOFollowerRobotConfig`. + +[[autodoc]] lerobot.robots.so_follower.SOFollower + - all + +[[autodoc]] lerobot.robots.so_follower.SOFollowerRobotConfig + +## BiSOFollower + +Two SO followers driven as one bimanual robot. + +[[autodoc]] lerobot.robots.bi_so_follower.BiSOFollower + - all + +[[autodoc]] lerobot.robots.bi_so_follower.BiSOFollowerConfig + +## KochFollower + +[[autodoc]] lerobot.robots.koch_follower.KochFollower + - all + +[[autodoc]] lerobot.robots.koch_follower.KochFollowerConfig + +## LeKiwi + +`LeKiwi` runs on the robot itself. `LeKiwiClient` is the host-side proxy that talks to it over the network +and presents the same [`Robot`] interface. + +[[autodoc]] lerobot.robots.lekiwi.LeKiwi + - all + +[[autodoc]] lerobot.robots.lekiwi.LeKiwiConfig + +[[autodoc]] lerobot.robots.lekiwi.LeKiwiClient + - all + +[[autodoc]] lerobot.robots.lekiwi.LeKiwiClientConfig + +## OpenArmFollower + +[[autodoc]] lerobot.robots.openarm_follower.OpenArmFollower + - all + +[[autodoc]] lerobot.robots.openarm_follower.OpenArmFollowerConfig + +## BiOpenArmFollower + +[[autodoc]] lerobot.robots.bi_openarm_follower.BiOpenArmFollower + - all + +[[autodoc]] lerobot.robots.bi_openarm_follower.BiOpenArmFollowerConfig + +## OmxFollower + +[[autodoc]] lerobot.robots.omx_follower.OmxFollower + - all + +[[autodoc]] lerobot.robots.omx_follower.OmxFollowerConfig + +## Reachy2Robot + +[[autodoc]] lerobot.robots.reachy2.Reachy2Robot + - all + +[[autodoc]] lerobot.robots.reachy2.Reachy2RobotConfig + +## UnitreeG1 + +[[autodoc]] lerobot.robots.unitree_g1.UnitreeG1 + - all + +[[autodoc]] lerobot.robots.unitree_g1.UnitreeG1Config + +## Hope Jr + +The Hope Jr humanoid is exposed as two independent robots, an arm and a hand. + +[[autodoc]] lerobot.robots.hope_jr.HopeJrArm + - all + +[[autodoc]] lerobot.robots.hope_jr.HopeJrArmConfig + +[[autodoc]] lerobot.robots.hope_jr.HopeJrHand + - all + +[[autodoc]] lerobot.robots.hope_jr.HopeJrHandConfig + +## RebotB601Follower + +[[autodoc]] lerobot.robots.rebot_b601_follower.RebotB601Follower + - all + +[[autodoc]] lerobot.robots.rebot_b601_follower.RebotB601FollowerRobotConfig + +## BiRebotB601Follower + +[[autodoc]] lerobot.robots.bi_rebot_b601_follower.BiRebotB601Follower + - all + +[[autodoc]] lerobot.robots.bi_rebot_b601_follower.BiRebotB601FollowerConfig + +## EarthRoverMiniPlus + +[[autodoc]] lerobot.robots.earthrover_mini_plus.EarthRoverMiniPlus + - all + +[[autodoc]] lerobot.robots.earthrover_mini_plus.EarthRoverMiniPlusConfig diff --git a/docs/source/api/teleoperators.mdx b/docs/source/api/teleoperators.mdx new file mode 100644 index 000000000..b890b73c2 --- /dev/null +++ b/docs/source/api/teleoperators.mdx @@ -0,0 +1,30 @@ +# Teleoperators + +A teleoperator produces actions for a robot to follow — a leader arm, a gamepad, a keyboard, a phone. All of +them implement the [`Teleoperator`] interface, so a recording script written against it works with any input +device. + +See [Phone teleoperation](../phone_teleop) and [Isaac Teleop](../isaac_teleop) for setup guides, and +[Imitation Learning for Robots](../il_robots) for the recording workflow. + +## Teleoperator + +[[autodoc]] lerobot.teleoperators.Teleoperator + - connect + - disconnect + - configure + - calibrate + - get_action + - send_feedback + - action_features + - feedback_features + - is_connected + - is_calibrated + +## TeleoperatorConfig + +[[autodoc]] lerobot.teleoperators.TeleoperatorConfig + +## make_teleoperator_from_config + +[[autodoc]] lerobot.teleoperators.make_teleoperator_from_config diff --git a/docs/source/writing_docstrings.mdx b/docs/source/writing_docstrings.mdx new file mode 100644 index 000000000..c19d70595 --- /dev/null +++ b/docs/source/writing_docstrings.mdx @@ -0,0 +1,287 @@ +# Writing docstrings + +LeRobot's API reference is generated directly from the docstrings in `src/lerobot/`. A docstring is not a +comment — it is the published documentation for that object, and the format below is what the renderer and +the CI checks parse. + +This page is the contract. If you are adding or editing anything public in `src/lerobot/`, follow it. + +> [!IMPORTANT] +> **An undocumented public method is an invisible one.** `[[autodoc]]` silently skips members that have no +> docstring — no warning, no error, it simply does not appear on the rendered page. Coverage and +> API-reference completeness are the same problem. + +## The format in one example + +Google section headers, Hugging Face type formatting. Both, not one or the other. + +````python +def send_action(self, action: RobotAction, rate_hz: float = 30.0) -> RobotAction: + """Command the robot to move to a target joint configuration. + + Values are clipped by the configured maximum relative target before reaching the motors, so the + returned action may differ from the requested one. + + Args: + action (`dict[str, float]`): + Target values keyed by motor name, e.g. `{"shoulder_pan.pos": 0.0}`. Keys must match the + robot's action features. + rate_hz (`float`, *optional*, defaults to `30.0`): + Control loop frequency. + + Returns: + `dict[str, float]`: The action actually written to the motors after safety clipping. + + Raises: + DeviceNotConnectedError: If the robot has not been connected. + + Example: + ```python + >>> from lerobot.robots.so_follower import SO101Follower, SO101FollowerConfig + >>> robot = SO101Follower(SO101FollowerConfig(port="/dev/ttyACM0")) # doctest: +SKIP + >>> robot.connect() # doctest: +SKIP + >>> robot.send_action({"shoulder_pan.pos": 0.0}) # doctest: +SKIP + ``` + """ +```` + +Cross-references are omitted from the examples on this page — see [Cross-references](#cross-references) for +their syntax and why they cannot be shown inside a code block. + +## Rules + +### Sections + +`Args:` · `Returns:` · `Raises:` · `Yields:` · `Example:` · `Note:` + +In that order. No other section headers. A one-line summary comes first, then an optional free-form +description, then the sections. + +### The `Args:` line is machine-parsed + +``` +name (`type`, *optional*, defaults to `X`): + Description, indented on its own line. +``` + +The `*optional*, defaults to` clause is **checked against the real signature default** by +`make check-docstrings`. It is not decorative — if you write a default that has drifted from the code, CI +fails. Omit the clause entirely for required parameters: + +```python + Args: + port (`str`): + Serial port the arm is connected to, e.g. `/dev/ttyACM0`. + max_relative_target (`float | dict[str, float]`, *optional*): + Caps the magnitude of the relative positional target vector. `None` disables clipping. + use_degrees (`bool`, *optional*, defaults to `True`): + Keep `True` for backward compatibility with existing policies and datasets. +``` + +Types go in backticks. Use `*optional*` with no `defaults to` when the default is `None` or is otherwise not +worth restating. + +### `Returns:` is type-first + +One indented line, type first, then a colon, then the description: + +```python + Returns: + `dict[str, float]`: The action actually written to the motors after safety clipping. +``` + +`Yields:` takes the same shape. + +### `**Attributes**:`, never `Attributes:` + +doc-builder parses a bare `Attributes:` as a **synonym for `Parameters:`**, so your attributes get rendered +as constructor arguments. This is silent and wrong. Whenever the attributes differ from the constructor +parameters, use the bold form with a `--` separator: + +```python +class Robot(abc.ABC): + """The base abstract class for all LeRobot-compatible robots. + + **Attributes**: + - **config_class** (`type[RobotConfig]`) -- The expected configuration class for this robot. + - **name** (`str`) -- The unique robot name used to identify this robot type. + """ +``` + +Note `--`, not `:`. + +### Cross-references + +Use doc-builder's bracket syntax: a square-bracketed backtick-quoted path. **Sphinx roles (`:pymeth:`, +`:pyattr:`) are not supported** and render as literal text on the page. + +| Want | Write | +| ---------------------------- | ----------------------------------- | +| Class in the main package | [`Robot`] | +| Method, show the full path | [`Robot.connect`] | +| Method, show the bare name | [`~Robot.connect`] | +| Nested path | [`~robots.Robot.connect`] | +| Object in another HF library | [`~accelerate.Accelerator`] | + +The `~` strips the path from the **link text only**; the link still resolves to the full path. + +> [!NOTE] +> doc-builder resolves this syntax everywhere in a page — including inside fenced code blocks. That is why +> the docstring examples on this page use plain prose instead of cross-references: a code block containing +> one would render the resolved link rather than the syntax you need to type. In your own docstrings, use +> cross-references freely; this restriction only affects documentation _about_ the syntax. + +### Callouts + +Use GitHub-style blockquotes: + +```markdown +> [!TIP] +> Call this once at startup — it takes about two seconds. + +> [!WARNING] +> Torque is disabled on disconnect. The arm will drop if it is holding a load. +``` + +The `` component is legacy per doc-builder; don't add new ones. + +### Examples must be fenced + +An example lives inside a fenced ` ```python ` block containing `>>> `. The fence is what makes it render +as a code block, and it is what the doctest preprocessor's regex looks for: + +````python + Example: + ```python + >>> from lerobot.robots.so_follower import SO101FollowerConfig + >>> cfg = SO101FollowerConfig(port="/dev/ttyACM0") + >>> cfg.use_degrees + True + ``` +```` + +> [!WARNING] +> An unfenced `>>>` is still collected — doctest finds prompts anywhere in a docstring. What you lose is the +> rendering, so it shows up as a wall of prose on the page. Every example needs the fence. + +Every example either executes in CI or carries `# doctest: +SKIP`. Anything that touches hardware, a GPU, or +downloads from the Hub gets `+SKIP`: + +````python + Example: + ```python + >>> robot.connect() # doctest: +SKIP + >>> policy = ACTPolicy.from_pretrained("lerobot/act_aloha_sim_transfer_cube_human") # doctest: +SKIP + ``` +```` + +Add files containing runnable examples to `utils/documentation_tests.txt`. + +Put examples on the three to five genuine entry points of a module. Examples on trivial accessors are noise. + +## Three patterns you will hit constantly + +### Config dataclasses + +Configuration fields are historically documented with `#` comments above each field. **doc-builder cannot +see inline comments** — such a class renders with every field listed and not a single description. Move them +into an `Args:` block on the class docstring: + +```python +@dataclass +class SOFollowerConfig: + """Configuration for SO-family follower arms. + + Args: + port (`str`): + Serial port the arm is connected to, e.g. `/dev/ttyACM0`. + max_relative_target (`float | dict[str, float]`, *optional*): + Caps the magnitude of the relative positional target vector. A scalar applies to all motors; + a dict maps motor name to a per-motor cap. `None` disables clipping. + use_degrees (`bool`, *optional*, defaults to `True`): + Keep `True` for backward compatibility with existing policies and datasets. + """ + + port: str + max_relative_target: float | dict[str, float] | None = None + use_degrees: bool = True +``` + +> [!IMPORTANT] +> **doc-builder does not inherit docstrings from base classes.** LeRobot's registered config classes are +> often thin multiple-inheritance shims: +> +> ```python +> @RobotConfig.register_subclass("so101_follower") +> @dataclass +> class SOFollowerRobotConfig(RobotConfig, SOFollowerConfig): +> pass +> ``` +> +> That class renders **every** field — including the ones it inherits — with no descriptions at all, no +> matter how well the bases are documented. The `Args:` block must live on the concrete class that +> `[[autodoc]]` names, and it must cover inherited fields too. + +### Base class, then concrete subclass + +The abstract base carries the canonical contract. Subclasses document only what deviates — port semantics, +calibration quirks, motor layout, supported feature keys. Do not copy the base contract into every subclass. + +`Robot`, `Teleoperator`, `Camera`, `MotorsBus`, `ProcessorStep`, and `PreTrainedPolicy` all follow this +shape. + +### Module-level aliases + +Several public names are aliases rather than distinct classes: + +```python +SO100FollowerConfig = SOFollowerRobotConfig +SO101FollowerConfig = SOFollowerRobotConfig +``` + +`[[autodoc]]` resolves the alias and renders the **canonical** class name, so a `## SO101FollowerConfig` +heading will show `class lerobot.robots.so_follower.SOFollowerRobotConfig` in the body. Document the +canonical class once, and mention the aliases in the page's prose rather than giving each alias its own +autodoc block. + +## What not to document + +- **Private members.** Anything starting with `_` is not part of the public API. +- **The type annotation restated as prose.** `port (`str`): A string.` adds nothing. Say what it is for. +- **Vendored upstream code.** `src/lerobot/policies/molmoact2/molmoact2_hf_model/` is vendored from + `transformers` and already carries upstream-style docstrings. Leave it alone — restyling it only creates + conflicts on the next sync. It is excluded from the API reference and from the docstring checks. + +## How this is enforced + +| Check | What it catches | +| ------------------------- | ---------------------------------------------------------------------------------------------------------- | +| `make check-docstrings` | An `Args:` entry that doesn't match the signature; a documented default that has drifted from the real one | +| `make doctest` | Examples that no longer run | +| `make check-doctest-list` | Stale or unsorted entries in `utils/documentation_tests.txt` | +| `ruff` (`D` rules) | Google-convention style violations | +| `interrogate` | Docstring coverage falling below the current threshold | +| doc-builder | A `[[autodoc]]` path that points at something that doesn't exist — this breaks the docs build | + +Run them together before opening a PR: + +```bash +make check-docstrings && make doctest && pre-commit run --all-files +``` + +Then render the page and actually look at it: + +```bash +doc-builder build lerobot docs/source/ --build_dir /tmp/doc-build +``` + +## Checklist + +- [ ] Every public member you touched has a docstring. +- [ ] Every `Args:` entry matches the signature, including the `*optional*, defaults to` clause. +- [ ] `Returns:` is type-first on one indented line. +- [ ] No bare `Attributes:` — use `**Attributes**:` with `--` separators. +- [ ] No Sphinx roles — cross-references use [`~module.Class.method`]. +- [ ] Examples are inside a fenced ` ```python ` block, and either run in CI or carry `# doctest: +SKIP`. +- [ ] Config dataclass fields are in an `Args:` block on the concrete class, not `#` comments. +- [ ] The rendered page has been eyeballed. diff --git a/pyproject.toml b/pyproject.toml index 4a9ac404d..e080b6784 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -401,7 +401,7 @@ exclude = ["tests/artifacts/**/*.safetensors", "*_pb2.py", "*_pb2_grpc.py"] # N: pep8-naming # TODO: Uncomment rules when ready to use select = [ - "E", "W", "F", "I", "B", "C4", "T20", "N", "UP", "SIM" #, "A", "S", "D", "RUF" + "E", "W", "F", "I", "B", "C4", "T20", "N", "UP", "SIM", "D" #, "A", "S", "RUF" ] ignore = [ "E501", # Line too long @@ -411,9 +411,53 @@ ignore = [ ] [tool.ruff.lint.per-file-ignores] -"__init__.py" = ["F401", "F403", "E402"] +"__init__.py" = ["F401", "F403", "E402", "D104"] # E402: conditional-import guards (TYPE_CHECKING / is_package_available) must precede the imports they protect "src/lerobot/scripts/convert_dataset_v21_to_v30.py" = ["E402"] + +# D (pydocstyle) is enabled globally, but only holds for code that has been converted to the docstring +# standard in docs/source/writing_docstrings.mdx. Every module below is still on the old style; each entry +# is deleted as that module is converted, and this block can be removed once it is empty. +# +# Not part of the API reference and not planned for conversion: tests, examples, benchmarks, templates, +# CI helper scripts and the packaging shim. +"tests/**" = ["D"] +"examples/**" = ["D"] +"benchmarks/**" = ["D"] +"scripts/**" = ["D"] +"setup.py" = ["D"] +"src/lerobot/templates/**" = ["D"] +# Vendored from transformers; keeps its upstream docstring style so syncs stay clean. +"src/lerobot/policies/molmoact2/molmoact2_hf_model/**" = ["D"] +# Awaiting conversion, one PR per module. +"src/lerobot/annotations/**" = ["D"] +"src/lerobot/async_inference/**" = ["D"] +"src/lerobot/cameras/**" = ["D"] +"src/lerobot/common/**" = ["D"] +"src/lerobot/configs/**" = ["D"] +"src/lerobot/data_processing/**" = ["D"] +"src/lerobot/datasets/**" = ["D"] +"src/lerobot/distributed/**" = ["D"] +"src/lerobot/envs/**" = ["D"] +"src/lerobot/jobs/**" = ["D"] +"src/lerobot/model/**" = ["D"] +"src/lerobot/motors/**" = ["D"] +"src/lerobot/optim/**" = ["D"] +"src/lerobot/policies/**" = ["D"] +"src/lerobot/processor/**" = ["D"] +"src/lerobot/rewards/**" = ["D"] +"src/lerobot/rl/**" = ["D"] +"src/lerobot/robots/**" = ["D"] +"src/lerobot/rollout/**" = ["D"] +"src/lerobot/scripts/**" = ["D"] +"src/lerobot/teleoperators/**" = ["D"] +"src/lerobot/transforms/**" = ["D"] +"src/lerobot/transport/**" = ["D"] +"src/lerobot/utils/**" = ["D"] +"src/lerobot/lerobot_types.py" = ["D"] +# Package root: two one-line docstring fixes land with the docstring PR. +"src/lerobot/__init__.py" = ["D"] +"src/lerobot/__version__.py" = ["D"] [tool.ruff.lint.isort] combine-as-imports = true known-first-party = ["lerobot"] @@ -457,21 +501,24 @@ default.extend-ignore-identifiers-re = [ "seperated_timestep", ] -# TODO: Uncomment when ready to use -# [tool.interrogate] -# ignore-init-module = true -# ignore-init-method = true -# ignore-nested-functions = false -# ignore-magic = false -# ignore-semiprivate = false -# ignore-private = false -# ignore-property-decorators = false -# ignore-module = false -# ignore-setters = false -# fail-under = 80 -# output-format = "term-missing" -# color = true -# paths = ["src/lerobot"] +# Docstring coverage gate. `fail-under` is a RATCHET, not a target: it is set just below the currently +# measured coverage so it passes today, and is raised in the same PR that documents a module. Never set it +# to a value that fails on main. The destination is 100; see docs/source/writing_docstrings.mdx. +[tool.interrogate] +ignore-init-module = true +ignore-init-method = true +ignore-nested-functions = false +ignore-magic = false +ignore-semiprivate = false +ignore-private = false +ignore-property-decorators = false +ignore-module = false +ignore-setters = false +fail-under = 52 +output-format = "term-missing" +color = true +paths = ["src/lerobot"] +exclude = ["src/lerobot/policies/molmoact2/molmoact2_hf_model"] # TODO: Enable mypy gradually module by module across multiple PRs # Uncomment [tool.mypy] first, then uncomment individual module overrides as they get proper type annotations diff --git a/src/lerobot/utils/doctest_utils.py b/src/lerobot/utils/doctest_utils.py new file mode 100644 index 000000000..32c22bc5d --- /dev/null +++ b/src/lerobot/utils/doctest_utils.py @@ -0,0 +1,205 @@ +# Copyright 2026 The HuggingFace Inc. team. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Doctest plumbing so the examples in our docstrings actually run. + +Adapted from `transformers.testing_utils`. Two stdlib limitations make this necessary: + +1. Ruff is configured with `docstring-code-format = true`, which reformats code inside docstrings and + removes the blank line before the closing fence. stdlib's `_EXAMPLE_RE` then swallows the ` ``` ` into + the expected-output group, so every example that has output fails. [`LeRobotDocTestParser`] patches the + regex to stop at a fence. +2. `doctest.DocTestFinder` reports the wrong line number for `@property` and `functools.wraps` objects + (https://bugs.python.org/issue17446). Our hardware API is property-heavy — `observation_features`, + `action_features`, `is_connected`, `is_calibrated` are all abstract properties — so + [`LeRobotDoctestModule`] unwraps them before locating the example. + +Two environment variables skip whole example blocks by content: + +- `SKIP_CUDA_DOCTEST=1` skips examples that need a GPU. +- `SKIP_HARDWARE_DOCTEST=1` skips examples that need a physical robot or a Hub download. + +Both are heuristics over the example source. They are deliberately blunt: an example that is skipped +needlessly costs nothing, whereas one that runs on a machine without the hardware hangs or fails. +""" + +import doctest +import functools +import inspect +import os +import re +import sys +from collections.abc import Iterable + +from _pytest.doctest import ( + DoctestItem, + DoctestModule, + _get_checker, + _get_continue_on_failure, + _get_runner, + get_optionflags, +) +from _pytest.nodes import Collector +from _pytest.outcomes import skip + +# Calls whose progress bars would otherwise be compared against the expected output. The lookahead leaves +# lines that already carry a directive alone. +_NOISY_CALL_PATTERN = re.compile(r"(>>> (?!.*# doctest:).*(?:load_dataset|LeRobotDataset)\(.*)") + +_CUDA_PATTERN = re.compile(r"cuda|to\(0\)|device=0") + +# Serial ports, video devices, and the connect/scan calls that talk to real hardware. +_HARDWARE_PATTERN = re.compile(r"/dev/tty|/dev/video|COM\d|\.connect\(|find_cameras\(|find_port\(") + +# Anything that reaches the Hub over the network. +_HUB_PATTERN = re.compile(r"from_pretrained\(|push_to_hub\(|snapshot_download\(|load_dataset\(") + + +def preprocess_string(string: str, skip_cuda_tests: bool, skip_hardware_tests: bool) -> str: + """Prepare a docstring or `.mdx` file to be run by doctest. + + Args: + string (`str`): + A whole file's contents for `.mdx`, or a single docstring for a Python file. Either may hold + several fenced examples. + skip_cuda_tests (`bool`): + Whether to drop examples that look like they need a GPU. + skip_hardware_tests (`bool`): + Whether to drop examples that look like they need a robot or a Hub download. + + Returns: + `str`: The input with `# doctest: +IGNORE_RESULT` injected on noisy calls, or an empty string if + the examples were skipped — in which case no doctest is collected for it at all. + """ + # Match against the example lines only, not the surrounding prose, so that a docstring merely + # *describing* CUDA or a serial port is not mistaken for one that uses them. + example_lines = "\n".join( + line for line in string.splitlines() if line.lstrip().startswith((">>>", "...")) + ) + if not example_lines: + return string + + if skip_cuda_tests and _CUDA_PATTERN.search(example_lines): + return "" + if skip_hardware_tests and ( + _HARDWARE_PATTERN.search(example_lines) or _HUB_PATTERN.search(example_lines) + ): + return "" + + return _NOISY_CALL_PATTERN.sub(r"\1 # doctest: +IGNORE_RESULT", string) + + +class LeRobotDocTestParser(doctest.DocTestParser): + """A `DocTestParser` that understands fenced, auto-formatted code blocks. + + Ruff's `docstring-code-format` removes the blank line before a closing fence, after which stdlib's + `_EXAMPLE_RE` reads the fence itself as part of the expected output and every example with output + fails. The regex below is the stdlib one plus a clause that stops matching at a fence. + """ + + # fmt: off + _EXAMPLE_RE = re.compile(r''' + # Source consists of a PS1 line followed by zero or more PS2 lines. + (?P + (?:^(?P [ ]*) >>> .*) # PS1 line + (?:\n [ ]* \.\.\. .*)*) # PS2 lines + \n? + # Want consists of any non-blank lines that do not start with PS1. + (?P (?:(?![ ]*$) # Not a blank line + (?![ ]*>>>) # Not a line starting with PS1 + (?:(?!```).)* # Stop at a closing fence: formatting drops the blank line before it + (?:\n|$) # Match a new line or end of string + )*) + ''', re.MULTILINE | re.VERBOSE + ) + # fmt: on + + skip_cuda_tests: bool = os.environ.get("SKIP_CUDA_DOCTEST", "0") == "1" + skip_hardware_tests: bool = os.environ.get("SKIP_HARDWARE_DOCTEST", "0") == "1" + + def parse(self, string, name=""): + """Preprocess `string`, then parse it as stdlib would. + + Args: + string (`str`): + The docstring or file contents to parse. + name (`str`, *optional*, defaults to `""`): + Name used in failure messages. + + Returns: + `list`: The examples and interleaved text, as returned by `doctest.DocTestParser.parse`. + """ + string = preprocess_string(string, self.skip_cuda_tests, self.skip_hardware_tests) + return super().parse(string, name) + + +class LeRobotDoctestModule(DoctestModule): + """A pytest `DoctestModule` that collects with [`LeRobotDocTestParser`]. + + `doctest.DocTestFinder` binds its default parser at class-definition time, so patching + `doctest.DocTestParser` in `conftest.py` does not reach the finder pytest builds. The parser has to be + passed in explicitly, which means reimplementing `collect`. It mirrors pytest's own implementation. + """ + + def collect(self) -> Iterable[DoctestItem]: + """Collect the doctests in this module. + + Returns: + `Iterable[DoctestItem]`: One item per example-bearing docstring. Docstrings whose examples were + dropped by `preprocess_string` yield nothing. + """ + + class MockAwareDocTestFinder(doctest.DocTestFinder): + """A doctest finder that reports correct line numbers for properties and wrapped callables.""" + + # Fixed upstream in CPython 3.11.9 / 3.12.3; kept for older interpreters. Our hardware API is + # property-heavy (`observation_features`, `is_connected`, ...), so a wrong line number here + # would point every failure at the decorator. https://github.com/python/cpython/issues/61648 + def _find_lineno(self, obj, source_lines): + if isinstance(obj, property): + obj = getattr(obj, "fget", obj) + if hasattr(obj, "__wrapped__"): + obj = inspect.unwrap(obj) + return super()._find_lineno(obj, source_lines) + + if sys.version_info < (3, 13): + # `cached_property` is otherwise never considered part of the current module and its + # examples are silently skipped. https://github.com/python/cpython/issues/107995 + def _from_module(self, module, object): + if isinstance(object, functools.cached_property): + object = object.func + return super()._from_module(module, object) + + try: + module = self.obj + except Collector.CollectError: + if self.config.getvalue("doctest_ignore_import_errors"): + skip(f"unable to import module {self.path!r}") + else: + raise + + # Doctests support fixtures via `getfixture` and autouse. + self.session._fixturemanager.parsefactories(self) + + finder = MockAwareDocTestFinder(parser=LeRobotDocTestParser()) + optionflags = get_optionflags(self.config) + runner = _get_runner( + verbose=False, + optionflags=optionflags, + checker=_get_checker(), + continue_on_failure=_get_continue_on_failure(self.config), + ) + for test in finder.find(module, module.__name__): + if test.examples: # Skip docstrings with no examples, and blocks dropped by the parser. + yield DoctestItem.from_parent(self, name=test.name, runner=runner, dtest=test) diff --git a/tests/utils/test_doctest_utils.py b/tests/utils/test_doctest_utils.py new file mode 100644 index 000000000..dc4432d57 --- /dev/null +++ b/tests/utils/test_doctest_utils.py @@ -0,0 +1,111 @@ +# Copyright 2026 The HuggingFace Inc. team. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import doctest + +from lerobot.utils.doctest_utils import LeRobotDocTestParser, preprocess_string + +# An example with expected output, formatted the way ruff's `docstring-code-format` leaves it: no blank +# line between the last output line and the closing fence. This is the exact shape that breaks stdlib. +FORMATTED_EXAMPLE = """Summary. + +Example: + ```python + >>> 1 + 1 + 2 + ``` +""" + + +def test_stdlib_parser_swallows_the_closing_fence(): + """Guards the premise of the port: without the patch, the fence lands in the expected output. + + Uses the base class rather than `doctest.DocTestParser`, which the root `conftest.py` has already + replaced with ours by the time this runs. + """ + stdlib_parser = LeRobotDocTestParser.__bases__[0]() + (example,) = (e for e in stdlib_parser.parse(FORMATTED_EXAMPLE) if isinstance(e, doctest.Example)) + assert "```" in example.want + + +def test_parser_stops_at_the_closing_fence(): + """The whole reason `LeRobotDocTestParser` exists: `want` must be the output and nothing else.""" + (example,) = ( + e for e in LeRobotDocTestParser().parse(FORMATTED_EXAMPLE) if isinstance(e, doctest.Example) + ) + assert example.source == "1 + 1\n" + assert example.want == "2\n" + + +def test_example_with_output_passes_end_to_end(): + """A formatted example with output should actually run green.""" + runner = doctest.DocTestRunner() + test = LeRobotDocTestParser().get_doctest(FORMATTED_EXAMPLE, {}, "formatted", None, 0) + results = runner.run(test, out=lambda _: None) + assert results.failed == 0 + assert results.attempted == 1 + + +def test_noisy_calls_get_ignore_result(): + string = """ + ```python + >>> ds = load_dataset("lerobot/pusht") + ``` + """ + assert "# doctest: +IGNORE_RESULT" in preprocess_string(string, False, False) + + +def test_ignore_result_is_not_added_twice(): + string = """ + ```python + >>> ds = load_dataset("lerobot/pusht") # doctest: +IGNORE_RESULT + ``` + """ + assert preprocess_string(string, False, False).count("# doctest: +IGNORE_RESULT") == 1 + + +def test_cuda_examples_are_dropped_when_requested(): + string = """ + ```python + >>> model.to("cuda") + ``` + """ + assert preprocess_string(string, True, False) == "" + assert preprocess_string(string, False, False) != "" + + +def test_hardware_examples_are_dropped_when_requested(): + """Serial ports, connect calls and Hub downloads all need real resources.""" + for source in [ + '>>> robot = SO101Follower(SO101FollowerConfig(port="/dev/ttyACM0"))', + ">>> robot.connect()", + '>>> policy = ACTPolicy.from_pretrained("lerobot/act")', + ]: + string = f""" + ```python + {source} + ``` + """ + assert preprocess_string(string, False, True) == "", source + assert preprocess_string(string, False, False) != "", source + + +def test_plain_examples_survive_both_skips(): + string = """ + ```python + >>> 1 + 1 + 2 + ``` + """ + assert preprocess_string(string, True, True) == string diff --git a/utils/check_config_docstrings.py b/utils/check_config_docstrings.py new file mode 100644 index 000000000..713f40815 --- /dev/null +++ b/utils/check_config_docstrings.py @@ -0,0 +1,153 @@ +# Copyright 2026 The HuggingFace Inc. team. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Check that every registered hardware config documents the fields users have to get right. + +Modelled on `transformers/utils/check_config_docstrings.py`, which checks that every model config links a +checkpoint. LeRobot's equivalent question is the one every new user hits: which port is the device on, and +what happens on calibration. A config that leaves those undocumented sends people to the source. + +Only fields the config actually declares are required — a config without a `port` is not asked to document +one. + +```bash +python utils/check_config_docstrings.py +``` +""" + +import inspect +import re +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from check_docstrings import _re_args, _re_parse_arg, find_indent, iter_objects_to_check # noqa: E402 + +# Fields whose semantics are not obvious from the name and that a user must set correctly on first run. +REQUIRED_FIELDS = ["port"] + +# A config must say something about calibration if it participates in it at all. +CALIBRATION_PATTERN = re.compile(r"calibrat", re.IGNORECASE) + +MODULES_TO_CHECK = ["lerobot.robots"] + +# Configs that document their fields with `#` comments above each field, which doc-builder cannot see. +# Each entry is removed as that config's comments are converted to an `Args:` block. +OBJECTS_TO_IGNORE: set[str] = { + "BiOpenArmFollowerConfig", + "BiRebotB601FollowerConfig", + "BiSOFollowerConfig", + "EarthRoverMiniPlusConfig", + "HopeJrArmConfig", + "HopeJrHandConfig", + "KochFollowerConfig", + "LeKiwiConfig", + "OmxFollowerConfig", + "OpenArmFollowerConfig", + "Reachy2RobotConfig", + "RebotB601FollowerRobotConfig", + "SOFollowerRobotConfig", +} + + +def documented_args(obj: object) -> set[str]: + """Return the argument names documented in an object's `Args:` block. + + Args: + obj (`object`): + The class to inspect. + + Returns: + `set[str]`: The documented argument names, empty if there is no `Args:` section. + """ + doc = getattr(obj, "__doc__", None) + if not doc: + return set() + + lines = doc.split("\n") + idx = 0 + while idx < len(lines) and _re_args.search(lines[idx]) is None: + idx += 1 + if idx == len(lines): + return set() + + indent = find_indent(lines[idx]) + names = set() + idx += 1 + while idx < len(lines) and (len(lines[idx].strip()) == 0 or find_indent(lines[idx]) > indent): + if find_indent(lines[idx]) == indent + 4: + match = _re_parse_arg.search(lines[idx]) + if match is not None: + names.add(match.groups()[1]) + idx += 1 + return names + + +def check_config_docstrings() -> list[str]: + """Check every registered config in `MODULES_TO_CHECK`. + + Returns: + `list[str]`: One message per config that is missing a required field or calibration semantics. + """ + from lerobot.robots import RobotConfig + + failures = [] + for module_name in MODULES_TO_CHECK: + for obj in iter_objects_to_check(module_name): + if not inspect.isclass(obj) or not issubclass(obj, RobotConfig) or obj is RobotConfig: + continue + if inspect.isabstract(obj) or obj.__qualname__ in OBJECTS_TO_IGNORE: + continue + + try: + fields = set(inspect.signature(obj).parameters) + except (TypeError, ValueError): + continue + + doc = getattr(obj, "__doc__", "") or "" + documented = documented_args(obj) + name = f"{obj.__module__}.{obj.__qualname__}" + + for field in REQUIRED_FIELDS: + if field in fields and field not in documented: + failures.append(f"{name}: does not document `{field}`") + + if "calibration_dir" in fields and CALIBRATION_PATTERN.search(doc) is None: + failures.append(f"{name}: says nothing about calibration") + + return failures + + +def main() -> int: + """Run the check. + + Returns: + `int`: `0` when every registered config is documented, `1` otherwise. + """ + failures = check_config_docstrings() + if failures: + print( + "The following robot configs are missing documentation a user needs on first run. See " + "docs/source/writing_docstrings.mdx:", + file=sys.stderr, + ) + for failure in failures: + print(f"- {failure}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/utils/check_docstrings.py b/utils/check_docstrings.py new file mode 100644 index 000000000..cc5fe159d --- /dev/null +++ b/utils/check_docstrings.py @@ -0,0 +1,566 @@ +# Copyright 2026 The HuggingFace Inc. team. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Check that documented arguments match the real signature. + +Adapted from the core of `transformers/utils/check_docstrings.py`. The parts of that file bound to +transformers internals — the `@auto_docstring` decorator system, modular-file propagation, `ModelArgs`, +GitPython — are deliberately not ported. + +What this enforces, for every public object in `MODULES_TO_CHECK`: + +- every parameter in the signature has an `Args:` entry, in signature order; +- no `Args:` entry names a parameter that does not exist; +- the `*optional*, defaults to `X`` clause matches the real default. + +That last one is why the clause is not decorative. See docs/source/writing_docstrings.mdx. + +Check, as CI does: + +```bash +python utils/check_docstrings.py +``` + +Rewrite the `Args:` blocks to match the signatures, inserting `` placeholders for +parameters that are missing entirely: + +```bash +python utils/check_docstrings.py --fix_and_overwrite +``` + +`MODULES_TO_CHECK` is the ratchet: add a module once its docstrings are converted. +""" + +import argparse +import ast +import enum +import importlib +import inspect +import operator as op +import pkgutil +import re +import sys +from pathlib import Path +from typing import Any + +PATH_TO_REPO = Path(__file__).resolve().parent.parent +PATH_TO_LEROBOT = PATH_TO_REPO / "src" / "lerobot" + +# Modules whose public objects are checked. Add a module here once its docstrings follow the standard. +MODULES_TO_CHECK = [ + "lerobot.robots", +] + +# Objects that do not yet follow the standard, so the check can be green from day one. Removing an entry +# and running `--fix_and_overwrite` is how a module gets converted. +# +# Every entry below has a bare `Attributes:` section, which this checker reads as an argument section (the +# same aliasing doc-builder does) and therefore compares against the signature. They are converted in the +# docstring PR that follows this one, which empties this set. +OBJECTS_TO_IGNORE: set[str] = { + "ChannelFactoryInitialize", + "EarthRoverMiniPlus", + "EarthRoverMiniPlusConfig", + "EEBoundsAndSafety", + "EEReferenceAndDelta", + "ForwardKinematicsJointsToEEAction", + "ForwardKinematicsJointsToEEObservation", + "GripperVelocityToJoint", + "InverseKinematicsEEToJoints", + "Robot", +} + +OPTIONAL_KEYWORD = "*optional*" + +_re_args = re.compile(r"^\s*(Args?|Arguments?|Attributes?|Params?|Parameters?):\s*$") +_re_parse_arg = re.compile(r"^(\s*)(\S+)\s+\((.+)\)(?:\:|$)") +_re_parse_description = re.compile(r"\*optional\*, defaults to (.*)$") + +MATH_OPERATORS = { + ast.Add: op.add, + ast.Sub: op.sub, + ast.Mult: op.mul, + ast.Div: op.truediv, + ast.Pow: op.pow, + ast.BitXor: op.xor, + ast.USub: op.neg, +} + + +def find_indent(line: str) -> int: + """Return the number of spaces a line is indented by. + + Args: + line (`str`): + The line to measure. + + Returns: + `int`: The indentation width. + """ + search = re.search(r"^(\s*)(?:\S|$)", line) + return 0 if search is None else len(search.groups()[0]) + + +def is_dataclass_factory_default(default: Any) -> bool: + """Whether a signature default came from a dataclass `field(default_factory=...)`. + + `inspect.signature` renders those as a `` sentinel, which must not be written into a + docstring as a literal default. + + Args: + default (`Any`): + The default value taken from the signature. + + Returns: + `bool`: `True` for the factory sentinel. + """ + return repr(default) == "" + + +def stringify_default(default: Any) -> str: + """Render a default value the way a docstring should show it. + + Args: + default (`Any`): + The default value to process. + + Returns: + `str`: Numbers are left bare, everything else is wrapped in backticks. + """ + if isinstance(default, bool): + # Must precede the int check: a bool passes isinstance(x, int). + return f"`{default}`" + elif isinstance(default, enum.Enum): + # Must also precede the int check: an IntEnum passes isinstance(x, int). + return f"`{str(default)}`" + elif isinstance(default, int): + return str(default) + elif isinstance(default, float): + result = str(default) + return str(round(default, 2)) if len(result) > 6 else result + elif isinstance(default, str): + return str(default) if default.isnumeric() else f'`"{default}"`' + elif isinstance(default, type): + return f"`{default.__name__}`" + else: + return f"`{default}`" + + +def eval_node(node): + """Evaluate one node of a arithmetic-only AST. + + Args: + node (`ast.AST`): + The node to evaluate. + + Returns: + `float | int | complex`: The node's value. + + Raises: + TypeError: If the node is not a number or a supported arithmetic operation. + """ + if isinstance(node, ast.Constant) and type(node.value) in (int, float, complex): + return node.value + elif isinstance(node, ast.BinOp): + return MATH_OPERATORS[type(node.op)](eval_node(node.left), eval_node(node.right)) + elif isinstance(node, ast.UnaryOp): + return MATH_OPERATORS[type(node.op)](eval_node(node.operand)) + else: + raise TypeError(node) + + +def eval_math_expression(expression: str) -> float | int | None: + """Safely evaluate an arithmetic expression found in a docstring. + + Docstrings often document a default as an expression (`1 / 255` is the classic), which should be left + alone rather than replaced by its computed value. + + Args: + expression (`str`): + The expression to evaluate. + + Returns: + `float | int | None`: The value, or `None` if it is not a plain arithmetic expression. + """ + try: + return eval_node(ast.parse(expression, mode="eval").body) + except (TypeError, SyntaxError, KeyError, ZeroDivisionError): + return None + + +def replace_default_in_arg_description(description: str, default: Any) -> str: + """Rewrite the `*optional*, defaults to X` clause of one argument description. + + Args: + description (`str`): + The argument description from the docstring, without the name. + default (`Any`): + The real default from the signature, or `inspect._empty` if the argument is required. + + Returns: + `str`: The description with its optional/default clause matching the signature. + """ + # Plenty of docstrings use `optional` or **optional** instead of *optional*. + description = description.replace("`optional`", OPTIONAL_KEYWORD) + description = description.replace("**optional**", OPTIONAL_KEYWORD) + + if default is inspect._empty: + # Required: the description must not claim otherwise. + idx = description.find(OPTIONAL_KEYWORD) + if idx != -1: + description = description[:idx].rstrip().removesuffix(",").rstrip() + elif default is None or is_dataclass_factory_default(default): + # A `None` default is not spelled out, and a `default_factory` has no literal value to show. + idx = description.find(OPTIONAL_KEYWORD) + if idx == -1: + description = f"{description}, {OPTIONAL_KEYWORD}" + elif re.search(r"defaults to `?None`?", description) is not None: + description = description[: idx + len(OPTIONAL_KEYWORD)] + else: + str_default = None + documented_match = re.search("defaults to `?(.*?)(?:`|$)", description) + if isinstance(default, (int, float)) and documented_match is not None: + documented = documented_match.groups()[0] + if default == eval_math_expression(documented): + try: + # Directly convertible means it was a plain literal. + str_default = str(type(default)(documented)) + except (TypeError, ValueError): + # Otherwise it was an expression; keep it as written. + str_default = f"`{documented}`" + + if str_default is None: + str_default = stringify_default(default) + + if OPTIONAL_KEYWORD not in description: + description = f"{description}, {OPTIONAL_KEYWORD}, defaults to {str_default}" + elif _re_parse_description.search(description) is None: + idx = description.find(OPTIONAL_KEYWORD) + description = f"{description[: idx + len(OPTIONAL_KEYWORD)]}, defaults to {str_default}" + else: + description = _re_parse_description.sub(f"*optional*, defaults to {str_default}", description) + + return description + + +def get_default_description(arg: inspect.Parameter) -> str: + """Build the parenthesised type-and-default part for an undocumented parameter. + + Args: + arg (`inspect.Parameter`): + The parameter to describe. + + Returns: + `str`: Something like ``` `int`, *optional*, defaults to 3 ```. + """ + if arg.annotation is inspect._empty: + arg_type = "" + elif hasattr(arg.annotation, "__name__"): + arg_type = arg.annotation.__name__ + else: + arg_type = str(arg.annotation) + + if arg.default is inspect._empty: + return f"`{arg_type}`" + elif arg.default is None or is_dataclass_factory_default(arg.default): + return f"`{arg_type}`, {OPTIONAL_KEYWORD}" + else: + return f"`{arg_type}`, {OPTIONAL_KEYWORD}, defaults to {stringify_default(arg.default)}" + + +def find_source_file(obj: Any) -> Path: + """Locate the file an object is defined in. + + Args: + obj (`Any`): + The object to locate. + + Returns: + `Path`: The source file. + """ + obj_file = PATH_TO_LEROBOT + for part in obj.__module__.split(".")[1:]: + obj_file = obj_file / part + return obj_file.with_suffix(".py") + + +def match_docstring_with_signature(obj: Any) -> tuple[str, str] | None: + """Compare an object's documented arguments against its signature. + + Dataclasses need no special handling: `inspect.signature` resolves the generated `__init__`, inherited + fields included, which is exactly the set a reader sees on the rendered page. + + Args: + obj (`Any`): + The class or function to check. + + Returns: + `tuple[str, str] | None`: The current `Args:` block and the one matching the signature, or `None` + when there is nothing to compare — no docstring, no documented arguments, or an unsupported + signature. + """ + if not getattr(obj, "__doc__", None): + return None + + try: + source, _ = inspect.getsourcelines(obj) + except (OSError, TypeError): + source = [] + + idx = 0 + while idx < len(source) and '"""' not in source[idx]: + idx += 1 + + ignore_order = False + if idx < len(source) and idx > 0: + line_before_docstring = source[idx - 1] + if re.search(r"^\s*#\s*no-format\s*$", line_before_docstring): + return None + elif re.search(r"^\s*#\s*ignore-order\s*$", line_before_docstring): + ignore_order = True + + try: + signature = inspect.signature(obj).parameters + except (ValueError, TypeError): + return None + + obj_doc_lines = obj.__doc__.split("\n") + idx = 0 + while idx < len(obj_doc_lines) and _re_args.search(obj_doc_lines[idx]) is None: + idx += 1 + if idx == len(obj_doc_lines): + # No arguments documented; coverage is interrogate's job, not this check's. + return None + + if "kwargs" in signature and signature["kwargs"].annotation != inspect._empty: + # Typed **kwargs are not introspectable in a useful way here. + return None + + indent = find_indent(obj_doc_lines[idx]) + arguments: dict[str, Any] = {} + current_arg = None + idx += 1 + start_idx = idx + # Consume until a non-empty line returns to the section's own indent, or the docstring ends. + while idx < len(obj_doc_lines) and ( + len(obj_doc_lines[idx].strip()) == 0 or find_indent(obj_doc_lines[idx]) > indent + ): + if find_indent(obj_doc_lines[idx]) == indent + 4: + re_search_arg = _re_parse_arg.search(obj_doc_lines[idx]) + if re_search_arg is not None: + _, name, description = re_search_arg.groups() + current_arg = name + if name in signature: + default = signature[name].default + if signature[name].kind is inspect._ParameterKind.VAR_KEYWORD: + default = None + new_description = replace_default_in_arg_description(description, default) + else: + new_description = description + arguments[current_arg] = [ + _re_parse_arg.sub(rf"\1\2 ({new_description}):", obj_doc_lines[idx]) + ] + elif current_arg is not None: + arguments[current_arg].append(obj_doc_lines[idx]) + idx += 1 + + # Walk back over the trailing blank lines we consumed. + idx -= 1 + if current_arg: + while len(obj_doc_lines[idx].strip()) == 0: + arguments[current_arg] = arguments[current_arg][:-1] + idx -= 1 + idx += 1 + + old_doc_arg = "\n".join(obj_doc_lines[start_idx:idx]) + + old_arguments = list(arguments.keys()) + arguments = {name: "\n".join(doc) for name, doc in arguments.items()} + for name in set(signature.keys()) - set(arguments.keys()): + arg = signature[name] + # Private parameters and *args/**kwargs are only documented if the author chose to. + if name.startswith("_") or arg.kind in [ + inspect._ParameterKind.VAR_KEYWORD, + inspect._ParameterKind.VAR_POSITIONAL, + ]: + arguments[name] = "" + else: + arguments[name] = ( + " " * (indent + 4) + f"{name} ({get_default_description(arg)}): " + ) + + if ignore_order: + new_param_docs = [arguments[name] for name in old_arguments if name in signature] + missing = set(signature.keys()) - set(old_arguments) + new_param_docs.extend([arguments[name] for name in missing if len(arguments[name]) > 0]) + else: + new_param_docs = [arguments[name] for name in signature if len(arguments[name]) > 0] + + return old_doc_arg, "\n".join(new_param_docs) + + +def fix_docstring(obj: Any, old_doc_args: str, new_doc_args: str) -> None: + """Rewrite an object's `Args:` block in its source file. + + Args: + obj (`Any`): + The object whose docstring is being fixed. + old_doc_args (`str`): + The current `Args:` block, as returned by [`match_docstring_with_signature`]. + new_doc_args (`str`): + The replacement block, as returned by [`match_docstring_with_signature`]. + + Raises: + ValueError: If the block found in the source does not match the one parsed from `__doc__`, which + means the boundaries were identified wrongly and rewriting would corrupt the file. + """ + source, line_number = inspect.getsourcelines(obj) + + idx = 0 + while idx < len(source) and _re_args.search(source[idx]) is None: + idx += 1 + if idx == len(source): + # Inherited docstring: do not rewrite it on the child. + return + + indent = find_indent(source[idx]) + idx += 1 + start_idx = idx + while idx < len(source) and (len(source[idx].strip()) == 0 or find_indent(source[idx]) > indent): + idx += 1 + idx -= 1 + while len(source[idx].strip()) == 0: + idx -= 1 + idx += 1 + + # `old_doc_args` comes from `__doc__`, whose indentation differs from the raw source lines. + source_args_as_str = "".join(source[start_idx:idx]) + if inspect.cleandoc(source_args_as_str) != inspect.cleandoc(old_doc_args): + raise ValueError( + f"Cannot fix the docstring of {obj.__name__} in {find_source_file(obj)}: the argument section " + f"in the source does not match the one parsed from __doc__, so the block boundaries are " + f"wrong and rewriting it would corrupt the file.\n\n" + f"Parsed:\n{old_doc_args!r}\n\nFound in source:\n{source_args_as_str.rstrip()!r}\n" + ) + + obj_file = find_source_file(obj) + lines = obj_file.read_text(encoding="utf-8").split("\n") + # `new_doc_args` is built from `__doc__`, and Python keeps every line after the first at its exact + # source indentation, so the block is already correctly indented for the file. transformers re-indents + # here because its docstrings are often assembled by decorators and no longer match the source. + lines = lines[: line_number + start_idx - 1] + [new_doc_args] + lines[line_number + idx - 1 :] + + print(f"Fixing the docstring of {obj.__name__} in {obj_file}.") + obj_file.write_text("\n".join(lines), encoding="utf-8") + + +def iter_objects_to_check(module_name: str): + """Yield the public classes and functions defined in a package. + + Args: + module_name (`str`): + An importable package name, e.g. `"lerobot.robots"`. + + Yields: + `Any`: Each public class or function whose `__module__` is inside the package, deduplicated so + that aliases (`SO101Follower = SOFollower`) are visited once. + """ + package = importlib.import_module(module_name) + module_names = [module_name] + if hasattr(package, "__path__"): + module_names += [ + name for _, name, _ in pkgutil.walk_packages(package.__path__, prefix=f"{module_name}.") + ] + + seen = set() + for name in module_names: + try: + module = importlib.import_module(name) + except Exception as error: # An optional extra is missing; not this check's problem. + print(f"Skipping {name}: {type(error).__name__}: {error}", file=sys.stderr) + continue + for attr_name, obj in vars(module).items(): + if attr_name.startswith("_") or not (inspect.isclass(obj) or inspect.isfunction(obj)): + continue + if not getattr(obj, "__module__", "").startswith(module_name): + continue + key = f"{obj.__module__}.{obj.__qualname__}" + if key in seen or obj.__qualname__ in OBJECTS_TO_IGNORE or key in OBJECTS_TO_IGNORE: + continue + seen.add(key) + yield obj + + +def check_docstrings(overwrite: bool = False) -> list[str]: + """Check every object in `MODULES_TO_CHECK`. + + Args: + overwrite (`bool`, *optional*, defaults to `False`): + Whether to rewrite mismatched `Args:` blocks in place. + + Returns: + `list[str]`: The names of objects whose documented arguments do not match their signature. Empty + when everything is consistent. + """ + failures = [] + hard_failures = [] + for module_name in MODULES_TO_CHECK: + for obj in iter_objects_to_check(module_name): + try: + result = match_docstring_with_signature(obj) + except Exception as error: + hard_failures.append(f"{obj.__qualname__}: {type(error).__name__}: {error}") + continue + if result is None: + continue + old_doc, new_doc = result + if old_doc == new_doc: + continue + if overwrite: + fix_docstring(obj, old_doc, new_doc) + else: + failures.append(f"{obj.__module__}.{obj.__qualname__}") + + if hard_failures: + print("The following objects could not be processed:", file=sys.stderr) + for failure in hard_failures: + print(f"- {failure}", file=sys.stderr) + return failures + + +def main() -> int: + """Run the check. + + Returns: + `int`: `0` when every documented argument matches its signature, `1` otherwise. + """ + parser = argparse.ArgumentParser() + parser.add_argument("--fix_and_overwrite", action="store_true", help="Whether to fix inconsistencies.") + args = parser.parse_args() + + failures = check_docstrings(overwrite=args.fix_and_overwrite) + if failures: + print( + "The docstrings of the following objects do not match their signature. Run " + "`make fix-docstrings` to rewrite them, then fill in any `` placeholders:", + file=sys.stderr, + ) + for failure in failures: + print(f"- {failure}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/utils/check_doctest_list.py b/utils/check_doctest_list.py new file mode 100644 index 000000000..7a8d38a37 --- /dev/null +++ b/utils/check_doctest_list.py @@ -0,0 +1,109 @@ +# Copyright 2026 The HuggingFace Inc. team. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Keep the doctest list honest: every path exists, and the file stays sorted. + +Adapted from `transformers/utils/check_doctest_list.py`. It is agnostic to whether the list is an allowlist +(what we have now) or a denylist (where transformers ended up), so it survives that inversion unchanged. + +Check, as CI does: + +```bash +python utils/check_doctest_list.py +``` + +Sort in place: + +```bash +python utils/check_doctest_list.py --fix_and_overwrite +``` +""" + +import argparse +import sys +from pathlib import Path + +REPO_PATH = Path(__file__).resolve().parent.parent +DOCTEST_FILE_PATHS = ["documentation_tests.txt"] + + +def split_header(lines: list[str]) -> tuple[list[str], list[str]]: + """Split a list file into its leading comment header and its path entries. + + Args: + lines (`list[str]`): + The file's lines, without trailing newlines. + + Returns: + `tuple[list[str], list[str]]`: The leading comment/blank lines, and the remaining lines. + """ + for i, line in enumerate(lines): + if line.strip() and not line.lstrip().startswith("#"): + return lines[:i], lines[i:] + return lines, [] + + +def clean_doctest_list(doctest_file: Path, overwrite: bool = False) -> None: + """Check, and optionally fix, one doctest list file. + + Args: + doctest_file (`Path`): + The list file to check or clean. + overwrite (`bool`, *optional*, defaults to `False`): + Whether to fix problems in place. When `False`, raises instead. + + Raises: + ValueError: If the file lists a path that does not exist, or is not alphabetically sorted and + `overwrite` is `False`. + """ + lines = doctest_file.read_text(encoding="utf-8").splitlines() + header, entries = split_header(lines) + paths = [line.strip().split(" ")[0] for line in entries if line.strip()] + + non_existent = [p for p in paths if not (REPO_PATH / p).exists()] + if non_existent: + listed = "\n".join(f"- {p}" for p in non_existent) + raise ValueError(f"`{doctest_file.name}` contains non-existent paths:\n{listed}") + + if paths != sorted(paths): + if not overwrite: + raise ValueError( + f"Files in `{doctest_file.name}` are not in alphabetical order, run " + "`make fix-docstrings` to fix this automatically." + ) + doctest_file.write_text("\n".join(header + sorted(paths)) + "\n", encoding="utf-8") + + +def main() -> int: + """Run the check over every doctest list file. + + Returns: + `int`: A process exit code — `0` when every file is clean, `1` otherwise. + """ + parser = argparse.ArgumentParser() + parser.add_argument("--fix_and_overwrite", action="store_true", help="Whether to fix inconsistencies.") + args = parser.parse_args() + + failed = False + for name in DOCTEST_FILE_PATHS: + try: + clean_doctest_list(REPO_PATH / "utils" / name, args.fix_and_overwrite) + except ValueError as error: + print(error, file=sys.stderr) + failed = True + return 1 if failed else 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/utils/documentation_tests.txt b/utils/documentation_tests.txt new file mode 100644 index 000000000..89e3d5c56 --- /dev/null +++ b/utils/documentation_tests.txt @@ -0,0 +1,12 @@ +# Files whose docstring examples are executed by `make doctest`. +# +# This is an ALLOWLIST: only the paths below are collected. transformers started the same way and has since +# inverted to a denylist (`utils/not_doctested.txt`), which is the better end state — it makes a new file +# tested by default. LeRobot cannot start there: at the time of writing, public docstring coverage is under +# 50% and only a handful of files carry any example at all, so a denylist would need hundreds of entries on +# day one and would say nothing about what is actually verified. +# +# Invert once coverage is high enough that the exclusions are the short list. +# `utils/check_doctest_list.py` does not care which way round it is. +# +# Keep alphabetically sorted: `make check-doctest-list` enforces it, `make fix-docstrings` sorts it.