docs: add API documentation infrastructure

LeRobot's documentation build passes `--not_python_module`, which tells
doc-builder there is no importable Python package and disables `[[autodoc]]`
entirely. The result is that all 90+ pages are hand-written guides and there is
no generated API reference at all.

This is the machinery to change that. It deliberately contains no docstring
changes of its own — every docstring edit lives in the follow-up PR, so this
one can be reviewed as tooling and configuration alone.

**The standard.** `docs/source/writing_docstrings.mdx` is the contract: Google
section headers with Hugging Face type formatting, the machine-checked argument
line, `**Attributes**:`, doc-builder cross-references, fenced doctest examples.
It also records three behaviours that are not discoverable from the source and
were verified against a local build: `[[autodoc]]` silently skips members with
no docstring; doc-builder does not inherit docstrings from base classes, so a
registered config shim whose body is `pass` renders every field with no
description; and module-level aliases resolve to the canonical class.

**Autodoc turned on**, with two changes that are not obvious:

- `--version main` on the main-docs job. Without `--not_python_module`,
  doc-builder resolves the version from `lerobot.__version__` and only maps it
  to the default branch when it contains "dev". transformers relies on that;
  our main carries 0.6.2. Verified by building both ways — dropping the flag
  alone would publish the main docs to /lerobot/v0.6.2/ instead of
  /lerobot/main/ and disable notebook building.
- `pre_command` on both jobs. doc-builder ships a mock-deps registry entry for
  lerobot, so the reusable workflow takes its light-install path, which cannot
  import the package. The heavy dependencies cannot be mocked either: draccus
  runs `register_subclass` at import time and `processor/converters.py` calls
  `functools.singledispatch.register(torch.Tensor)`, which needs a real class.
  `[dataset]` is the only extra required.

Workflow triggers gain `src/**`, since the reference is now generated from
docstrings. `docs/source/api/` is excluded from the prettier hook, which reads
`[[autodoc]]` member lists as lazy paragraph continuations and joins a ten-entry
list onto one line.

Nine API reference pages, scaffolded with each module's base class.

**Doctests.** `LeRobotDocTestParser` is mandatory rather than optional here:
ruff's `docstring-code-format = true` drops the blank line before a closing
fence, after which stdlib's `_EXAMPLE_RE` reads the fence as expected output and
every example with output fails. It is written against the installed pytest
rather than copied from transformers, whose version predates pytest 9's
`import_path` signature and its own fix for the `@property` line-number bug.
`preprocess_string` also diverges: the upstream fenced-block split puts a
single-line example's code in a chunk with no `>>>` in it, so neither the CUDA
skip nor the `+IGNORE_RESULT` injection fires for it.

**Checkers.** `utils/check_docstrings.py` is the ~300-line core of the
2203-line transformers original; the `@auto_docstring` system, modular
propagation, GitPython and `checkers.py` are not ported.
`utils/check_config_docstrings.py` checks that every registered robot config
documents its port and calibration semantics.

**Gates**, all set to values that pass today: ruff `D` with per-file-ignores
per unconverted module, `interrogate` at `fail-under = 52` against a measured
52.1%, and Makefile targets wired into the quality workflow. The doctest
allowlist ships empty and the `doctest` target handles that, because the files
carrying runnable examples arrive with the docstring PR.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Pepijn
2026-08-06 20:57:26 +02:00
parent ef88d4e52b
commit 2e8345a5cc
24 changed files with 2022 additions and 24 deletions
+111
View File
@@ -0,0 +1,111 @@
# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import doctest
from lerobot.utils.doctest_utils import LeRobotDocTestParser, preprocess_string
# An example with expected output, formatted the way ruff's `docstring-code-format` leaves it: no blank
# line between the last output line and the closing fence. This is the exact shape that breaks stdlib.
FORMATTED_EXAMPLE = """Summary.
Example:
```python
>>> 1 + 1
2
```
"""
def test_stdlib_parser_swallows_the_closing_fence():
"""Guards the premise of the port: without the patch, the fence lands in the expected output.
Uses the base class rather than `doctest.DocTestParser`, which the root `conftest.py` has already
replaced with ours by the time this runs.
"""
stdlib_parser = LeRobotDocTestParser.__bases__[0]()
(example,) = (e for e in stdlib_parser.parse(FORMATTED_EXAMPLE) if isinstance(e, doctest.Example))
assert "```" in example.want
def test_parser_stops_at_the_closing_fence():
"""The whole reason `LeRobotDocTestParser` exists: `want` must be the output and nothing else."""
(example,) = (
e for e in LeRobotDocTestParser().parse(FORMATTED_EXAMPLE) if isinstance(e, doctest.Example)
)
assert example.source == "1 + 1\n"
assert example.want == "2\n"
def test_example_with_output_passes_end_to_end():
"""A formatted example with output should actually run green."""
runner = doctest.DocTestRunner()
test = LeRobotDocTestParser().get_doctest(FORMATTED_EXAMPLE, {}, "formatted", None, 0)
results = runner.run(test, out=lambda _: None)
assert results.failed == 0
assert results.attempted == 1
def test_noisy_calls_get_ignore_result():
string = """
```python
>>> ds = load_dataset("lerobot/pusht")
```
"""
assert "# doctest: +IGNORE_RESULT" in preprocess_string(string, False, False)
def test_ignore_result_is_not_added_twice():
string = """
```python
>>> ds = load_dataset("lerobot/pusht") # doctest: +IGNORE_RESULT
```
"""
assert preprocess_string(string, False, False).count("# doctest: +IGNORE_RESULT") == 1
def test_cuda_examples_are_dropped_when_requested():
string = """
```python
>>> model.to("cuda")
```
"""
assert preprocess_string(string, True, False) == ""
assert preprocess_string(string, False, False) != ""
def test_hardware_examples_are_dropped_when_requested():
"""Serial ports, connect calls and Hub downloads all need real resources."""
for source in [
'>>> robot = SO101Follower(SO101FollowerConfig(port="/dev/ttyACM0"))',
">>> robot.connect()",
'>>> policy = ACTPolicy.from_pretrained("lerobot/act")',
]:
string = f"""
```python
{source}
```
"""
assert preprocess_string(string, False, True) == "", source
assert preprocess_string(string, False, False) != "", source
def test_plain_examples_survive_both_skips():
string = """
```python
>>> 1 + 1
2
```
"""
assert preprocess_string(string, True, True) == string