mirror of
https://github.com/huggingface/lerobot.git
synced 2026-08-08 17:39:44 +00:00
Pepijn
2e8345a5cc
LeRobot's documentation build passes `--not_python_module`, which tells doc-builder there is no importable Python package and disables `[[autodoc]]` entirely. The result is that all 90+ pages are hand-written guides and there is no generated API reference at all. This is the machinery to change that. It deliberately contains no docstring changes of its own — every docstring edit lives in the follow-up PR, so this one can be reviewed as tooling and configuration alone. **The standard.** `docs/source/writing_docstrings.mdx` is the contract: Google section headers with Hugging Face type formatting, the machine-checked argument line, `**Attributes**:`, doc-builder cross-references, fenced doctest examples. It also records three behaviours that are not discoverable from the source and were verified against a local build: `[[autodoc]]` silently skips members with no docstring; doc-builder does not inherit docstrings from base classes, so a registered config shim whose body is `pass` renders every field with no description; and module-level aliases resolve to the canonical class. **Autodoc turned on**, with two changes that are not obvious: - `--version main` on the main-docs job. Without `--not_python_module`, doc-builder resolves the version from `lerobot.__version__` and only maps it to the default branch when it contains "dev". transformers relies on that; our main carries 0.6.2. Verified by building both ways — dropping the flag alone would publish the main docs to /lerobot/v0.6.2/ instead of /lerobot/main/ and disable notebook building. - `pre_command` on both jobs. doc-builder ships a mock-deps registry entry for lerobot, so the reusable workflow takes its light-install path, which cannot import the package. The heavy dependencies cannot be mocked either: draccus runs `register_subclass` at import time and `processor/converters.py` calls `functools.singledispatch.register(torch.Tensor)`, which needs a real class. `[dataset]` is the only extra required. Workflow triggers gain `src/**`, since the reference is now generated from docstrings. `docs/source/api/` is excluded from the prettier hook, which reads `[[autodoc]]` member lists as lazy paragraph continuations and joins a ten-entry list onto one line. Nine API reference pages, scaffolded with each module's base class. **Doctests.** `LeRobotDocTestParser` is mandatory rather than optional here: ruff's `docstring-code-format = true` drops the blank line before a closing fence, after which stdlib's `_EXAMPLE_RE` reads the fence as expected output and every example with output fails. It is written against the installed pytest rather than copied from transformers, whose version predates pytest 9's `import_path` signature and its own fix for the `@property` line-number bug. `preprocess_string` also diverges: the upstream fenced-block split puts a single-line example's code in a chunk with no `>>>` in it, so neither the CUDA skip nor the `+IGNORE_RESULT` injection fires for it. **Checkers.** `utils/check_docstrings.py` is the ~300-line core of the 2203-line transformers original; the `@auto_docstring` system, modular propagation, GitPython and `checkers.py` are not ported. `utils/check_config_docstrings.py` checks that every registered robot config documents its port and calibration semantics. **Gates**, all set to values that pass today: ruff `D` with per-file-ignores per unconverted module, `interrogate` at `fail-under = 52` against a measured 52.1%, and Makefile targets wired into the quality workflow. The doctest allowlist ships empty and the `doctest` target handles that, because the files carrying runnable examples arrive with the docstring PR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
112 lines
3.6 KiB
Python
112 lines
3.6 KiB
Python
# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
|
|
import doctest
|
|
|
|
from lerobot.utils.doctest_utils import LeRobotDocTestParser, preprocess_string
|
|
|
|
# An example with expected output, formatted the way ruff's `docstring-code-format` leaves it: no blank
|
|
# line between the last output line and the closing fence. This is the exact shape that breaks stdlib.
|
|
FORMATTED_EXAMPLE = """Summary.
|
|
|
|
Example:
|
|
```python
|
|
>>> 1 + 1
|
|
2
|
|
```
|
|
"""
|
|
|
|
|
|
def test_stdlib_parser_swallows_the_closing_fence():
|
|
"""Guards the premise of the port: without the patch, the fence lands in the expected output.
|
|
|
|
Uses the base class rather than `doctest.DocTestParser`, which the root `conftest.py` has already
|
|
replaced with ours by the time this runs.
|
|
"""
|
|
stdlib_parser = LeRobotDocTestParser.__bases__[0]()
|
|
(example,) = (e for e in stdlib_parser.parse(FORMATTED_EXAMPLE) if isinstance(e, doctest.Example))
|
|
assert "```" in example.want
|
|
|
|
|
|
def test_parser_stops_at_the_closing_fence():
|
|
"""The whole reason `LeRobotDocTestParser` exists: `want` must be the output and nothing else."""
|
|
(example,) = (
|
|
e for e in LeRobotDocTestParser().parse(FORMATTED_EXAMPLE) if isinstance(e, doctest.Example)
|
|
)
|
|
assert example.source == "1 + 1\n"
|
|
assert example.want == "2\n"
|
|
|
|
|
|
def test_example_with_output_passes_end_to_end():
|
|
"""A formatted example with output should actually run green."""
|
|
runner = doctest.DocTestRunner()
|
|
test = LeRobotDocTestParser().get_doctest(FORMATTED_EXAMPLE, {}, "formatted", None, 0)
|
|
results = runner.run(test, out=lambda _: None)
|
|
assert results.failed == 0
|
|
assert results.attempted == 1
|
|
|
|
|
|
def test_noisy_calls_get_ignore_result():
|
|
string = """
|
|
```python
|
|
>>> ds = load_dataset("lerobot/pusht")
|
|
```
|
|
"""
|
|
assert "# doctest: +IGNORE_RESULT" in preprocess_string(string, False, False)
|
|
|
|
|
|
def test_ignore_result_is_not_added_twice():
|
|
string = """
|
|
```python
|
|
>>> ds = load_dataset("lerobot/pusht") # doctest: +IGNORE_RESULT
|
|
```
|
|
"""
|
|
assert preprocess_string(string, False, False).count("# doctest: +IGNORE_RESULT") == 1
|
|
|
|
|
|
def test_cuda_examples_are_dropped_when_requested():
|
|
string = """
|
|
```python
|
|
>>> model.to("cuda")
|
|
```
|
|
"""
|
|
assert preprocess_string(string, True, False) == ""
|
|
assert preprocess_string(string, False, False) != ""
|
|
|
|
|
|
def test_hardware_examples_are_dropped_when_requested():
|
|
"""Serial ports, connect calls and Hub downloads all need real resources."""
|
|
for source in [
|
|
'>>> robot = SO101Follower(SO101FollowerConfig(port="/dev/ttyACM0"))',
|
|
">>> robot.connect()",
|
|
'>>> policy = ACTPolicy.from_pretrained("lerobot/act")',
|
|
]:
|
|
string = f"""
|
|
```python
|
|
{source}
|
|
```
|
|
"""
|
|
assert preprocess_string(string, False, True) == "", source
|
|
assert preprocess_string(string, False, False) != "", source
|
|
|
|
|
|
def test_plain_examples_survive_both_skips():
|
|
string = """
|
|
```python
|
|
>>> 1 + 1
|
|
2
|
|
```
|
|
"""
|
|
assert preprocess_string(string, True, True) == string
|