mirror of
https://github.com/huggingface/lerobot.git
synced 2026-08-08 17:39:44 +00:00
CarolinePascal
1834f819a6
Second module of Wave 4 (training & eval): brings src/lerobot/rl/ to 100% public docstring coverage, following the standard in docs/source/writing_docstrings.mdx. - Documents the remaining gaps across the SAC algorithm (SACAlgorithm.__init__, CriticHead, CriticEnsemble.__init__/forward, get_optimizers), the RLAlgorithm/RLAlgorithmConfig base contract (optimization_step setter, from_pretrained), SACAlgorithmConfig (converts inline `#` field comments to a proper Args: block), ReplayBuffer/BatchTransition, OnlineOfflineMixer, TrainRLServerPipelineConfig (documents every inherited TrainPipelineConfig field, since the base class itself is undocumented and out of scope), and the actor/learner gRPC entry points (actor_cli, train_cli, transitions_stream/interactions_stream, LearnerService's 5 servicer methods) and their smaller helpers (queue.get_last_item_from_queue, crop_dataset_roi.mouse_callback, eval_policy). - Also documents 3 dunder methods (RLTrainer's _PreprocessedIterator.__iter__/ __next__, ReplayBuffer.__len__) that a naive "skip all underscore-prefixed names" gap scan misses but interrogate's ignore-magic=false requires. - Removing the D-ignore surfaced ~30 pre-existing docstrings with D205/D415/ D417 issues (missing blank line after summary, missing punctuation, stale Args entries that didn't match the real signature) across actor.py, crop_dataset_roi.py, gym_manipulator.py, learner.py, and learner_service.py — all fixed as part of this PR. - Removes "src/lerobot/rl/**" = ["D"] from pyproject.toml's ruff ignore list; the whole module is now checked (no per-family split to narrow, unlike policies). - Adds lerobot.rl to check_docstrings.py's MODULES_TO_CHECK ratchet. - Creates docs/source/api/rl.mdx from scratch (algorithm base contract, SAC, replay buffer, data mixers, trainer, actor/learner CLIs and gRPC service) and wires it into _toctree.yml, cross-linked from the existing hilserl.mdx/ hilserl_sim.mdx guides. Verified via a full doc-builder build — no dead cross-references, no leftover placeholder text. - Ratchets interrogate's fail-under from 55 to 55.5 (measured 55.9% with this PR). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
88 lines
2.2 KiB
Plaintext
88 lines
2.2 KiB
Plaintext
# Reinforcement Learning
|
|
|
|
`lerobot.rl` is the distributed actor/learner reinforcement-learning stack behind
|
|
[Train a Robot with RL](../hilserl) (HIL-SERL) and [Train RL in Simulation](../hilserl_sim). Algorithms,
|
|
the replay buffer, data sources, and the trainer are gRPC-free and usable standalone; the actor/learner
|
|
entry points (`actor`, `learner`, `learner_service`) additionally require `pip install 'lerobot[hilserl]'`.
|
|
|
|
## TrainRLServerPipelineConfig
|
|
|
|
Top-level configuration for both the `lerobot-actor` and `lerobot-learner` CLIs.
|
|
|
|
[[autodoc]] lerobot.rl.train_rl.TrainRLServerPipelineConfig
|
|
|
|
## RLAlgorithm
|
|
|
|
Abstract base every RL algorithm subclasses.
|
|
|
|
[[autodoc]] lerobot.rl.algorithms.base.RLAlgorithm
|
|
|
|
## RLAlgorithmConfig
|
|
|
|
[[autodoc]] lerobot.rl.algorithms.configs.RLAlgorithmConfig
|
|
|
|
## TrainingStats
|
|
|
|
[[autodoc]] lerobot.rl.algorithms.configs.TrainingStats
|
|
|
|
## make_algorithm
|
|
|
|
[[autodoc]] lerobot.rl.algorithms.factory.make_algorithm
|
|
|
|
## make_algorithm_config
|
|
|
|
[[autodoc]] lerobot.rl.algorithms.factory.make_algorithm_config
|
|
|
|
## get_algorithm_class
|
|
|
|
[[autodoc]] lerobot.rl.algorithms.factory.get_algorithm_class
|
|
|
|
## SAC
|
|
|
|
[[autodoc]] lerobot.rl.algorithms.sac.sac_algorithm.SACAlgorithm
|
|
- all
|
|
|
|
[[autodoc]] lerobot.rl.algorithms.sac.configuration_sac.SACAlgorithmConfig
|
|
|
|
## ReplayBuffer
|
|
|
|
In-memory replay buffer of transitions, sampled in batches for off-policy training.
|
|
|
|
[[autodoc]] lerobot.rl.buffer.ReplayBuffer
|
|
- all
|
|
|
|
[[autodoc]] lerobot.rl.buffer.BatchTransition
|
|
|
|
## DataMixer
|
|
|
|
Abstract interface for combining online and offline data sources into training batches.
|
|
|
|
[[autodoc]] lerobot.rl.data_sources.DataMixer
|
|
- all
|
|
|
|
[[autodoc]] lerobot.rl.data_sources.OnlineOfflineMixer
|
|
- all
|
|
|
|
## RLTrainer
|
|
|
|
Unified training-step orchestrator: holds the algorithm, a `DataMixer`, and an optional preprocessor.
|
|
|
|
[[autodoc]] lerobot.rl.trainer.RLTrainer
|
|
- all
|
|
|
|
## Actor / learner CLIs
|
|
|
|
The distributed actor and learner processes communicate over gRPC; see [Train a Robot with
|
|
RL](../hilserl) for the full workflow.
|
|
|
|
[[autodoc]] lerobot.rl.actor.actor_cli
|
|
|
|
[[autodoc]] lerobot.rl.learner.train_cli
|
|
|
|
[[autodoc]] lerobot.rl.learner_service.LearnerService
|
|
- all
|
|
|
|
## eval_policy
|
|
|
|
[[autodoc]] lerobot.rl.eval_policy.eval_policy
|