Wave 4 (training & eval) kickoff: brings src/lerobot/rollout/ to 100% public
docstring coverage, following the standard in docs/source/writing_docstrings.mdx.
Picked as the smallest of the five Wave 4 modules (scripts/envs/rewards/rl/rollout)
to validate the workflow before the larger ones.
- Documents the remaining gaps across configs.py (RolloutConfig, the
RolloutStrategyConfig hierarchy, DAggerKeyboardConfig/DAggerPedalConfig),
inference/ (InferenceEngineConfig hierarchy, RTCInferenceEngine.__init__,
SyncInferenceEngine.__init__), ring_buffer.py, robot_wrapper.py, and every
strategy's __init__ (core/dagger/episodic/highlight/sentry), converting
RolloutConfig's and the strategy configs' inline `#` field comments into
type-annotated Args: blocks and RolloutRingBuffer's numpydoc Parameters
section into Google-style Args:.
- Also documents two dunder methods (RolloutConfig.__get_path_fields__,
RolloutRingBuffer.__len__) that a naive "skip all underscore-prefixed
names" gap scan misses but interrogate's ignore-magic=false requires.
- Removes "src/lerobot/rollout/**" = ["D"] from pyproject.toml's ruff ignore
list — the whole module is now checked, no deferred internals (unlike the
policies module, rollout has no per-family split to narrow the scope of).
- Adds lerobot.rollout to check_docstrings.py's MODULES_TO_CHECK ratchet.
- Creates docs/source/api/rollout.mdx from scratch (strategies, inference
backends, RolloutContext and its sub-contexts, ThreadSafeRobot,
RolloutRingBuffer) and wires it into _toctree.yml under API Reference.
Verified via a full doc-builder build — no dead cross-references, no
leftover placeholder text.
- Ratchets interrogate's fail-under from 55 to 55.5 (measured 55.6% with
this PR).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Every docstring change for the API reference, on top of the infrastructure PR
which contains none. Two halves: a repo-wide pass over what the renderer cannot
handle, and `src/lerobot/robots/` taken to 100% as the worked example.
**Renderer fixes, repo-wide.** Both of these render incorrectly the moment
`[[autodoc]]` is on, and both were verified against a local build:
- 24 Sphinx roles across three files. They are unsupported and render as literal
`:pymeth:` text. Method references become doc-builder cross-references; the
ones pointing at instance attributes become inline code, since attributes get
no autodoc anchor and a cross-reference would be a dead link.
- 43 `Attributes:` sections across 27 files. doc-builder parses a bare
`Attributes:` as a synonym for `Parameters:` — `Robot`'s attributes rendered
inside `<paramsdesc>`, presenting `config_class` and `name` to readers as
constructor arguments when the actual parameter is `config`. Where the
original carried no type, the type comes from the real class annotation rather
than being invented.
The four base classes every other module inherits from — `robot.py`,
`teleoperator.py`, `motors_bus.py`, `camera.py` — are rewritten to the standard,
since subclasses document only their deviations from that text.
Three docstring errors corrected in passing: `Teleoperator.get_action` pointed
at `observation_features`, which `Teleoperator` does not have; `send_feedback`
documented a `Returns:` for a method returning `None`; and `config_class` was
typed `RobotConfig` instead of `type[TeleoperatorConfig]`.
**`robots/`, 109/306 -> 306/306.** The configuration dataclasses were the
substantial part. Their fields were documented only with `#` comments above each
field, which doc-builder cannot see: before this, `SO101FollowerConfig` rendered
all eleven of its fields with not one description. Each config now carries an
`Args:` block on the concrete registered class, covering inherited fields too,
because doc-builder renders only a class's own docstring and several of these
configs are thin multiple-inheritance shims whose body is `pass`.
The inline comments are kept rather than removed, so fields stay annotated in
the source as well as on the rendered page. Note this leaves each field
described twice, and only the `Args:` block is checked against the signature by
`make check-docstrings`, so the two can drift.
Writing them turned up things worth stating plainly on the page rather than
leaving in a comment: which configs have no serial port at all because they talk
over a network or the cloud (Reachy 2, Unitree G1, LeKiwi's client, EarthRover),
which manage their own calibration so `calibration_dir` does nothing, that
OpenArm's default joint limits are deliberately tiny until `side` is set, and
that reBot's `port` means a different thing depending on `can_adapter`.
Two pre-existing docstring bugs that the doctest infrastructure surfaced are
fixed here: `SerialMotorsBus` used `>>>` inside a ```bash block to show CLI
output, which doctest read as Python and failed on with a SyntaxError, and
`MotorsBus.torque_disabled`'s example referenced an undefined name.
`ensure_safe_goal_position` gains a genuinely executing example so the doctest
gate is not vacuous.
**Gates ratcheted**, each of which the infrastructure PR left deliberately
loose:
- `check_docstrings.py`'s ignore list emptied — the ten objects it held all had
bare `Attributes:` sections, now converted.
- `check_config_docstrings.py`'s ignore list emptied — every registered robot
config documents its port and calibration semantics.
- `robots/` removed from the ruff `D` per-file-ignores, as are the two
package-root files, whose one-line docstring issues are fixed here. D100 and
D104 are ignored globally instead: they ask for a banner on every file and
every `__init__.py`, which appears on no rendered page.
- `interrogate` raised 52 -> 55 against a measured 55.3%.
- The four `robots/` files carrying examples added to the doctest allowlist,
which shipped empty.
Verified: all 66 changed files under `src/lerobot/` are provably docstring-only
(AST with docstrings stripped is byte-identical to main), no comment line is
removed anywhere in `robots/`, and 708 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
LeRobot's documentation build passes `--not_python_module`, which tells
doc-builder there is no importable Python package and disables `[[autodoc]]`
entirely. The result is that all 90+ pages are hand-written guides and there is
no generated API reference at all.
This is the machinery to change that. It deliberately contains no docstring
changes of its own — every docstring edit lives in the follow-up PR, so this
one can be reviewed as tooling and configuration alone.
**The standard.** `docs/source/writing_docstrings.mdx` is the contract: Google
section headers with Hugging Face type formatting, the machine-checked argument
line, `**Attributes**:`, doc-builder cross-references, fenced doctest examples.
It also records three behaviours that are not discoverable from the source and
were verified against a local build: `[[autodoc]]` silently skips members with
no docstring; doc-builder does not inherit docstrings from base classes, so a
registered config shim whose body is `pass` renders every field with no
description; and module-level aliases resolve to the canonical class.
**Autodoc turned on**, with two changes that are not obvious:
- `--version main` on the main-docs job. Without `--not_python_module`,
doc-builder resolves the version from `lerobot.__version__` and only maps it
to the default branch when it contains "dev". transformers relies on that;
our main carries 0.6.2. Verified by building both ways — dropping the flag
alone would publish the main docs to /lerobot/v0.6.2/ instead of
/lerobot/main/ and disable notebook building.
- `pre_command` on both jobs. doc-builder ships a mock-deps registry entry for
lerobot, so the reusable workflow takes its light-install path, which cannot
import the package. The heavy dependencies cannot be mocked either: draccus
runs `register_subclass` at import time and `processor/converters.py` calls
`functools.singledispatch.register(torch.Tensor)`, which needs a real class.
`[dataset]` is the only extra required.
Workflow triggers gain `src/**`, since the reference is now generated from
docstrings. `docs/source/api/` is excluded from the prettier hook, which reads
`[[autodoc]]` member lists as lazy paragraph continuations and joins a ten-entry
list onto one line.
Nine API reference pages, scaffolded with each module's base class.
**Doctests.** `LeRobotDocTestParser` is mandatory rather than optional here:
ruff's `docstring-code-format = true` drops the blank line before a closing
fence, after which stdlib's `_EXAMPLE_RE` reads the fence as expected output and
every example with output fails. It is written against the installed pytest
rather than copied from transformers, whose version predates pytest 9's
`import_path` signature and its own fix for the `@property` line-number bug.
`preprocess_string` also diverges: the upstream fenced-block split puts a
single-line example's code in a chunk with no `>>>` in it, so neither the CUDA
skip nor the `+IGNORE_RESULT` injection fires for it.
**Checkers.** `utils/check_docstrings.py` is the ~300-line core of the
2203-line transformers original; the `@auto_docstring` system, modular
propagation, GitPython and `checkers.py` are not ported.
`utils/check_config_docstrings.py` checks that every registered robot config
documents its port and calibration semantics.
**Gates**, all set to values that pass today: ruff `D` with per-file-ignores
per unconverted module, `interrogate` at `fail-under = 52` against a measured
52.1%, and Makefile targets wired into the quality workflow. The doctest
allowlist ships empty and the `doctest` target handles that, because the files
carrying runnable examples arrive with the docstring PR.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(train): parallel training engine with FSDP2, HSDP, and DCP checkpoints
Replace the FSDP1 training path with a config-owned parallel-training
engine:
- Topology and runtime configs (--parallelism.*, --accelerator.*):
dp_replicate x dp_shard degrees select single-process, DDP (unchanged
default), FSDP2, or HSDP; mixed precision, first-class gradient
accumulation, and FSDP/DDP tuning knobs are mirrored as plain
dataclasses that build the accelerate objects at runtime, so every
run is reproducible from its train_config.json alone. Accelerate env
vars are guarded against configuring the engine behind the config
system's back.
- Declarative policy surface: policies declare FSDP2 wrap units
(_fsdp_wrap_modules) and non-forward entry points
(_fsdp_forward_methods); a shared engine resolves them around
accelerator.prepare(). Context-parallel fields are reserved and
validated to 1.
- Checkpoints: selectable --checkpoint_format (safetensors | dcp |
safetensors_dcp); the sharded optimizer channel is always DCP;
two-phase resume (step+RNG before prepare, DCP model/optimizer after)
reshards across GPU-topology changes; lerobot-convert-dcp merges DCP
shards into a distributable model.safetensors offline.
- Publishing: PreTrainedPolicy.push_model_to_hub is replaced by the
free publish_trained_model (model + processors + card + train config,
all-ranks gather with main-rank writes);
PreTrainedPolicy._save_pretrained gathers state dicts internally,
removing the state_dict= threading from save_pretrained.
- lerobot_train is restructured around the engine: optimizer built
before the single prepare() call, deferred weight load on DCP
resumes, collective save_checkpoint with no call-site rank branches,
dp-world-size-based sample accounting.
Breaking changes: FSDP checkpoints from lerobot <= 0.6.x are not
resumable (weights stay loadable via from_pretrained; pin
lerobot==0.6.x to finish old runs); the `accelerate launch
--config_file` yaml flow is superseded by the config flags; training
autocast is owned exclusively by --accelerator.mixed_precision
(policy.dtype only casts parameters).
Also fixes: reward-model hub publishing crash (TypeError on extra
kwargs).
Verified by ~200 new CPU tests (config round-trips, checkpoint
round-trips per format, two-phase resume, publisher contracts,
converter equivalence, accelerate canaries), a 5-test 4-GPU suite
(FSDP2 save/resume bit-exactness, HSDP/DDP loss parity,
changed-topology resume, all-ranks save_pretrained, grad-accum
equivalence), and end-to-end ACT (1/4/8 GPUs) + FastWAM 6B
(FSDP2 + HSDP) training runs.
`send_action` keys `arm_goal_pos` as "<motor>.pos" but a Present_Position
read is keyed by bare motor name, so pairing the two raised KeyError as soon
as `max_relative_target` was set, making the safety cap unusable.
SOFollower strips the suffix before the same lookup.
* fix(sarm): warn when dense/sparse targets silently collapse to all-zero
In dense_only/dual modes, if meta/episodes/*.parquet has no usable
subtask columns (column absent or NaN), _load_episode_annotations
returns None and find_stage_and_tau yields stage 0 / tau 0 for every
frame. Training "succeeds" but the head silently learns to predict 0
everywhere, with no warning. This complements #2880 (which restored
loading of episodes_df): there the DataFrame is loaded but the
*_subtask_names column is missing/NaN.
Add a one-time validation at processor construction that logs a clear
warning (all episodes missing -> predict-all-zero; some missing ->
partial). Purely additive logging, no change to training math.
Closes#3842
* fix(sarm): fail fast on unusable episode annotations
---------
Co-authored-by: 1thanShih <Smartshithan1620.en12@nycu.edu.tw>
* perf(pi0fast): stop FAST decoding at the end-of-action marker
The decode loop always runs all max_decoding_steps (256) tokens, but
detokenize_actions() cuts the output at the first "|", so most of them get
generated and then thrown away. On LIBERO the marker lands around token 25-35.
Stopping there gives the same actions ~4x faster end to end. Checked on
libero_10: 50/50 episodes came out identical to the old path, same success
and same state trajectories.
* feat(policies): early stopping + termination
---------
Co-authored-by: Thomas Landeg <tomlandeg@gmail.com>
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* fix(video): only default to torchcodec when it can actually be loaded
get_safe_default_video_backend() promotes torchcodec to the default video
backend whenever importlib.util.find_spec("torchcodec") succeeds. But
installed is not the same as loadable: torchcodec links against FFmpeg
shared libraries at runtime, which are commonly absent on Windows (and on
minimal Linux images). find_spec still sees the package, the backend
defaults to torchcodec, and every dataset video decode then crashes with a
DLL-load RuntimeError unless the user knows to pass video_backend="pyav"
explicitly.
Probe the actual import (torchcodec.decoders) instead, and fall back to
pyav with an actionable warning when it fails. On healthy installs the
probed module is exactly what decoding imports anyway, so nothing extra is
loaded.
Tested on Windows 11 (Python 3.12, torchcodec installed via the dataset
extra, no FFmpeg shared libraries): LeRobotDataset video decode previously
required the explicit pyav override and now works with defaults. New unit
tests cover absent / loadable / installed-but-unloadable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(reviews): taking reviews into account - Adding RuntimeError in the catch, reformatting the warning message and un-bloating code.
---------
Co-authored-by: Ahmed Sohail Butt <butt4320@mylaurier.ca>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* docs(third party): adding a small list of noticeable third party robots packages
* feat(upgrade): adding new robots, teleoperators and sensors to the third party packages list
* feat(format): improving robots and cameras page formatting
* feat(format): formatting third party list
* chore(format): formatting code
* fix(typo): fixing typo
* fix(layout): fixing arrays layout
* fix(tables): fixing tables
* fix(borders): removing tables border lines
* fix(cameras): add color_format config and auto-recovery for RealSense D405
The D405 delivers color from its stereo depth module, not a dedicated
RGB sensor. The driver previously hardcoded rs.format.rgb8, causing
silent frame capture failure on D405.
Changes:
- Add color_format field to RealSenseCameraConfig (default rgb8, D405
users set bgr8), validated against whitelist
- Use configured format in _configure_rs_pipeline_config
- Fix _postprocess_image to handle both rgb8 and bgr8 source formats
- Add _hardware_reset auto-recovery: if warmup times out, perform USB
hardware reset and retry once (common D405 recovery path)
- Fix thread race in _read_loop (local ref to stop_event)
Continuation of #3164 (closed due to deleted fork).
Tested on Intel RealSense D405 at 1280x720@30fps with color_format=bgr8.
* style: fix ruff lint B904 and format
* refactor(cameras): clarify RealSense connection retry
* refactor(cameras): address review, retry RealSense connect before hardware reset
Drop color_format (device-side streaming state was the actual cause), keep _open_pipeline attempt-agnostic, catch only retry-worthy errors, reset only as last resort, guard read loop against late frame publication after stop.
* refactor(cameras): restore BaseException teardown, shorten stop-check comment
* test(cameras): expect ConnectionError after retries are exhausted
Same fix as #4207 (SO follower/leader), applied to LeKiwi, which reads its
motors through the same Feetech bus and had the same gap: `sync_read` was
called without `num_retry`, so a single corrupted status packet raised
ConnectionError and took the control loop down.
Adds `num_read_retries` (default 2, matching #4207) to LeKiwiConfig and
forwards it at all three read sites. Two of them are Present_Position, as in
#4207; the third is the Present_Velocity read of the omniwheel base, which is
LeKiwi-specific and is where this was observed in the field:
File "lerobot/robots/lekiwi/lekiwi.py", line 351, in get_observation
base_wheel_vel = self.bus.sync_read("Present_Velocity", self.base_motors)
ConnectionError: Failed to sync read 'Present_Velocity' on ids=[7, 8, 9]
after 1 tries. [TxRxResult] There is no status packet!
All nine servos pinged 20/20 with the bus idle immediately afterwards, so this
is transient corruption under load rather than a wiring fault.
Relates to #4207. Refs #3131.
rollout() runs `while not np.all(done)` with `done` latched, so a sub-env that
terminates early keeps being driven -- physics and offscreen rendering included
-- until the slowest sub-env in the batch finishes. A batch of N runs for
max(episode_lengths) iterations to complete work that only needs
mean(episode_lengths), and all of the surplus is discarded.
Adds FreezeAfterEpisodeEnd, applied to each sub-env of the eval vector env. It
caches the terminal transition and replays it for any further step(), and also
absorbs Gymnasium's autoreset -- under AutoresetMode.NEXT_STEP the vector env
otherwise rebuilds a finished sub-env and runs it through an entire extra episode
the rollout throws away. Reward is zeroed on replay so a frozen sub-env cannot
inflate a return if a caller sums over the padded tail.
Thawing is signalled explicitly: rollout() passes NEW_ROLLOUT_OPTION in
reset(options=...). Gymnasium's autoreset calls reset() with no arguments, but so
would a caller passing seeds=None, and inferring from that would strand an env
frozen for a whole rollout.
AutoresetMode.DISABLED is not an alternative -- Gymnasium asserts that no
terminated env is ever stepped in that mode, so the wrapper is never reached. An
earlier version of this patch used it and the vector-env test caught the assert.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* feat(cameras): prefer MJPG when fourcc unspecified; allow fourcc in ZMQ image server
With no fourcc set, OpenCV's V4L2 auto-negotiation selects uncompressed
YUYV when the camera offers it: ~16x the USB bandwidth of MJPG for
identical frames, which silently caps the frame rate on shared USB 2.0
buses (e.g. a Raspberry Pi with multiple cameras).
OpenCVCamera now prefers MJPG when config.fourcc is None: it requests
MJPG and keeps it only if the camera reports support (read-back),
falling back to the camera's own negotiation otherwise. An explicit
config.fourcc always takes precedence, so opting out is fourcc="YUYV".
The preference mirrors the existing pre/post size-and-fps ordering to
preserve the Windows DSHOW FOURCC-override handling.
Also adds a fourcc passthrough to the ZMQ image server, which previously
could not request a pixel format at all.
Behavior change: cameras that relied on implicit uncompressed capture
now receive camera-side JPEG unless fourcc is set explicitly.
* chore(cameras): address feedback
---------
Co-authored-by: Xingdong Zuo <18168681+zuoxingdong@users.noreply.github.com>
* fix(train): honor policy.use_amp when the policy has no dtype field
Since the Accelerate migration, accelerator.autocast() has been a no-op for
policies that do not expose a string dtype field (act, diffusion,
multi_task_dit, ...): PR #3912 wires mixed_precision from policy.dtype, so
these policies resolve to None and always train in full fp32 regardless of
--policy.use_amp=true. Meanwhile lerobot-eval does honor use_amp, and
PreTrainedConfig documents the flag as applying to training and evaluation.
Fall back to use_amp when no dtype field is present: bf16 where supported,
fp16 otherwise (matching torch.autocast's cuda default used by eval).
Measured on multi_task_dit / pusht, batch 256, H100: peak allocated drops
from ~75 GiB (fp32) to ~46.5 GiB under bf16 autocast.
* fix(train): conservative check
---------
Co-authored-by: Reece O'Mahoney <reece.omahoney3@gmail.com>
* fix(libero): don't reset inside step() on termination
LiberoEnv.step() called self.reset() when an episode terminated. Gymnasium's
vector envs default to AutoresetMode.NEXT_STEP, so the vector env resets the
sub-env again on the following step -- every termination paid two full resets.
The self-reset was also pure waste: `observation` is built from the terminal
raw_obs before it, so the reset's return value was discarded outright.
Worse, LiberoEnv.reset() advances init_state_id by _reset_stride, so the extra
reset skipped an initial state on every episode.
Measured with a counting subclass under SyncVectorEnv (gymnasium 1.3.0,
terminating every 4 steps over a 14-step loop):
n_envs=1: 3 terminations -> 6 resets = 1 initial + 3 self + 2 autoreset
n_envs=2: 6 terminations -> 12 resets = 2 initial + 6 self + 4 autoreset
(the final termination's autoreset does not fire before the loop ends)
Each LiberoEnv.reset() is a full LIBERO reset plus num_steps_wait settle steps,
so on the current default reset path this duplicates roughly 1.3 s per
termination.
Standalone (non-vectorised) users must now call reset() themselves after
termination, which is the Gymnasium contract.
* test(libero): pin the autoreset *default*, not the enum's spelling
The previous assertion checked `AutoresetMode.NEXT_STEP.value == "NextStep"`,
which is a naming detail. If a future Gymnasium flipped the vector-env default
to SAME_STEP, that assertion would still pass and the bug this fixes would come
back as a missing reset instead of a double one.
Assert the observable default instead. Verified on gymnasium 1.1.1 (the floor in
pyproject) and 1.3.0, for both SyncVectorEnv and AsyncVectorEnv. Negative control:
constructing with autoreset_mode=SAME_STEP fails the new assertion and passes the
old one.
Also document why `env._env = inner` is not redundant with the monkeypatched
factory: `LiberoEnv.__init__` defers simulator creation, so pre-binding keeps
`_ensure_env()` a no-op and avoids a stray `reset()` on the mock.
* test(libero): drop the redundant `env._env` prebind
@noron12234 flagged this as redundant. Their stated reason was wrong -- `__init__`
defers simulator creation (`self._env = None`, libero.py:180), so the monkeypatched
factory has not been called by the time it returns -- but the conclusion holds:
`_ensure_env()` pulls the same `inner` from the mocked factory on first `step()`.
I claimed the prebind was load-bearing because a stray `reset()` would pollute the
tests' call counts. That was wrong and I had not run it. Ran both variants against
the real class: the stray reset lands on `inner.reset`, which no assertion touches,
and both tests pass with or without the line. Dropping it.
* refactor(env): add explicit NEXT_STEP
* fix(style): pre-commit
---------
Co-authored-by: Dimitar Dimitrov <dvdimitrov13@gmail.com>
Co-authored-by: Dimitar Dimitrov <60075474+dvdimitrov13@users.noreply.github.com>
* perf(libero): skip the discarded scene rebuild on reset
LIBERO's OffScreenRenderEnv defaults to hard_reset=True, so every reset() frees
the MjSim, re-serialises the scene with model.get_xml(), recompiles it with
MjSim.from_xml_string(), constructs a fresh offscreen GL context and re-wires
every observable.
When init states are in use, LiberoEnv.reset() immediately calls
set_init_state(), which overwrites the whole sim state -- so all of that work is
discarded. This passes hard_reset=not init_states instead. Without init states
the randomisation reset() performs is the only thing placing the objects, so the
hard reset is kept.
Measured on an RTX 3060 Ti (EGL, robosuite 1.4.0, mujoco 3.2.7, 256x256 x2
cameras), through LiberoEnv.reset(), fresh env per arm:
suite hard soft saved
libero_spatial 1697 ms 233 ms 1464 ms
libero_object 1394 ms 172 ms 1222 ms
libero_goal 1177 ms 164 ms 1013 ms
libero_10 1528 ms 207 ms 1321 ms
Equivalence
-----------
Immediately after set_init_state, qpos, qvel, ctrl and act are bit-identical
between the two paths on every suite tested.
After the 10 settle steps that reset() runs, 9 of 41 qpos entries differ:
robot0_joint1..7 (<= 2.4e-5 rad) and gripper0_finger_joint1/2 (<= 2.1e-4 rad).
No object joint differs on any suite. The drift is driven by the gripper
component of the settle action ([0,0,0,0,0,0,-1]); replacing it with zeros keeps
the two paths bit-identical for 12 further steps, and with num_steps_wait=0 there
is no divergence at all.
Wrist-camera pixels can differ by up to ~87/255, because a sub-millimetre finger
displacement crosses rasterisation boundaries at 256x256. The pixel metric badly
overstates the physical difference here; 2.1e-4 rad is 0.012 degrees.
So this is not bit-identical end to end, and reviewers should decide whether
0.012 degrees of gripper drift is acceptable for the benchmark. It does not
change object placement, which is what the fixed init states exist to control.
* refactor(env): config param libero + docs
---------
Co-authored-by: Dimitar Dimitrov <dvdimitrov13@gmail.com>
Four kinematic processor steps read the observation as
observation = self.transition.get(TransitionKey.OBSERVATION).copy()
if observation is None:
raise ValueError("Joints observation is require for computing robot kinematics")
so `.copy()` runs first and the guard below it is unreachable. A transition
without an observation raises `AttributeError: 'NoneType' object has no
attribute 'copy'` instead of the intended message.
That transition is not hypothetical: `RobotProcessorPipeline.process_action`
builds one with `create_transition(action=action)`, which sets
`TransitionKey.OBSERVATION` to None.
Reads the value first, checks it, then copies. Affects EEReferenceAndDelta,
InverseKinematicsEEToJoints, GripperVelocityToJoint and InverseKinematicsRLStep.
Adds a parametrised regression test covering all four; each fails with the
AttributeError if the fix is reverted.
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
* fix(edit-dataset): redirect dataset.root via backup_path returned from get_output_path
The in-place backup logic compared `output_dir` (run through `.resolve()`) to
`dataset.root` (unresolved), so when `HF_LEROBOT_HOME` was a symlink the swap
to the `_old` backup never fired and `_copy_and_reindex_videos` then read from
the now-empty output directory. Have `get_output_path` return the backup path
directly so callers don't rely on path equality.
* feat(samefile): making same file detection more robust using os.path.samefile
---------
Co-authored-by: Reece O'Mahoney <reece.omahoney3@gmail.com>
* extend index.mdx
* bring community section up + bigger logo
* fix formatting
* fix logo size
* text fix
* improved explanation
* fix emoji and formatting
* fix link
- Quickstart on LIBERO: finetune lerobot/pi05_libero_base on lerobot/libero
with a complete, copy-pasteable command; feature table mapping the dataset
keys to how pi05 consumes them; gated PaliGemma tokenizer tip.
- Explain --policy.path vs --policy.pretrained_path (weights+config vs
weights-only) and why n_action_steps/empty_cameras must be passed
explicitly with pretrained_path.
- Quantile statistics section: the exact error message, the
lerobot-edit-dataset recompute_stats fix (replacing the removed
augment_dataset_quantile_stats.py reference), where the result lands, and
the MEAN_STD alternative.
- Update stale link lerobot/pi05_libero -> lerobot/pi05_libero_base; add
PyPI install variant.
Co-authored-by: Xingdong Zuo <18168681+zuoxingdong@users.noreply.github.com>
* feat(rollout): add smooth_handover flag to DAgger strategy config
The DAgger phase transitions run blocking smooth handovers: on pause the
leader is driven to the follower (~2 s), and on correction start the
follower is slid to the teleop pose (~1 s), both inside the record loop.
For clutch-style teleoperators (e.g. VR controllers) that re-reference
their command frame at the current robot pose on engage, the handover is
already continuous — the interpolation only delays the start of the
correction and eats its first frames.
Add --strategy.smooth_handover (default true, existing behavior
unchanged) to let such setups skip it, mirroring the episodic strategy's
smooth_leader_to_follower_handover flag.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: pre-commit auto-fix (prettier markdown table formatting)
---------
Co-authored-by: griffinaddison <griffinnosidda@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(rollout): add smooth_handover flag to episodic strategy config
Follow-up to #3985, which added the same flag to the DAgger strategy.
The episodic strategy's reset-phase handover had two gaps:
- Non-actuated teleops could not skip the blocking follower slide at all.
- Setting smooth_leader_to_follower_handover=false on an actuated teleop
swapped which arm moves instead of skipping the handover.
Add --strategy.smooth_handover (default true, existing behavior
unchanged) as a master switch that skips the interpolation entirely,
for clutch-style teleops that re-reference at the current robot pose
on engage.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: fix table formatting via prettier
---------
Co-authored-by: griffinaddison <gaddison@seas.upenn.edu>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(vla_jepa): use device-safe autocast instead of hardcoded bfloat16
VLA-JEPA hardcodes torch.autocast with dtype=torch.bfloat16, which
crashes on MPS (no AMP support) and silently misbehaves on pre-Ampere
CUDA GPUs (no bf16). Add a _get_autocast_context() helper that reuses
the existing is_amp_available() utility to pick a safe strategy per
device, matching the pattern used by pi05 and molmoact2.
Fixes#3744
* fix: use _get_autocast_context for fp32 action head (MPS compat)
---------
Co-authored-by: devangpratap <115096812+devangpratap@users.noreply.github.com>
* feat(transforms): add 8 robotics-relevant image augmentations
Add GaussianNoise, MotionBlur, JPEGCompression, GaussianPatchBrightness,
RandomShadow, CoarseDropout, GammaCorrection, and PlanckianJitter.
Each transform addresses a real-world failure mode not covered by the
existing 6 defaults (sensor noise, motion blur, compression artifacts,
uneven lighting, cast shadows, partial occlusion, exposure variation,
color temperature shift).
All transforms are pure PyTorch, follow the make_params/transform
pattern, and integrate with ImageTransformConfig via a registry.
* add augmentation showcase image for PR
* update showcase with better sample frame
* tune showcase to balanced augmentation intensity
* tune showcase: softer shadow, dropout, jitter intensity
* refactor(transforms): several updates
* update image
* chore(media): remove example
* chore: add link to example
---------
Co-authored-by: Yuxian LI <liyuxian1358@gmail.com>
* Fix add_features for multi-dimensional per-frame features
* Use generic names in multidimensional add_features test
* tests(all shapes): enhancing tests to cover all possible features shapes
* chore(format): formatting code
---------
Co-authored-by: felixmin <felix.minze@live.de>
* Support task replacement mappings in modify_tasks
Part of #2326.
Signed-off-by: 陈伟 <woshei0a0a0a@qq.com>
* feat(task modification precedence): Improving task modification precedence so that all modes can be used in a single run. Adapting tests accordinginly.
* chore(fromat): formatting code
* docs(docstrings): updating docstrings
* docs(update): updating docs with the task modification features
---------
Signed-off-by: 陈伟 <woshei0a0a0a@qq.com>
Co-authored-by: vvezre <93599357+vvezre@users.noreply.github.com>
* docs: fix broken and placeholder links
Fix the two links called out in #4094 plus one more found while scanning
all docs: the NOTE:addlinktoLOC placeholder in async.mdx now points at
_aggregate_action_queues in robot_client.py; the dead
#mapping-videoencoderconfig--ffmpeg-options anchor in
video_encoding_parameters.mdx now points at get_codec_options in
video.py; and the stale ./cameras#setup-cameras fragment in il_robots.mdx
is dropped to link the cameras guide page.
Closes#4094.
* docs: apply prettier formatting to the video-encoding parameter table
Re-align the Markdown table columns with prettier (v3.6.2, --prose-wrap=preserve)
so the pre-commit 'Format Markdown with Prettier' hook passes. Whitespace only.
CarolinePascal