fix(rtc): validate trained prefix capacity

This commit is contained in:
Pepijn
2026-07-16 12:30:40 +02:00
parent fa477fca5a
commit c395286f3c
10 changed files with 125 additions and 18 deletions
+1 -1
View File
@@ -280,7 +280,7 @@ lerobot-rollout \
| `--inference.rtc.mode` | `guided` (default) or trained-prefix `trained` for compatible Pi052 checkpoints |
| `--inference.rtc.max_guidance_weight` | Consistency enforcement strength (default: varies by policy) |
| `--inference.rtc.prefix_attention_schedule` | Blend schedule: `LINEAR`, `EXP`, `ONES`, `ZEROS` |
| `--inference.queue_threshold` | Max queue size before backpressure (default: 30) |
| `--inference.queue_threshold` | Backpressure threshold; trained RTC requires at least its maximum delay |
See the [Real-Time Chunking](./rtc) guide for details on tuning RTC parameters.
+3 -2
View File
@@ -152,8 +152,9 @@ than `chunk_size`. Choose it to cover the largest inference latency expected at
deployment: at 50 Hz, for example, 10 steps correspond to 200 ms. A delay of
zero is included in the uniform sampling distribution, so the checkpoint also
continues to receive ordinary flow-matching examples. Set rollout's
`inference.rtc.execution_horizon` to at least this maximum so the previous
chunk cache retains enough actions to construct every supported prefix.
`inference.rtc.execution_horizon` and `inference.queue_threshold` to at least
this maximum so inference starts early enough and the previous chunk retains
every action needed for the committed prefix.
Run the resulting checkpoint with the asynchronous `lerobot-rollout` backend
and select the trained-prefix path explicitly:
+8
View File
@@ -97,6 +97,10 @@ for step in range(num_steps):
- `guided` (default) applies the original Jacobian guidance during denoising and works with ordinary flow-matching checkpoints.
- `trained` hard-inpaints the previous chunk's prefix with per-action flow timesteps. It currently requires a Pi052 checkpoint trained with `policy.rtc_training_max_delay > 0` and avoids the guidance backward pass.
For trained mode, both `execution_horizon` and the rollout backend's
`inference.queue_threshold` must be at least the checkpoint's
`rtc_training_max_delay`; rollout validates this before connecting the robot.
**`execution_horizon`**: How many timesteps from the previous chunk to maintain consistency with. Higher values mean smoother transitions but potentially less reactivity.
Typical values: 8-12 steps
@@ -129,6 +133,10 @@ python examples/rtc/eval_dataset.py \
--device=cuda
```
Add `--rtc.mode=trained` when evaluating a compatible training-time RTC Pi052
checkpoint. Unsupported policies reject trained mode instead of falling back to
guided RTC.
The script generates a visualization of the denoising process, comparing standard generation (left) with RTC (right). In the RTC plots, you can see how the first few steps (blue/purple lines) are guided to match the red ground truth trajectory (previous chunk's tail), ensuring a smooth transition between chunks.
<p align="center">