admin/lerobot
1
0
Fork 0
mirror of https://github.com/huggingface/lerobot.git synced 2026-07-21 17:01:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(rollout): remote inference draft

Browse Source
This commit is contained in:
Steven Palma
2026-06-12 02:01:41 +02:00
parent 87242cfced
commit fc019d3902
54 changed files with 7883 additions and 3392 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/source/rtc.mdx
+9 -9
View File
@@ -151,18 +151,18 @@ lerobot-rollout \
--device=cuda
```
## How It Differs from the Async Inference in LeRobot
## How It Relates to Remote Inference
Both RTC and [async inference](./async) improve real-time robot control, but they solve different problems.
Both RTC and [remote inference](./remote_inference) improve real-time robot control, but they solve different problems.
| Aspect | Async Inference | RTC |
| ------------- | -------------------------------------------------------------------------- | --------------------------------------------------- |
| **Problem** | Idle frames while waiting for inference | Discontinuities between action chunks |
| **Solution** | Decouple prediction from execution | Guide new chunks to continue smoothly from previous |
| **Benefit** | No waiting, continuous action | Smooth transitions, natural motion |
| **Best Used** | Async inference is best used with large models with high inference latency | Flow-matching based policies |
| Aspect | Remote Inference | RTC |
| ------------- | ------------------------------------------------------------------------ | --------------------------------------------------- |
| **Problem** | The policy is too large (or too slow) for the edge machine | Discontinuities between action chunks |
| **Solution** | Run inference on a GPU server; the robot executes buffered action chunks | Guide new chunks to continue smoothly from previous |
| **Benefit** | Weightless edge clients, one GPU serves many robots | Smooth transitions, natural motion |
| **Best Used** | Large models with high inference latency, robot fleets | Flow-matching based policies |
**Use both together** for maximum smoothness and reactivity!
**Use both together** (`--inference.type=remote` with `--inference.rtc.execution_horizon=...`) for maximum smoothness and reactivity: the remote engine reuses RTC's chunk-merging machinery client-side while the server runs prefix-conditioned chunk generation.
## Advanced: Debug Tracking