lerobot/INTERACTIVE_ROLLOUT.md

# Interactive Rollout — Design Notes

Branch: `feat/add_interactive_rollout` · Status: Phases 1–2 committed; Round 2
(programmatic API, sentry support, muting v2, stdin move) implemented and tested,
uncommitted.

---

## 1. Vision

`lerobot-rollout` runs inference on a real robot: it connects hardware, loads the policy,
builds the processor pipelines, optionally records a dataset, and spins the control loop.
Today that is a **one-shot, fire-and-forget** program. You pass `--task="pick up the cube"`
on the command line, the robot starts moving immediately, and the only interaction left is
Ctrl-C. If you want a different instruction, you kill the process and pay the full startup
cost again — reconnecting motors, re-homing, re-loading a multi-GB VLA onto the GPU.

Since LeRobot gained subtask annotation and language conditioning, that model is the
bottleneck. The **north star** is a chat-style CLI over stdin, where the operator issues
commands *concurrently with the robot moving*:

```
/start                              begin (or resume) the policy control loop
/subtask Grab the red cube          re-instruct the policy on the fly
/ask what's the capital of France?  query an LLM while the robot keeps moving
/reset                              stop movement, return home, clear the subtask —
                                    but keep hardware and policy warm
/stop                               graceful shutdown
```

The unifying idea: **the expensive things (hardware, policy weights, processors) stay warm
across commands.** Only the cheap things — the instruction, the control loop — start and
stop. That turns a rollout from a batch job into a session you can steer.

## 2. Objective (scoped)

Phased, so each phase lands as a reviewable unit:

| Phase | Scope | Status |
|---|---|---|
| **1** | `--interactive` flag, non-blocking stdin listener, command parser, `/start` `/reset` `/stop` `/help` | ✅ done |
| **1.5** | Mute system logs so they stop fighting the prompt for the terminal | ✅ done |
| **2** | `/subtask <text>` — change the policy's instruction mid-run | ✅ done |
| **2.5** | Round 2: `RolloutController` public API, sentry recording support, muting v2 (errors surface), stdin listener → `lerobot/utils` | ✅ done (see §5) |
| **3** | `/ask` + hierarchical task-vs-subtask semantics (LLM in the loop) | not started |

An explicit constraint through Phases 1–2: **do not couple this to the language runtime yet.**
Build the mechanism; keep the door open.

## 3. Inspiration — three reference PRs

We read all three and deliberately implemented none of them verbatim.

**PR #4108 — online subtask switching.** Introduces a `PromptBroker` + `PromptListenerBase`
+ `StdinPromptListener`, a `RuntimeContext.prompt_broker` field, `register_on_change`
callbacks, an `--online_task_switching_flush` config flag, and `flush_action_queue()` /
`_apply_pending_flush()` on `PreTrainedPolicy` — **with edits to 14 policy files** to call
the flush at the top of `select_action`. Its architecture is designed for pluggable input
sources (network, voice), which is the right long-term shape but more machinery than we
need. *What we took:* the core insight that a mid-run instruction change must invalidate
actions precomputed under the old instruction, and that the flush must happen on a thread
that is safe to touch policy state from.

**PR #4183 — experimental full-UX draft.** Achieves the whole north-star vision, but does
so by adding a `lerobot.runtime` / `language_runtime.py` that **duplicates** `BaseStrategy`,
`send_next_action`, and the rollout control loop. *What we took:* the UX target and the
command vocabulary. *What we rejected:* the parallel runtime — a second control loop is a
second thing to keep correct, and everything it does is already in `rollout/strategies/`.

**PR #4234 — policy-side edits enabling #4183's runtime.** Read for context on where the
language plumbing lands inside a policy. Relevant to Phase 3, not to what we built.

## 4. What we built, and why

Three commits on the branch:

```
072c697c0  feat(rollout): interactive v1
d3ee0b820  feat(rollout): mute logs in interactive mode
39c4e746f  feat(rollout): add subtask command
```

Cumulative footprint — one new module, one new test file, small surgical edits elsewhere:

```
 src/lerobot/rollout/interactive.py         | 580 +++++   (new)
 tests/test_interactive_rollout.py          | 788 +++++   (new)
 docs/source/inference.mdx                  |  87 +++
 src/lerobot/rollout/inference/base.py      |  66 +++
 src/lerobot/rollout/inference/rtc.py       |  61 +-
 src/lerobot/rollout/inference/sync.py      |  21 +-
 src/lerobot/scripts/lerobot_rollout.py     |  32 +-
 src/lerobot/policies/pretrained.py         |  24 +
 src/lerobot/rollout/strategies/core.py     |  21 +-
 src/lerobot/rollout/configs.py             |  18 +
 src/lerobot/rollout/__init__.py            |  16 +-
 src/lerobot/rollout/strategies/episodic.py |   4 +-
```

The ratio matters: **~1400 of ~1680 added lines are the new module and its tests.** The
existing rollout architecture was reused, not reshaped.

### 4.1 Segments over a linked event — the load-bearing idea

Every rollout strategy's control loop already polls `ctx.runtime.shutdown_event.is_set()`
to know when to stop. So instead of teaching strategies about interactivity, we **swap in a
smarter event**:

```python
class LinkedEvent(Event):
    """is_set() reflects the local flag OR a parent event."""
    def is_set(self) -> bool:
        return super().is_set() or self.parent.is_set()
```

`lerobot-rollout` wraps the `ProcessSignalHandler`'s shutdown event in a `LinkedEvent` when
`--interactive=true`. The session sets the **local** flag to end a run *segment*; SIGINT /
SIGTERM still arrive through the **parent**, so Ctrl-C behaves exactly as before.

`InteractiveSession.run()` then drives `strategy.run(ctx)` in restartable segments:

```
setup(ctx)  →  [idle]  →  /start → run(ctx) → /reset → [idle] → /start → run(ctx) → /stop  →  teardown(ctx)
                                                  ↑ hardware + policy stay warm throughout
```

**Zero strategy code changed** to support this. The only additions to `strategies/core.py`
were `reset_control_state()` (engine + interpolator + cached-observation reset, factored
out of `_init_engine` so a segment can restart cleanly) and making
`_return_to_initial_position` public.

### 4.2 Threading model

```
 listener thread  ──publishes flags / strings──▶  main thread
 (stdin reader)      never touches hardware        (session loop → strategy.run → control loop)
                     never mutates policy state
```

The listener only ever writes `threading.Event` flags and a lock-guarded string. Everything
that touches hardware or policy state happens on the thread that already owns it. This
mirrors the existing DAgger events pattern rather than inventing a new concurrency idiom.

### 4.3 stdin must be read with `os.read`, not `readline`

Non-obvious and load-bearing. The first implementation used `select()` + `stream.readline()`
and **two tests failed**: a buffered file object slurps *several* lines off the file
descriptor in one syscall, after which `select` reports the drained fd as not-ready and the
buffered lines are never delivered. Pasted or piped command batches got stuck. The reader
now does `select()` + `os.read(fd, 4096)` + manual `\n` splitting, with a
blocking-`readline` fallback for streams without a `fileno()` (non-POSIX, test doubles).

Also: unlike `TerminalKeyListener`, this reader leaves the terminal in **canonical mode** —
the operator is typing chat commands, not pressing hotkeys.

### 4.4 EOF means stop

A closed stdin means there is no way left to command the robot, so EOF (Ctrl-D, or an
exhausted piped script) stops the session. An unexpected read error is treated the same way,
for the same reason. Consequence, documented: piped scripts must hold stdin open —

```bash
(printf '/start\n'; sleep 60; printf '/stop\n') | lerobot-rollout ... --interactive=true
```

### 4.5 Commands are last-write-wins

`/reset` and `/stop` cancel a still-pending `/start`, so the robot never starts moving after
the operator's most recent command said not to. Handlers set their intent flag *first* and
the segment-stop event *second*; `_run_segment` clears the segment-stop flag *before*
re-checking the intent flags. A `/reset` racing a `/start` is therefore either seen before
the segment begins or ends it on its first tick.

### 4.6 Base strategy only (enforced by config validation)

`--interactive=true` with a recording strategy raises a `ValueError`. Two reasons: recording
strategies finalize their dataset inside `run()` (so `run()` is not restartable), and their
keyboard listeners contend with the command reader for the same TTY. This is a deliberate,
documented limitation — not an oversight.

### 4.7 Log muting (Phase 1.5)

Policy, robot and control-loop logs at every level interleave with the chat prompt and
destroy the typing UX. Simplest workable answer, per explicit request: **mute console output
for the duration of the session.**

- Every logger's console `StreamHandler` is raised above `CRITICAL` — **not just root**,
  because `transformers` and `datasets` attach their own stderr handlers with
  `propagate=False`.
- `warnings.simplefilter("ignore")`, with `warnings.filters` saved and restored.
- **File handlers are untouched** — anyone wanting a persistent log can attach one.
- Restored in `run()`'s `finally`, *before* the closing `log_say`, so teardown logs are visible.

The obvious hazard: muting hides fatal errors. So `InferenceEngine` gained a
`failure_traceback` property, RTC captures its traceback in the fatal handler, and the
session prints it on failure. **Do not remove that when touching the failure path.**

"See both logs and prompt" — a pinned input line, `prompt_toolkit`-style — was deliberately
deferred: it needs a new dependency and a real TUI layer.

### 4.8 `/subtask` — the engine *is* the broker

The pivotal call on Phase 2: **skip PR #4108's `PromptBroker`.** After Phase 1, the session
already owns the stdin thread and the parser, so a broker + listener base + on-change
callbacks + a new `RuntimeContext` field would be duplicate machinery — and callbacks firing
on the listener thread are exactly the cross-thread hazard we designed against.

Instead, `InferenceEngine` (the ABC every backend already implements) became the thread-safe
task holder:

```python
@property
def task(self) -> str: ...              # lock-guarded read

def set_task(self, task) -> bool:       # callable from ANY thread; True if it changed
    ...

def _take_task(self) -> tuple[str, bool]:   # consumed on the INFERENCE thread;
    ...                                     # returns (task, changed) and clears the edge
```

`/subtask` is then three lines: read `engine.task`, call `engine.set_task(text)`, print the
transition. No new module, no new context field, no callbacks.

**The flush problem, and why it got small.** When the instruction changes, a chunking policy
is still serving actions computed under the old one — up to `chunk_size` ticks of stale
behavior. PR #4108 solved this by adding `flush_action_queue()` / `_apply_pending_flush()`
to `PreTrainedPolicy` **and editing 14 policy files**, because its flush request arrived from
a foreign thread and had to be deferred to a safe point inside `select_action`.

Ours already runs *on* the thread that calls `select_action`. So: one concrete method on
`PreTrainedPolicy` and **zero per-policy edits**.

```python
def drop_queued_actions(self) -> None:
    queues = getattr(self, "_queues", None)
    if isinstance(queues, dict) and ACTION in queues:
        queues[ACTION].clear()
    action_queue = getattr(self, "_action_queue", None)
    if action_queue is not None:
        action_queue.clear()
```

Two `getattr`s cover the repo's two queue idioms across all ~18 policies
(`_queues[ACTION]`: diffusion, smolvla, tdmpc, vqbet, wall_x, xvla, multi_task_dit, vla_jepa;
`_action_queue`: act, pi0, pi05, pi0_fast, eo1, evo1, groot, molmoact2, fastwam, lingbot_va).
Policies with no queue inherit a no-op.

**Why not `policy.reset()`?** That was the first implementation, and review caught it as too
blunt. For Diffusion it wipes the observation history, so the next chunk is planned from a
history of the current frame repeated — a visible discontinuity mid-motion. And ACT /
Diffusion / VQBeT / TDMPC don't read `task` at all, so they'd pay that jerk for nothing.
`drop_queued_actions` keeps episode state and drops only what is actually stale.

**RTC deliberately does *not* flush.** Clearing its queue would leave the robot with no
commands for a full inference latency (~1 s on a VLA). Instead the next chunk is generated
under the new instruction and merged over the previous chunk's leftover prefix — the switch
lands within one inference and the motion stays continuous. That is exactly what RTC's
blending exists for. Documented per-backend in `inference.mdx`; no config flag, one sensible
default per backend.

**`/reset` restores the launch task on the listener thread.** Subtle and worth preserving:
the restore lives in `_cmd_reset`, not in `_reset_robot` (which runs later, on the main
thread). Otherwise `/reset` followed immediately by `/subtask` would be ordered by *service*
time rather than *command* time, and the deferred restore would silently revert the new
instruction — deterministically so, for pasted or piped input. Both writers now run on the
same thread, so command order wins. There is a regression test driving this through a real pipe.

## 5. Round 2 — the feature becomes a library API

Four follow-up asks landed together (currently uncommitted on the branch):
make the components programmatic-API friendly (the priority), extend interactive
to recording where cheap, simplify muting / surface errors, and settle the
ssh/headless + `keyboard_input` question.

### 5.1 `RolloutController` — programmatic control

`interactive.py` bisected cleanly, so the generic control logic moved to a new
`rollout/controller.py`:

```python
controller = RolloutController(strategy, ctx, on_event=my_observer)
controller.serve()      # blocking loop (run it on whatever thread you like)
controller.start()      # -> bool: False when a segment is already running
controller.set_task(t)  # -> bool: re-instruct mid-run, from any thread
controller.reset()      # -> bool: True when the launch task was restored
controller.stop()
controller.task / .initial_task / .running / .failed / .failure_traceback
```

- **No I/O of its own** — no stdin, no prints, no log muting, no TTS. Every
  state transition that used to be a `print` is now a `RolloutEvent`
  (`SEGMENT_STARTED`, `SEGMENT_ENDED`, `RESET_STARTED/DONE/SKIPPED`,
  `ENGINE_FAILED`, `STOPPED`) emitted on the serve thread.
- **Thread-safe by lock, not by convention.** The old ordering guarantee
  (`/subtask` right after `/reset` must win) relied on both writes running on
  the single stdin thread. The controller serializes `start`/`reset`/`stop`/
  `set_task` with an internal lock, so the guarantee now holds for arbitrary
  caller threads — the prerequisite for network/voice front-ends.
- `InteractiveSession` shrank to a thin adapter: stdin listener + parser +
  rendering + muting; each command maps 1:1 onto a controller method, and the
  controller is exposed as `session.controller`.
- Exported from `lerobot.rollout`: `RolloutController`, `RolloutEvent`,
  `LinkedEvent`. `docs/source/inference.mdx` gained a **Programmatic control**
  section with a complete embedding example.

### 5.2 Sentry + interactive — recording while you steer

Decision, per the agreed criteria: the `/record` keyboard-handoff idea is
**medium-to-large** (listeners have no suspend/resume API and start at
creation, `esc` handlers are hardcoded and collide, pynput captures globally
while you type, and each strategy carries per-run stale flags) → rejected.
But the investigation showed **sentry has zero keyboard code** — the config
comment lumping it with the keyboard strategies was simply wrong — and its
only real blocker was one line: `with VideoEncodingManager(dataset)` inside
`run()` finalizes the dataset the first time `run()` returns, after which a
restarted segment would silently truncate the finalized parquet.

So `--interactive=true` now supports `--strategy.type=sentry`:

- **Finalization moved to `teardown()`** (which already called
  `dataset.finalize()`); `run()` is segment-restartable. Each segment saves
  complete episodes plus one tail partial episode; on a failed tail save the
  in-flight streaming encode is cancelled *and* the half-mutated episode
  buffer is discarded (see §6, round 2).
- **Frames are labeled with the live `engine.task`** instead of a config
  snapshot — the writer already stores a task per frame — so `/subtask`
  changes the policy conditioning and the recorded label from the same frame
  onward. This also resolved the "recorded frames ignore `/subtask`" open item
  for sentry.
- `episodes_since_push` hoisted to instance state so upload cadence survives
  segments.
- dagger / highlight / episodic stay excluded: keyboard conflicts plus per-run
  recording state that does not survive a restart.

### 5.3 Muting v2 — two lines, and errors surface

The ~30-line per-handler walk became `logging.disable(logging.WARNING)` with
the previous disable level restored afterwards. Strictly better coverage: the
gate applies before handler dispatch, so it covers `propagate=False` library
loggers *and* loggers created mid-session (the old snapshot missed those) —
and **ERROR/CRITICAL now reach the console**, which the audit showed is safe:
no ERROR-level emitter fires periodically in healthy operation (the periodic
nuisances — slow-loop, camera hiccups — are WARNINGs and stay muted).
Documented trade-off: the gate also withholds INFO/WARNING from file handlers
during the session; acceptable because no default code path attaches one
(only `rl/actor`, `rl/learner`, `async_inference` pass `log_file`). The
`warnings` suppression stays (nothing calls `logging.captureWarnings`), and
`failure_traceback` surfacing stays as the belt-and-suspenders for fatal
engine errors.

### 5.4 stdin listener → `lerobot/utils/stdin_input.py`

The ssh/headless audit confirmed the listener was already the right design:
`select`+`os.read` works over SSH (the session pty is a normal fd), from
pipes, and headless — it's `keyboard_input`'s **pynput** backend that needs a
display server. Nothing in `keyboard_input` overlaps enough to reuse
(1-byte cbreak hotkey decoding vs canonical-mode line assembly), so
`StdinCommandListener` moved to a **new** utils module — deliberately not
into `keyboard_input.py`, which attempts a pynput import at module load.
Canonical import only: `lerobot.utils.stdin_input` (removed from
`lerobot.rollout`'s exports).

The move fixed a real bug the audit found: with `sys.stdin is None`
(daemonized processes), the blocking fallback died with an uncaught
`AttributeError` without firing `on_eof` — leaving a session idling with no
command channel. `start()` now treats a missing stream as immediate EOF.

## 6. Bugs the adversarial reviews caught

Four multi-agent review passes were run across the phases (28 / 5 / 27 / 12 agents;
findings adversarially verified before acting). The ones that mattered:

**Round 2 (2 confirmed, 0 refuted):**

- **Controller `start()` race → phantom segment.** `start()` gated on `_running`, but the
  serve loop cleared `_start_requested` *before* setting `_running` — a second `start()`
  landing in that window (spanning `reset_control_state` and the SEGMENT_STARTED emission)
  returned `True` and re-armed the flag, which nothing consumed during the segment; the
  robot would start again, uncommanded, when the segment later ended on its own. Fixed:
  the serve loop consumes the request and sets `_running` atomically under the control
  lock, and `_running` spans the whole startup sequence.
- **Sentry poisoned episode buffer.** `save_episode` mutates the buffer in place (pops
  `size`/`task`) *before* the fallible writes; a failed tail save left a half-mutated dict
  and the next segment's first `add_frame` crashed with `KeyError('size')`. Fixed: the
  except branch discards the buffer so `add_frame` recreates it.

**Rounds 1–3 (Phases 1–2):**

- **RTC stale observation (critical).** `RTCInferenceEngine.reset()` never cleared
  `_obs_holder["obs"]`. After `/reset` physically moved the robot home, the next `/start`
  computed its first chunk from the **pre-reset pose** — a lurch back toward where the arm
  used to be. Fixed by clearing the observation and adding a `_reset_epoch` counter so an
  in-flight chunk computed across a reset is discarded rather than merged. This also fixes a
  pre-existing DAgger staleness path.
- **Muting hid fatal errors** → `failure_traceback` capture + session print (§4.7).
- **Muting scope too narrow** → root-only missed `transformers` / `datasets`; `warnings`
  output bypassed logging entirely.
- **Command ordering** → `/reset` and `/stop` didn't cancel a pending `/start` (§4.5); the
  `/reset`-then-`/subtask` clobber (§4.8).
- **Flush too heavy** → `policy.reset()` → `drop_queued_actions()` (§4.8).
- **Empty-task rendering** → `''` replaced with `(none — set one with /subtask <text>)`.
- **Silent switch** → the confirmation now says "(applies from the next policy inference)",
  since the explanatory logs are muted.

## 7. Verification

After Round 2:

```
uv run --extra dataset pytest tests/test_interactive_rollout.py \
    tests/utils/test_stdin_input.py tests/test_rollout.py -q
    → 81 passed

pre-commit (all changed files)
    → 0 failures
```

Phase 1–2 numbers (still green at the time): 64 rollout/interactive tests;
223 passed / 5 skipped across `tests/policies/rtc`, factory, and common
(confirming the shared `pretrained.py` change); pre-commit 0 failures.

`tests/test_interactive_rollout.py` covers the parser, `LinkedEvent` semantics,
`RolloutController` (start/reset/stop/set_task flows, events, startup-race
rejection, failure surfacing, broken observers), session flows (start / reset /
restart / stop, cancel-pending-start, engine failure with traceback, natural
end, EOF, and a real `BaseStrategy` end-to-end), muting (INFO/WARNING blocked,
ERROR surfaces, pre-existing disable level restored), `/subtask` semantics,
sentry restartability + live labels + failed-tail-save recovery, the engine
task holder, the sync flush, and `drop_queued_actions`.
`tests/utils/test_stdin_input.py` covers the listener (select path, batched
lines, blocking fallback, EOF, handler errors, None-stdin, broken streams).

## 8. Extension points for Phase 3

The design was built to make `/ask` an additive change:

- **Command table.** `InteractiveSession._commands` is `name → (handler, arg hint, help)`.
  `/help` and the startup banner render from it, so a new command is documented for free.
- **Controller API.** New front-ends (network, voice, `/ask`'s LLM worker) call
  `RolloutController.start/reset/stop/set_task` from their own threads — the internal lock
  makes that safe — and observe `RolloutEvent`s instead of scraping terminal output.
- **Thread discipline.** A command handler runs on the listener thread and must only call
  controller methods. An LLM call belongs on its own worker thread so the robot keeps
  moving — precisely the concurrency `/ask` is meant to demonstrate.
- **Task holder.** `set_task` / `_take_task` already give any producer a safe way to
  re-instruct the policy. Hierarchical task-vs-subtask semantics (per #4183 / #4234) layer
  on top of it rather than replacing it.

Open items, deliberately not addressed:

- dagger / highlight / episodic remain non-interactive (keyboard conflicts + per-run
  recording state); they also still snapshot the task label per run. Sentry is the
  supported recording path for interactive sessions.
- The "see logs and prompt simultaneously" TUI (pinned input line).
- Non-stdin input sources (network, voice) — now unblocked by `RolloutController`; #4108's
  pluggable-listener shape remains the reference for the transport layer.