lerobot/INTERACTIVE_ROLLOUT.md

Interactive Rollout — Design Notes

Branch: feat/add_interactive_rollout · Status: Phases 1–2 committed; Round 2 (programmatic API, sentry support, muting v2, stdin move) implemented and tested, uncommitted.

---

1. Vision

lerobot-rollout runs inference on a real robot: it connects hardware, loads the policy, builds the processor pipelines, optionally records a dataset, and spins the control loop. Today that is a one-shot, fire-and-forget program. You pass --task="pick up the cube" on the command line, the robot starts moving immediately, and the only interaction left is Ctrl-C. If you want a different instruction, you kill the process and pay the full startup cost again — reconnecting motors, re-homing, re-loading a multi-GB VLA onto the GPU.

Since LeRobot gained subtask annotation and language conditioning, that model is the bottleneck. The north star is a chat-style CLI over stdin, where the operator issues commands concurrently with the robot moving:

/start                              begin (or resume) the policy control loop
/subtask Grab the red cube          re-instruct the policy on the fly
/ask what's the capital of France?  query an LLM while the robot keeps moving
/reset                              stop movement, return home, clear the subtask —
                                    but keep hardware and policy warm
/stop                               graceful shutdown

The unifying idea: the expensive things (hardware, policy weights, processors) stay warm across commands. Only the cheap things — the instruction, the control loop — start and stop. That turns a rollout from a batch job into a session you can steer.

2. Objective (scoped)

Phased, so each phase lands as a reviewable unit:

Phase	Scope	Status
1	--interactive flag, non-blocking stdin listener, command parser, /start /reset /stop /help	✅ done
1.5	Mute system logs so they stop fighting the prompt for the terminal	✅ done
2	/subtask <text> — change the policy's instruction mid-run	✅ done
2.5	Round 2: RolloutController public API, sentry recording support, muting v2 (errors surface), stdin listener → lerobot/utils	✅ done (see §5)
3	/ask + hierarchical task-vs-subtask semantics (LLM in the loop)	not started

An explicit constraint through Phases 1–2: do not couple this to the language runtime yet. Build the mechanism; keep the door open.

3. Inspiration — three reference PRs

We read all three and deliberately implemented none of them verbatim.

PR #4108 — online subtask switching. Introduces a PromptBroker + PromptListenerBase

• StdinPromptListener, a RuntimeContext.prompt_broker field, register_on_change callbacks, an --online_task_switching_flush config flag, and flush_action_queue() / _apply_pending_flush() on PreTrainedPolicy — with edits to 14 policy files to call the flush at the top of select_action. Its architecture is designed for pluggable input sources (network, voice), which is the right long-term shape but more machinery than we need. What we took: the core insight that a mid-run instruction change must invalidate actions precomputed under the old instruction, and that the flush must happen on a thread that is safe to touch policy state from.

PR #4183 — experimental full-UX draft. Achieves the whole north-star vision, but does so by adding a lerobot.runtime / language_runtime.py that duplicates BaseStrategy, send_next_action, and the rollout control loop. What we took: the UX target and the command vocabulary. What we rejected: the parallel runtime — a second control loop is a second thing to keep correct, and everything it does is already in rollout/strategies/.

PR #4234 — policy-side edits enabling #4183's runtime. Read for context on where the language plumbing lands inside a policy. Relevant to Phase 3, not to what we built.

4. What we built, and why

Three commits on the branch:

072c697c0  feat(rollout): interactive v1
d3ee0b820  feat(rollout): mute logs in interactive mode
39c4e746f  feat(rollout): add subtask command

Cumulative footprint — one new module, one new test file, small surgical edits elsewhere:

 src/lerobot/rollout/interactive.py         | 580 +++++   (new)
 tests/test_interactive_rollout.py          | 788 +++++   (new)
 docs/source/inference.mdx                  |  87 +++
 src/lerobot/rollout/inference/base.py      |  66 +++
 src/lerobot/rollout/inference/rtc.py       |  61 +-
 src/lerobot/rollout/inference/sync.py      |  21 +-
 src/lerobot/scripts/lerobot_rollout.py     |  32 +-
 src/lerobot/policies/pretrained.py         |  24 +
 src/lerobot/rollout/strategies/core.py     |  21 +-
 src/lerobot/rollout/configs.py             |  18 +
 src/lerobot/rollout/__init__.py            |  16 +-
 src/lerobot/rollout/strategies/episodic.py |   4 +-

The ratio matters: ~1400 of ~1680 added lines are the new module and its tests. The existing rollout architecture was reused, not reshaped.

4.1 Segments over a linked event — the load-bearing idea

Every rollout strategy's control loop already polls ctx.runtime.shutdown_event.is_set() to know when to stop. So instead of teaching strategies about interactivity, we swap in a smarter event:

class LinkedEvent(Event):
    """is_set() reflects the local flag OR a parent event."""
    def is_set(self) -> bool:
        return super().is_set() or self.parent.is_set()

lerobot-rollout wraps the ProcessSignalHandler's shutdown event in a LinkedEvent when --interactive=true. The session sets the local flag to end a run segment; SIGINT / SIGTERM still arrive through the parent, so Ctrl-C behaves exactly as before.

InteractiveSession.run() then drives strategy.run(ctx) in restartable segments:

setup(ctx)  →  [idle]  →  /start → run(ctx) → /reset → [idle] → /start → run(ctx) → /stop  →  teardown(ctx)
                                                  ↑ hardware + policy stay warm throughout

Zero strategy code changed to support this. The only additions to strategies/core.py were reset_control_state() (engine + interpolator + cached-observation reset, factored out of _init_engine so a segment can restart cleanly) and making _return_to_initial_position public.

4.2 Threading model

 listener thread  ──publishes flags / strings──▶  main thread
 (stdin reader)      never touches hardware        (session loop → strategy.run → control loop)
                     never mutates policy state

The listener only ever writes threading.Event flags and a lock-guarded string. Everything that touches hardware or policy state happens on the thread that already owns it. This mirrors the existing DAgger events pattern rather than inventing a new concurrency idiom.

4.3 stdin must be read with os.read, not readline

Non-obvious and load-bearing. The first implementation used select() + stream.readline() and two tests failed: a buffered file object slurps several lines off the file descriptor in one syscall, after which select reports the drained fd as not-ready and the buffered lines are never delivered. Pasted or piped command batches got stuck. The reader now does select() + os.read(fd, 4096) + manual \n splitting, with a blocking-readline fallback for streams without a fileno() (non-POSIX, test doubles).

Also: unlike TerminalKeyListener, this reader leaves the terminal in canonical mode — the operator is typing chat commands, not pressing hotkeys.

4.4 EOF means stop

A closed stdin means there is no way left to command the robot, so EOF (Ctrl-D, or an exhausted piped script) stops the session. An unexpected read error is treated the same way, for the same reason. Consequence, documented: piped scripts must hold stdin open —

(printf '/start\n'; sleep 60; printf '/stop\n') | lerobot-rollout ... --interactive=true

4.5 Commands are last-write-wins

/reset and /stop cancel a still-pending /start, so the robot never starts moving after the operator's most recent command said not to. Handlers set their intent flag first and the segment-stop event second; _run_segment clears the segment-stop flag before re-checking the intent flags. A /reset racing a /start is therefore either seen before the segment begins or ends it on its first tick.

4.6 Base strategy only (enforced by config validation)

--interactive=true with a recording strategy raises a ValueError. Two reasons: recording strategies finalize their dataset inside run() (so run() is not restartable), and their keyboard listeners contend with the command reader for the same TTY. This is a deliberate, documented limitation — not an oversight.

4.7 Log muting (Phase 1.5)

Policy, robot and control-loop logs at every level interleave with the chat prompt and destroy the typing UX. Simplest workable answer, per explicit request: mute console output for the duration of the session.

• Every logger's console StreamHandler is raised above CRITICAL — not just root, because transformers and datasets attach their own stderr handlers with propagate=False.
• warnings.simplefilter("ignore"), with warnings.filters saved and restored.
• File handlers are untouched — anyone wanting a persistent log can attach one.
• Restored in run()'s finally, before the closing log_say, so teardown logs are visible.

The obvious hazard: muting hides fatal errors. So InferenceEngine gained a failure_traceback property, RTC captures its traceback in the fatal handler, and the session prints it on failure. Do not remove that when touching the failure path.

"See both logs and prompt" — a pinned input line, prompt_toolkit-style — was deliberately deferred: it needs a new dependency and a real TUI layer.

4.8 /subtask — the engine is the broker

The pivotal call on Phase 2: skip PR #4108's PromptBroker. After Phase 1, the session already owns the stdin thread and the parser, so a broker + listener base + on-change callbacks + a new RuntimeContext field would be duplicate machinery — and callbacks firing on the listener thread are exactly the cross-thread hazard we designed against.

Instead, InferenceEngine (the ABC every backend already implements) became the thread-safe task holder:

@property
def task(self) -> str: ...              # lock-guarded read

def set_task(self, task) -> bool:       # callable from ANY thread; True if it changed
    ...

def _take_task(self) -> tuple[str, bool]:   # consumed on the INFERENCE thread;
    ...                                     # returns (task, changed) and clears the edge

/subtask is then three lines: read engine.task, call engine.set_task(text), print the transition. No new module, no new context field, no callbacks.

The flush problem, and why it got small. When the instruction changes, a chunking policy is still serving actions computed under the old one — up to chunk_size ticks of stale behavior. PR #4108 solved this by adding flush_action_queue() / _apply_pending_flush() to PreTrainedPolicy and editing 14 policy files, because its flush request arrived from a foreign thread and had to be deferred to a safe point inside select_action.

Ours already runs on the thread that calls select_action. So: one concrete method on PreTrainedPolicy and zero per-policy edits.

def drop_queued_actions(self) -> None:
    queues = getattr(self, "_queues", None)
    if isinstance(queues, dict) and ACTION in queues:
        queues[ACTION].clear()
    action_queue = getattr(self, "_action_queue", None)
    if action_queue is not None:
        action_queue.clear()

Two getattrs cover the repo's two queue idioms across all ~18 policies (_queues[ACTION]: diffusion, smolvla, tdmpc, vqbet, wall_x, xvla, multi_task_dit, vla_jepa; _action_queue: act, pi0, pi05, pi0_fast, eo1, evo1, groot, molmoact2, fastwam, lingbot_va). Policies with no queue inherit a no-op.

Why not policy.reset()? That was the first implementation, and review caught it as too blunt. For Diffusion it wipes the observation history, so the next chunk is planned from a history of the current frame repeated — a visible discontinuity mid-motion. And ACT / Diffusion / VQBeT / TDMPC don't read task at all, so they'd pay that jerk for nothing. drop_queued_actions keeps episode state and drops only what is actually stale.

RTC deliberately does not flush. Clearing its queue would leave the robot with no commands for a full inference latency (~1 s on a VLA). Instead the next chunk is generated under the new instruction and merged over the previous chunk's leftover prefix — the switch lands within one inference and the motion stays continuous. That is exactly what RTC's blending exists for. Documented per-backend in inference.mdx; no config flag, one sensible default per backend.

/reset restores the launch task on the listener thread. Subtle and worth preserving: the restore lives in _cmd_reset, not in _reset_robot (which runs later, on the main thread). Otherwise /reset followed immediately by /subtask would be ordered by service time rather than command time, and the deferred restore would silently revert the new instruction — deterministically so, for pasted or piped input. Both writers now run on the same thread, so command order wins. There is a regression test driving this through a real pipe.

5. Round 2 — the feature becomes a library API

Four follow-up asks landed together (currently uncommitted on the branch): make the components programmatic-API friendly (the priority), extend interactive to recording where cheap, simplify muting / surface errors, and settle the ssh/headless + keyboard_input question.

5.1 RolloutController — programmatic control

interactive.py bisected cleanly, so the generic control logic moved to a new rollout/controller.py:

controller = RolloutController(strategy, ctx, on_event=my_observer)
controller.serve()      # blocking loop (run it on whatever thread you like)
controller.start()      # -> bool: False when a segment is already running
controller.set_task(t)  # -> bool: re-instruct mid-run, from any thread
controller.reset()      # -> bool: True when the launch task was restored
controller.stop()
controller.task / .initial_task / .running / .failed / .failure_traceback

• No I/O of its own — no stdin, no prints, no log muting, no TTS. Every state transition that used to be a print is now a RolloutEvent (SEGMENT_STARTED, SEGMENT_ENDED, RESET_STARTED/DONE/SKIPPED, ENGINE_FAILED, STOPPED) emitted on the serve thread.
• Thread-safe by lock, not by convention. The old ordering guarantee (/subtask right after /reset must win) relied on both writes running on the single stdin thread. The controller serializes start/reset/stop/ set_task with an internal lock, so the guarantee now holds for arbitrary caller threads — the prerequisite for network/voice front-ends.
• InteractiveSession shrank to a thin adapter: stdin listener + parser + rendering + muting; each command maps 1:1 onto a controller method, and the controller is exposed as session.controller.
• Exported from lerobot.rollout: RolloutController, RolloutEvent, LinkedEvent. docs/source/inference.mdx gained a Programmatic control section with a complete embedding example.

5.2 Sentry + interactive — recording while you steer

Decision, per the agreed criteria: the /record keyboard-handoff idea is medium-to-large (listeners have no suspend/resume API and start at creation, esc handlers are hardcoded and collide, pynput captures globally while you type, and each strategy carries per-run stale flags) → rejected. But the investigation showed sentry has zero keyboard code — the config comment lumping it with the keyboard strategies was simply wrong — and its only real blocker was one line: with VideoEncodingManager(dataset) inside run() finalizes the dataset the first time run() returns, after which a restarted segment would silently truncate the finalized parquet.

So --interactive=true now supports --strategy.type=sentry:

• Finalization moved to teardown() (which already called dataset.finalize()); run() is segment-restartable. Each segment saves complete episodes plus one tail partial episode; on a failed tail save the in-flight streaming encode is cancelled and the half-mutated episode buffer is discarded (see §6, round 2).
• Frames are labeled with the live engine.task instead of a config snapshot — the writer already stores a task per frame — so /subtask changes the policy conditioning and the recorded label from the same frame onward. This also resolved the "recorded frames ignore /subtask" open item for sentry.
• episodes_since_push hoisted to instance state so upload cadence survives segments.
• dagger / highlight / episodic stay excluded: keyboard conflicts plus per-run recording state that does not survive a restart.

5.3 Muting v2 — two lines, and errors surface

The ~30-line per-handler walk became logging.disable(logging.WARNING) with the previous disable level restored afterwards. Strictly better coverage: the gate applies before handler dispatch, so it covers propagate=False library loggers and loggers created mid-session (the old snapshot missed those) — and ERROR/CRITICAL now reach the console, which the audit showed is safe: no ERROR-level emitter fires periodically in healthy operation (the periodic nuisances — slow-loop, camera hiccups — are WARNINGs and stay muted). Documented trade-off: the gate also withholds INFO/WARNING from file handlers during the session; acceptable because no default code path attaches one (only rl/actor, rl/learner, async_inference pass log_file). The warnings suppression stays (nothing calls logging.captureWarnings), and failure_traceback surfacing stays as the belt-and-suspenders for fatal engine errors.

5.4 stdin listener → lerobot/utils/stdin_input.py

The ssh/headless audit confirmed the listener was already the right design: select+os.read works over SSH (the session pty is a normal fd), from pipes, and headless — it's keyboard_input's pynput backend that needs a display server. Nothing in keyboard_input overlaps enough to reuse (1-byte cbreak hotkey decoding vs canonical-mode line assembly), so StdinCommandListener moved to a new utils module — deliberately not into keyboard_input.py, which attempts a pynput import at module load. Canonical import only: lerobot.utils.stdin_input (removed from lerobot.rollout's exports).

The move fixed a real bug the audit found: with sys.stdin is None (daemonized processes), the blocking fallback died with an uncaught AttributeError without firing on_eof — leaving a session idling with no command channel. start() now treats a missing stream as immediate EOF.

6. Bugs the adversarial reviews caught

Four multi-agent review passes were run across the phases (28 / 5 / 27 / 12 agents; findings adversarially verified before acting). The ones that mattered:

Round 2 (2 confirmed, 0 refuted):

• Controller start() race → phantom segment. start() gated on _running, but the serve loop cleared _start_requested before setting _running — a second start() landing in that window (spanning reset_control_state and the SEGMENT_STARTED emission) returned True and re-armed the flag, which nothing consumed during the segment; the robot would start again, uncommanded, when the segment later ended on its own. Fixed: the serve loop consumes the request and sets _running atomically under the control lock, and _running spans the whole startup sequence.
• Sentry poisoned episode buffer. save_episode mutates the buffer in place (pops size/task) before the fallible writes; a failed tail save left a half-mutated dict and the next segment's first add_frame crashed with KeyError('size'). Fixed: the except branch discards the buffer so add_frame recreates it.

Rounds 1–3 (Phases 1–2):

• RTC stale observation (critical). RTCInferenceEngine.reset() never cleared _obs_holder["obs"]. After /reset physically moved the robot home, the next /start computed its first chunk from the pre-reset pose — a lurch back toward where the arm used to be. Fixed by clearing the observation and adding a _reset_epoch counter so an in-flight chunk computed across a reset is discarded rather than merged. This also fixes a pre-existing DAgger staleness path.
• Muting hid fatal errors → failure_traceback capture + session print (§4.7).
• Muting scope too narrow → root-only missed transformers / datasets; warnings output bypassed logging entirely.
• Command ordering → /reset and /stop didn't cancel a pending /start (§4.5); the /reset-then-/subtask clobber (§4.8).
• Flush too heavy → policy.reset() → drop_queued_actions() (§4.8).
• Empty-task rendering → '' replaced with (none — set one with /subtask <text>).
• Silent switch → the confirmation now says "(applies from the next policy inference)", since the explanatory logs are muted.

7. Verification

After Round 2:

uv run --extra dataset pytest tests/test_interactive_rollout.py \
    tests/utils/test_stdin_input.py tests/test_rollout.py -q
    → 81 passed

pre-commit (all changed files)
    → 0 failures

Phase 1–2 numbers (still green at the time): 64 rollout/interactive tests; 223 passed / 5 skipped across tests/policies/rtc, factory, and common (confirming the shared pretrained.py change); pre-commit 0 failures.

tests/test_interactive_rollout.py covers the parser, LinkedEvent semantics, RolloutController (start/reset/stop/set_task flows, events, startup-race rejection, failure surfacing, broken observers), session flows (start / reset / restart / stop, cancel-pending-start, engine failure with traceback, natural end, EOF, and a real BaseStrategy end-to-end), muting (INFO/WARNING blocked, ERROR surfaces, pre-existing disable level restored), /subtask semantics, sentry restartability + live labels + failed-tail-save recovery, the engine task holder, the sync flush, and drop_queued_actions. tests/utils/test_stdin_input.py covers the listener (select path, batched lines, blocking fallback, EOF, handler errors, None-stdin, broken streams).

8. Extension points for Phase 3

The design was built to make /ask an additive change:

• Command table. InteractiveSession._commands is name → (handler, arg hint, help). /help and the startup banner render from it, so a new command is documented for free.
• Controller API. New front-ends (network, voice, /ask's LLM worker) call RolloutController.start/reset/stop/set_task from their own threads — the internal lock makes that safe — and observe RolloutEvents instead of scraping terminal output.
• Thread discipline. A command handler runs on the listener thread and must only call controller methods. An LLM call belongs on its own worker thread so the robot keeps moving — precisely the concurrency /ask is meant to demonstrate.
• Task holder. set_task / _take_task already give any producer a safe way to re-instruct the policy. Hierarchical task-vs-subtask semantics (per #4183 / #4234) layer on top of it rather than replacing it.

Open items, deliberately not addressed:

• dagger / highlight / episodic remain non-interactive (keyboard conflicts + per-run recording state); they also still snapshot the task label per run. Sentry is the supported recording path for interactive sessions.
• The "see logs and prompt simultaneously" TUI (pinned input line).
• Non-stdin input sources (network, voice) — now unblocked by RolloutController; #4108's pluggable-listener shape remains the reference for the transport layer.