refactor(runtime): make language runtime policy-agnostic; drop VQA viz

Set up the runtime so a second language-conditioned policy reuses the
CLI/REPL/UI instead of copying pi052's. The tick loop, REPL, panel, and
interactive CLI are now policy-independent in lerobot/runtime/; a policy
plugs in only a LanguageConditionedPolicyAdapter.

- Move repl.py, ui.py, and runtime_cli.py (-> cli.py) from
  pi052/inference/ into lerobot/runtime/. Generalize labels/titles
  (panel_label param, [runtime] prefixes).
- lerobot.runtime.cli.run(argv, *, adapter_factory, panel_label, prog)
  is the shared entry; policy loading already dispatches generically via
  the factory on cfg.type.
- lerobot-pi052-runtime is now a thin entry (scripts/lerobot_pi052_runtime.py)
  that passes PI052PolicyAdapter into run(). pi052/inference/ keeps only
  the adapter.
- Drop PI052Runtime back-compat wrapper (no consumers).
- Drop VQA visualization: delete inference/vqa.py + test_pi052_vqa_loc.py,
  remove answer_vqa/VQAResult from the Protocol + adapter, and the
  /question command + overlay paths from the CLI/REPL.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Pepijn
2026-07-02 15:12:33 +02:00
parent 4fa9578e3d
commit 171e06c6ba
15 changed files with 140 additions and 897 deletions
@@ -23,9 +23,6 @@ def test_pi052_adapter_builds_recipe_prompts_from_runtime_state():
{"role": "assistant", "content": "Previous plan:\npick then place"},
{"role": "user", "content": "wait"},
]
assert adapter.messages_for("vqa", state, user_text="where is the cup?") == [
{"role": "user", "content": "where is the cup?"}
]
def test_pi052_adapter_strips_say_markers_from_plan_text():
@@ -37,15 +34,19 @@ def test_pi052_adapter_strips_say_markers_from_plan_text():
def test_pi052_runtime_cli_smoke_does_not_load_model(monkeypatch):
from lerobot.policies.pi052.inference import runtime_cli
"""The pi052 entry wires its adapter into the generic runtime CLI."""
from lerobot.runtime import cli
from lerobot.scripts import lerobot_pi052_runtime
fake_policy = SimpleNamespace(config=SimpleNamespace(device="cpu"))
monkeypatch.setattr(
runtime_cli,
cli,
"_load_policy_and_preprocessor",
lambda policy_path, dataset_repo_id: (fake_policy, None, None, None),
)
monkeypatch.setattr(runtime_cli, "_run_repl", lambda runtime, initial_task, max_ticks: 0)
monkeypatch.setattr(cli, "_run_repl", lambda runtime, **kwargs: 0)
assert runtime_cli.main(["--policy.path=fake", "--no_robot", "--task=clean", "--max_ticks=0"]) == 0
assert (
lerobot_pi052_runtime.main(["--policy.path=fake", "--no_robot", "--task=clean", "--max_ticks=0"]) == 0
)