mirror of
https://github.com/huggingface/lerobot.git
synced 2026-07-24 10:16:09 +00:00
Pepijn
9c82c39c7b
* feat(annotate): run lerobot-annotate on HF Jobs via --job.target Annotation needed a hand-edited launcher script (examples/annotations/run_hf_job.py) to reach a GPU: users copied it, rewrote the embedded CMD string for their dataset, and ran it with `python`. Fold that into the CLI instead, mirroring `lerobot-train`: `lerobot-annotate --job.target=h200` submits the exact command you'd run locally. - AnnotationJobConfig extends JobConfig with the annotation runtime's defaults (vllm/vllm-openai image, 2h cap) plus --job.lerobot_ref, so an unmerged branch can be exercised remotely without editing a script. - lerobot.jobs.annotate builds the pod command by replaying the user's own CLI flags (minus --job.*/--root, with --repo_id re-emitted from the config) after a setup prelude that installs lerobot on top of the vLLM image. Job monitoring, log tailing and Ctrl-C-detaches reuse the training submitter's plumbing. - Remote runs require --repo_id; a local-only dataset is pushed privately first. The generated pod command is byte-for-byte the script's old CMD. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(annotate): reject client-side config files on remote runs draccus exposes `--config_path` plus a `--<field>` config-file arg for every nested dataclass (`--vlm`, `--plan`, `--job`, ...). All name files on the client's disk, so forwarding them to the pod silently dropped whatever settings they carried. Reject them up front instead. Bare `--job` also slipped past the `--job.` prefix filter, so a `--job=cfg.yaml` holding `target: h200` would have reached the pod and had the job submit a job of its own, recursively. It is dropped from the forwarded args as well. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(jobs): share the submit-and-follow loop between both submitters `submit_annotate_to_hf` reused the leaf helpers (`_poll_until_done`, `_tail_logs`, `_pod_forwarded_args`) but duplicated the orchestration around them: ~40 of the 50 lines that spawn the poll/log threads, install the Ctrl-C-detaches handler and raise on a non-COMPLETED stage were identical in both files. Extract that into `follow_job(job_id, *, detach, success_marker=None) -> bool`, returning True when the job finished and False when we stopped watching without a verdict (detach or Ctrl-C). Training keeps its model-pushed marker by passing it in; annotation has no equivalent line (the CLI keeps working after the upload log to write the card and tag) so its completion stays stage-based. Kept in hf.py rather than a new module so every existing monkeypatch target in test_hf.py still resolves. Behaviour change: a training run whose job reaches COMPLETED without the marker matching now prints its completion line instead of returning silently. The marker was already documented as an optimisation with a stage-based fallback; the fallback just never reported success. Tests: adds annotate coverage for the non-detach path (completion and failure) — previously only ever exercised with detach=true — plus a detach short-circuit test. Both new annotate tests verified to fail under a mutation that stubs out follow_job. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>