Commit Graph

12 Commits

Author SHA1 Message Date
Nicolas Rabault fe6c0c7db1 fix(jobs): handle str-typed job stage from huggingface_hub
inspect_job's status.stage is an enum (with .value) in some
huggingface_hub versions and a plain str in others. The poller
assumed the enum shape, raising "'str' object has no attribute
'value'" on resume for users on the str-returning version.

Read it via getattr(..., "value", ...) so both shapes work, and
parametrize the poll test over enum and str stages so the str case
is actually exercised (the old mock only ever simulated the enum).
2026-06-29 09:36:54 +02:00
Nicolas Rabault 7f781b73e7 fix(jobs): drop --dataset.root on resume + restore keyboard-control docs
Address the latest Claude review on #3856:

- _build_resume_job no longer forwards --dataset.root to the pod (a
  host-local path it can't read); the fresh-run path already nulls it in
  build_remote_config_file, so this makes resume consistent. Add a unit
  test for _pod_forwarded_args covering the drop in both flag forms.
- Restore the display-independent keyboard-control docs (n/r/q letter
  equivalents + X11/Wayland/headless Tip) in il_robots.mdx that this
  branch was stale on relative to main (#3875).
2026-06-29 09:36:54 +02:00
Nicolas Rabault 77e4ea2afa fix(jobs): default remote job timeout to 2d instead of the platform default
HF Jobs applies its own short 30-minute timeout when none is sent, which
silently kills long training runs. Pass an explicit, generous 2d cap by
default; users can still override --job.timeout to fail fast or extend it.
2026-06-29 09:36:54 +02:00
Nicolas Rabault b27411c7f2 fix(jobs): hoist ensure_dataset_available to a module-level import
Addresses Caroline's review comment on PR #3856: the local import of
ensure_dataset_available inside submit_to_hf was vestigial. dataset.py
does not import hf.py, so there is no circular-import risk and no extra
load cost (its heavy deps stay lazy), so make it a top-level import.
2026-06-29 09:36:53 +02:00
Nicolas Rabault b34b3da3c0 fix(jobs): address claude review findings on remote training
Resolve the claude[bot] review on #3856:

- Reject reward-model training under --job.target with a clear error instead
  of crashing on a None policy inside build_remote_config_file.
- Support --policy.path remote runs: validate() no longer requires repo_id for
  remote runs (it is auto-generated in submit_to_hf), and repo_id/push_to_hub
  are now set after validate() resolves the policy.
- Narrow the bare `except Exception` in _tail_logs/_poll_until_done to
  (OSError, httpx.HTTPError) so programming errors surface instead of being
  silently retried or counted as job failures.
- Install the SIGINT detach handler only on the main thread.
- Generate model repo timestamps in UTC.
2026-06-29 09:36:53 +02:00
Nicolas Rabault 5ff3b25c39 test(jobs): skip dataset import guard in base-deps test
The fast test env installs base deps only, so require_package('datasets')
raised ImportError before the mocked lerobot.datasets import was reached.
Monkeypatch the guard to a no-op so the unit test exercises the upload logic.
2026-06-29 09:36:53 +02:00
Nicolas Rabault 3a05512ca2 refactor(jobs): use HF_LEROBOT_HOME constant for the local cache root
Resolve the local dataset cache via lerobot.utils.constants.HF_LEROBOT_HOME
instead of re-reading the env var by hand, dropping the os/Path imports.
Tests now patch the imported constant and assert on a stable message
substring (the previous "neither" match only passed by accident, matching
the test name embedded in the pytest tmp_path).
2026-06-29 09:36:52 +02:00
Nicolas Rabault ac67e64ed2 refactor(jobs): check dataset presence with api.repo_exists
Replace the dataset_info try/except RepositoryNotFoundError dance with a
direct api.repo_exists(repo_id, repo_type="dataset") call, dropping the
httpx/RepositoryNotFoundError test scaffolding.
2026-06-29 09:36:52 +02:00
Nicolas Rabault 62badcba4e refactor(configs): export JobConfig from the configs package
Re-export JobConfig in lerobot/configs/__init__.py so external callers
import it as `from lerobot.configs import JobConfig`, matching the other
config classes. Adapt the train script and test imports.
2026-06-29 09:36:52 +02:00
Nicolas Rabault b07a73f781 refactor(jobs): hoist huggingface_hub imports to module level in hf.py
huggingface_hub is a core dependency, so the per-function dynamic imports
had no lazy-loading rationale. Move them to a single module-level import
and update test monkeypatch targets to lerobot.jobs.hf.* accordingly.
2026-06-29 09:36:52 +02:00
Nicolas Rabault 607a8a6b68 feat(train): run training remotely on HF Jobs via --job.target
When --job.target names a GPU flavor, train() dispatches to lerobot.jobs.submit_to_hf
instead of training locally: it authenticates, ensures the dataset is on the Hub
(pushing a local-only one privately), serializes a pod-compatible train_config.json
(strips client-only fields, points at the model repo), submits via HfApi.run_job
with HF_TOKEN/WANDB_API_KEY secrets, then streams logs and finishes when the model
is pushed. Wires push_checkpoint_to_hub into the training loop behind
save_checkpoint_to_hub, and tags jobs/datasets/model with 'lerobot' + --job.tags.
2026-06-29 09:36:51 +02:00
Nicolas Rabault 870d71aeb0 feat(train): add JobConfig group, save_checkpoint_to_hub flag, Hub checkpoint helper
Introduce a JobConfig draccus group on TrainPipelineConfig (--job.target/image/
timeout/detach/tags) whose is_remote property gates remote dispatch, plus a
save_checkpoint_to_hub flag and validation. Add push_checkpoint_to_hub(), which
uploads a saved checkpoint directory to the model repo under checkpoints/<step>/
and creates the repo idempotently (private propagates from policy.private).
2026-06-29 09:36:51 +02:00