mirror of
https://github.com/huggingface/lerobot.git
synced 2026-08-08 17:39:44 +00:00
fix(g05): match released checkpoint oracle
This commit is contained in:
+21
-3
@@ -100,11 +100,29 @@ lerobot-rollout \
|
||||
CPU unit tests cover factory loading, config incompatibilities, prompt pass-through,
|
||||
LIBERO and `atomic_4` mappings, padding masks, inverse action projection, strict
|
||||
conversion diagnostics, a finite forward/backward/update, and save/reload parity.
|
||||
Numerical author-oracle parity and 50-episode LIBERO/RoboTwin evaluation require
|
||||
licensed checkpoint access and suitable CUDA hardware; no benchmark number is
|
||||
claimed by this integration until those gates run.
|
||||
The opt-in author-oracle gate compares the exact prompt tokens, every image/state/action
|
||||
tensor and mask, seeded flow loss and samples, and de-normalized environment actions
|
||||
for batch sizes one and two:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/policies/g05 tests/runtime/test_g05_adapter.py -q
|
||||
uv run ruff check src/lerobot/policies/g05 tests/policies/g05
|
||||
|
||||
PYTHONPATH=/path/to/GalaxeaVLA/src uv run python benchmarks/g05_checkpoint_parity.py \
|
||||
--author-source /path/to/GalaxeaVLA \
|
||||
--checkpoint outputs/g05-libero-lerobot \
|
||||
--batch-size 1 \
|
||||
--compare-training-loss \
|
||||
--output outputs/g05-libero-parity-b1.json
|
||||
|
||||
PYTHONPATH=/path/to/GalaxeaVLA/src uv run python benchmarks/g05_checkpoint_parity.py \
|
||||
--author-source /path/to/GalaxeaVLA \
|
||||
--checkpoint outputs/g05-libero-lerobot \
|
||||
--batch-size 2 \
|
||||
--output outputs/g05-libero-parity-b2.json
|
||||
```
|
||||
|
||||
This command still requires licensed checkpoint access and suitable CUDA hardware.
|
||||
A 50-episode LIBERO/RoboTwin success-rate comparison additionally requires the
|
||||
matching simulator, task assets, reset seeds, and author evaluator; no task-level
|
||||
benchmark number is claimed until that separate gate runs.
|
||||
|
||||
Reference in New Issue
Block a user