admin/lerobot
1
0
Fork 0
mirror of https://github.com/huggingface/lerobot.git synced 2026-05-22 03:59:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(evaluation): remove benchmark table, rename section header

Browse Source 
Made-with: Cursor
This commit is contained in:
Pepijn Kooijmans
2026-04-07 14:49:22 +02:00
committed by Pepijn
parent 2c32c04cca
commit 43abbccbf6
1 changed files with 1 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/source/evaluation.mdx
+1 -10
View File
@@ -98,15 +98,6 @@ For multi-task benchmarks (e.g. LIBERO with 10 tasks), environments are wrapped
| Out of GPU memory | Decrease `batch_size`, or use `--policy.use_amp=true` | | Out of GPU memory | Decrease `batch_size`, or use `--policy.use_amp=true` |
| Debugging / single-stepping | `--eval.batch_size=1 --eval.use_async_envs=false` | | Debugging / single-stepping | `--eval.batch_size=1 --eval.use_async_envs=false` |
### Benchmarks
Measured with `pepijn223/smolvla_libero` on `libero_spatial` (10 tasks, 100 episodes total):
| Configuration | Wall time | GPU util |
|---|---|---|
| `batch_size=1` (sync) | ~400s | 0–8% |
| `batch_size=10` (async) | ~189s | 0–99% |
## Output ## Output
Results are written to `output_dir` (default: `outputs/eval/<date>/<time>_<job_name>/`): Results are written to `output_dir` (default: `outputs/eval/<date>/<time>_<job_name>/`):
@@ -149,7 +140,7 @@ lerobot-eval \
--eval.n_episodes=10 --eval.n_episodes=10
``` ```
## Programmatic usage ## API usage
You can call the eval functions directly from Python: You can call the eval functions directly from Python: