admin/lerobot
1
0
Fork 0
mirror of https://github.com/huggingface/lerobot.git synced 2026-05-25 05:29:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(dataset): 2x faster dataloader via parallel decode, uint8 transport, and persistent workers (#3406)

Browse Source 
* feat(dataset): 2xfaster dataloader

* fix(dataset): streaming return uint8 decode

* fix(tests): adjust normalization step comparison

* fix(dataset): with threadexecutor + False default

* chore(dataset): make it a config

* fix(test): account for uint8 in training path testing
This commit is contained in:
Steven Palma
2026-04-19 00:08:22 +02:00
committed by GitHub
parent 760220d532
commit a8b72d9615
10 changed files with 78 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/policies/test_policies.py
+2
View File
@@ -196,6 +196,8 @@ def test_policy(ds_repo_id, env_name, env_kwargs, policy_name, policy_kwargs):
for key in batch:
if isinstance(batch[key], torch.Tensor):
if batch[key].dtype == torch.uint8:
batch[key] = batch[key].to(dtype=torch.float32) / 255.0
batch[key] = batch[key].to(DEVICE, non_blocking=True)
# Test updating the policy (and test that it does not mutate the batch)