Address review: shorten config comment, note spawn startup tradeoff

Per @jashshah999, mention that spawn workers re-import modules and so
add some startup time vs fork. Also trim the failure-mode dump from
the inline comment — the linked issue covers the symptoms in detail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
0o8o0-blip
2026-05-11 20:02:49 +01:00
committed by Steven Palma
parent f8fc30bfda
commit acbab791f1
+4 -7
View File
@@ -101,13 +101,10 @@ class TrainPipelineConfig(HubMixin):
batch_size: int = 8
prefetch_factor: int = 4
persistent_workers: bool = True
# DataLoader multiprocessing start method. "spawn" is the safe default on
# Linux because workers do not inherit fork-time state from the parent —
# "fork" can crash with non-fork-safe libraries that the parent has loaded
# (e.g. PyAV / torchcodec / ffmpeg) with errors like
# `multiprocessing.context.AuthenticationError: digest received was wrong`,
# `Pin memory thread exited unexpectedly`, or random worker segfaults.
# See https://github.com/huggingface/lerobot/issues/2488.
# DataLoader worker start method. "spawn" is safer than "fork" with
# non-fork-safe libs (PyAV / torchcodec / ffmpeg — see #2488), but
# adds some worker-startup time per run since workers re-import
# modules instead of inheriting parent state.
dataloader_multiprocessing_context: str = "spawn"
steps: int = 100_000
# Run policy in the simulation environment every N steps to measure reward/success (0 = disabled).