The 64-vs-128-worker benchmark pair proved a per-host throughput ceiling
(~270 MiB/s) on the HF bucket path: doubling connections exactly halved
per-connection speed (4.8 -> 2.2 MiB/s) and left the aggregate flat,
while per-episode latency doubled (5.7s -> 12s) and keep-up worsened.
Steady-state demand (148 MiB/s) is well below the ceiling; the keep-up
misses come entirely from consumer stalls (refill_wait 14-19s of ~84s):
the sim blocks the training hot path on ensure_ready() for the FIFO-head
replacement while episodes take 5.7-12s to arrive.
Two fixes:
- Non-blocking replacements: EpisodeByteCache.is_ready() (all cameras
cached or futures done, no blocking) and the stream sim now swaps a
replacement only when it is already resident, deferring otherwise;
fetch capacity (~2x demand) repays the debt on later batches. A
deferred_swaps metric is reported.
- Sub-range parallel fetch (native-http): --range-subranges N splits one
camera GET into N concurrent sub-range GETs. Under a per-host ceiling
this adds no bandwidth but divides per-episode latency by ~N. Keep
workers x subranges near the ~64-connection saturation point (e.g.
--workers 16 --range-subranges 4).
Verified: sub-range span math + order-preserving concat and is_ready
semantics (unit-level, network stubbed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pepijn