mirror of
https://github.com/huggingface/lerobot.git
synced 2026-05-17 01:30:14 +00:00
fix(inference): gibberish detector catches long repetition collapse
The ``_looks_like_gibberish`` low-unique-token check was gated on ``len(stripped) < 80``, so an LM head that loops an n-gram for the whole 256-token budget — "the arm the arm … the the the the" — sailed straight through (``gibberish:0`` in the panel) and the garbage subtask got accepted and fed to the action expert. Added a length-independent check: ``>= 8 tokens`` but unique-token count ``<= max(3, tokens // 10)`` ⇒ repetition collapse. Now the runtime rejects the looped output and keeps the previous (real) subtask instead of propagating nonsense. This is a guard, not a cure — the underlying issue is the LM head on the current checkpoint being undertrained / collapsed; re- annotate with the short prompts and train longer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -683,12 +683,19 @@ def _looks_like_gibberish(text: str) -> bool:
|
||||
for marker in ("Assistant", "User", "Ass "):
|
||||
if marker in cleaned and len(cleaned.split()) < 4:
|
||||
return True
|
||||
# Too few unique alphabetic tokens — model stuck on ``the`` or
|
||||
# similar memorised single-token continuations.
|
||||
tokens = [t for t in cleaned.split() if any(c.isalpha() for c in t)]
|
||||
unique_alpha = {t.lower() for t in tokens}
|
||||
# Short degenerate output — model stuck on ``the`` or a couple of
|
||||
# memorised single-token continuations.
|
||||
if len(unique_alpha) < 3 and len(stripped) < 80:
|
||||
return True
|
||||
# Long repetition collapse — the LM head loops an n-gram for the
|
||||
# whole generation budget ("the arm the arm … the the the the").
|
||||
# Length-independent: many tokens but a tiny unique ratio. The
|
||||
# earlier ``< 80`` check missed these because the looped string
|
||||
# blows well past 80 chars.
|
||||
if len(tokens) >= 8 and len(unique_alpha) <= max(3, len(tokens) // 10):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user