Commit Graph

5 Commits

Author SHA1 Message Date
Pepijn Kooijmans c2aa31bd92 Add joint-sequence subtask training and collision-free FAST vocab mapping
- recipes/subtask_joint.yaml: paper-style single sequence (pi0.5 §IV-B) —
  the supervised subtask span gets text CE and conditions the FAST and
  flow losses in the same forward.
- joint_subtask_conditioning config flag rebuilds the same layout at
  inference: state on the task turn, generated subtask as a causal
  assistant turn (encode_prompt_with_targets + lang_causal_marks through
  sample_actions), in both the policy select_action path and the runtime
  adapter.
- fast_skip_tokens default 128 -> 1152 so FAST codes land below the <loc>
  range and never collide with VQA loc targets; _FAST_ACTION_VOCAB_SIZE
  tightened to the universal tokenizer's 1024 codes.
- Strip the trailing space from the 'Assistant:' generation prefill —
  SentencePiece folds the space into the first target token, so the
  space-suffixed prefill ended in a lone '▁' never seen in training.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 12:53:45 +02:00
Pepijn 2e43ca0d54 docs(pi052): describe merged training optimizations 2026-07-15 15:07:01 +02:00
Pepijn 5242e9195c fix(pi052): use base learning rate for lm head 2026-07-15 15:06:22 +02:00
Pepijn 6a89c7be45 fix(pi052): default flow loss weight to ten 2026-07-15 15:05:13 +02:00
Pepijn 6094058203 docs: add PI052 training and inference guide 2026-07-15 13:58:32 +02:00