Commit Graph

2 Commits

Author SHA1 Message Date
Khalil Meftah d0348b1803 refactor(rewards): rewrite distributional VF with SigLIP2 + Gemma3 backbone
Replace PaliGemma-based value function with a monolithic VLM architecture
using SigLIP2-so400m (vision encoder) and Gemma3-270M (shared backbone),
matching the pi*0.6 paper.

Key changes:
- SigLIP2-so400m as vision encoder, Gemma3-270M as unified transformer
- [CLS] token readout with bidirectional prefix attention
- 2-layer MLP value head (Linear -> LayerNorm -> GELU -> Dropout -> Linear)
- Multi-camera support with per-camera validity masks
- Image preprocessing in processor (resize with pad, normalize to [-1,1])
- Freeze controls for vision encoder and language model independently
2026-07-18 19:31:03 +02:00
Khalil Meftah f8fa8ba394 test(rewards): add unit tests for distributional value function model 2026-07-03 16:55:40 +02:00