Full-Weight VLA Fine-Tuning Without an fp32 Master Copy Silently Erases Most Updates

What happened

arXiv:2610.31859v1 Announce Type: new Abstract: When a vision-language-action (VLA) model is fine-tuned (further training of an existing model for a narrower job) with its trainable weights in bfloat16 (bf16) and no fp32 master copy, AdamW writes each update straight into bf16, silently rounding away any update smaller than 1/512 of a weight. On LIBERO-Object, with rules fixed in advance, an fp32 copy raises success over LeRobot's storage from 4.3% to 49.3% for SmolVLA and from 5.1% to 80.9% for $\pi_{0.5}$ at a ten-times lower learning rate, where both models are undertrained, and from 89.0% to 97.7% for $\pi_{0.5}$ at its default rate.

For SmolVLA at its default rate it adds 10.2 points with open-loop 50-action chunks but only 4.2 when re-planning every 10 actions. In 9 of 12 open VLA codebases a default or documented recipe trains this way, 4 on full weight matrices, including LeRobot's SmolVLA default and its $\pi_0$/$\pi_{0.5}$ guides.

On the Hugging Face Hub, 99% of sampled full-weight SmolVLA fine-tunes store the action expert in bf16, and 84% of recent $\pi_{0.5}$ fine-tunes with a training configuration set the bf16 flag. The fix is to keep the trainable weights in fp32 during training, a one-line change for SmolVLA in LeRobot. arXiv is committed to these values and only works with partners that adhere to them.