Reaction
@siddkaramcheti asks whether VLA fine-tuning warning concerns pure BF16, not mixed precision
@m_wulfmeier flags BF16 trouble in VLA fine-tuning; @siddkaramcheti tentatively questions whether mixed precision is implicated.
TLDR
After @m_wulfmeier highlighted a paper about BF16 “breaking” VLA fine-tuning, @siddkaramcheti asked whether its instability argument concerns pure BF16, with AdamW master weights also in BF16, rather than traditional BF16 mixed precision through torch AMP. The reply explicitly notes that @siddkaramcheti hadn’t read the paper closely.
Combined views
5.5K
2 Sources, first seen ago
Comments