Kamal Gupta on Early Vision Token Injection Benefits

Kamal Gupta, previously at Tesla Optimus, posted that with a fixed vision-plus-text token budget, injecting vision tokens earlier in training improves results on both vision and text tasks, citing K2.5 and other works. He added that nearly all LLMs already add vision early in pre-training. The post includes a table titled Table 1: Performance comparison across different vision-text joint-training strategies, which indicates early fusion with a lower vision ratio yields better results.