Training Only Projector Adds Modalities to LLMs Without Regressing Skills
Reactions
2 postsRP 
Rohan Paul@rohanpaul_aiYou can teach a language model a new modality without ever touching the backbone's weights. Train only the projector between encoder and backbone, and the capabilities you already shipped cannot regress. To add a new modality to a language model, you only need to train the…
- likes: 291
- replies: 17
- bookmarks: 286
- reposts: 31
RP Rohan Paul@rohanpaul_aiYou can teach a language model a new modality without ever touching the backbone's weights. Train only the projector between encoder and backbone, and the capabilities you already shipped cannot regress. To add a new modality to a language model, you only need to train the…
- likes: 291
- replies: 17
- bookmarks: 286
- reposts: 31