Gavin Leech attributes vision model size gap to data compression
Gavin Leech noted that vision models are roughly 1000 times smaller than text models. He attributed the disparity to language's data compression properties, which support compositional semantics and abstractions at higher density. Research engineer 1a3orn linked the size difference to evaluations of Chain of Thought reasoning intelligibility. Creator rohit suggested exploring Chain of Thought performed in images instead of words to preserve both efficiency and clarity.
@1a3orn CoT in pictures but not words would be quite neat
1a3orn@1a3ornthis is a relevant consideration for projecting how Lindy intelligible CoT is likely to be
#13801a3orn@1A3ORNthis is a relevant consideration for projecting how Lindy intelligible CoT is likely to be
gavin leech (Non-Reasoning)@gleechone of the major failures of my life was being so surprised to find out that vision models were ~1000x smaller than text models. Just total failure to understand language's god-tier data compression
#1480gavin leech (Non-Reasoning)@GLEECHone of the major failures of my life was being so surprised to find out that vision models were ~1000x smaller than text models. Just total failure to understand language's god-tier data compression