The era of big and expensive models is over
We are successfully experimenting with techniques where big models are used as teachers to small models
In turn, the small models get smarter over time and perform any given task as well as Mythos
Positive users hail large AI models teaching small ones to match performance at lower cost as a real inflection point and powerful distillation, while some criticize it for keeping top models exclusive to special users.
No Digg Deeper questions have been answered for this story yet.
The ubiquitous use of big expensive models is coming to an end
We have successfully experimented with big models iteratively teaching small models to be better.
Over time, big models can teach small models to perform at Mythos level intelligence on specific tasks
@bindureddy Small models learning from giants feels like the real inflection point.
@bindureddy But the very best are locked for the special guests.
@bindureddy "Big models as teachers" is a powerful distillation of distillation. How does this scale with task complexity, or when a small model needs to master multiple diverse skills?
@bindureddy The first reply already said it all: “the very best are locked for special guests.” Small models for the masses, big ones for those who pay. Nothing has changed.
@bindureddy Agreed. Although smarter isn’t the right measure for small models
@bindureddy Maybe the split is not big vs small. It’s teacher vs worker. Big models are useful when the task space is still being discovered. Small models win once the pattern is stable enough to compress.
@bindureddy @myogiadityanath @narendramodi are responsible for insulting lakhs of kar sevaks & every Hindu, for not protecting Lord Ram donations . @VHPDigital @RSSorg workers looted the money. Shame. Now shouting & blaming by @myogiadityanath have no meaning. @aajtak @htTweets @BJP4India
@bindureddy Yes that's the right direction ⬆️