I think that Awni leaving was a wake up call and Ternus realizes the value of MLX to Apple. You also have great people like @ivanfioravanti and @Prince_Canuma for example doing great things from the outside on MLX.
macOS is a great base on on-device AI of all flavors. iPhone and iPad are great for ML and small models. I think no one else (even google, who is more focused on cloud) has their act together on small model on-device AI. Large model on-device AI you've got a ton of action from AMD, NVIDIA, etc. The good thing is MLX isn't the only story, you can make your own tooling on top the OS using Metal if you want.
I'm VERY pro Apple for mobile/small/medium on-device AI. I'm somewhat pro Apple for large model on-device AI deployment, I think that's where the risk and maybe lack of engineering commitment lies. Ultimately AirPods make more money for Apple than Macs, and ultimately it gets resourced based on that. If Apple sells 10,000 Mac Studio AI clusters that's not really a meaningful amount of money to Apple as an entire corporation.
There are a large number of people that care internally, and I think from WWDC you can see that the plumbing (Core AI, the enhancements to running local models n the OS, etc) is getting better.
The advantage for Apple is they almost accidentally made iPhone and Mac GREAT for third party AI, while first-party AI is kind of meh. So now most of the AI third parties develop for Apple first. Great support for third party AI, not a threat to their business.
Apple has the silicon, the devices, and the distribution. They were in the best position to own local AI. Billions of capable devices are already in people’s hands. Instead, the most useful AI experiences still depend on someone else’s cloud. Feels like they’re leaving a massive opportunity on the table.
@grok why do you think Apple did not pursue this?
@hnshah They have good- enough CPU/GPUs, and they probably will eventually do. But they are too slowww.
