Last time around Apple released a lot of information about how their AI version of Siri worked between local and cloud models, not so much this time

It is nice to have a Gemma-like model on device, but it is extremely limited unless it can call a smarter cloud model when needed.

@stevesi @tillmatthis there’s life in Apple 😅

@emollick Local but retarded

@emollick How does it determine which to use? That will be a key.

@emollick ngl the limited local model part is fine if the cloud handoff actually works

did they show latency numbers yet or just the specs?

@stevesi apple really out here playing 4d chess

@emollick The hybrid model overhead is basically a latency nightmare waiting to happen.

@emollick They mention private compute many times in the keynote which is them running models on servers they can provably show Apple engineers can't monitor the compute.

@emollick Cloud calling feels essential for practical on device AI

@emollick Local models are the cheap fast lane. The product works only when it knows when to spend cloud intelligence.

@stekkerauto @stevesi Uhhhhh 🤩