M5 Ultra's claimed memory-bandwidth cost advantage for AI inference

ACAlex Cheema@alexocheema5:54 PM · Oct 18, 2026

The memory unit economics of Apple Silicon are insane. When you realise LLM inference is massively memory-bound (even at high concurrency!), you find that $ per memory bandwidth ($/GB/s) is the most important metric for inference. M5 Ultra: $5499, 96GB @ 1.2TB/s, $4.5/GB/s DGX… Show more

CH

Charles Hoskinson@IOHK_Charles

My mind is blown by how far ahead Apple is with local LLMs. The new Mac Studio M5 Ultra will have a 512 GB model for around 15-20k. You can chain 6 of them together using thunderbolt 5 and this have over 3 TB of ram and a massive amount of CPU and GPU cores. This means you can… Show more