Chamath Explains Prefill And Decode Phases In AI Inference

Chamath on all important “prefill” and “decode.” in AI compute. Prefill is compute-bound; massive parallel GPUs win, so Nvidia dominates as context grows. Decode is memory-bandwidth bound as each next token depends on scanning what’s already generated