SemiAnalysis also claims up to 10x better performance per dollar than GB300 NVL72 on the vLLM engine.
SemiAnalysis claims Nvidia Rubin inference running on vLLM offers 3.2x better profit per gigawatt and up to 10x better performance per dollar than GB300 NVL72. It describes vLLM as a widely used production LLM engine.
216
7 Sources, first seen ago
—
Not ranked yet
—
Not ranked yet
@SemiAnalysis_HOLY ALERT🚨: NVIDIA vLLM INFERENCE RUBIN IS 3.2x BETTER IN PROFIT💰️ PER GIGAWATT & HAS UP TO 🚀 10x BETTER PERF PER DOLLAR THAN EVEN GB300 NVL72. This is on the widely used production LLM engine called @vllm_project. We explain below.👇️(1/3)🧵
@vllm_projectvLLM now supports NVIDIA Vera Rubin. The early results show more than 7.8x the throughput of GB200 on MiniMax M3 on AgentX. @inferact, @NVIDIAAI, @RedHat_AI, and the vLLM community have been bringing vLLM up on Rubin since it was announced. Here is where things stand. 🧵 1/5
@inferactvLLM on NVIDIA Vera Rubin NVL72 delivers more than 7.8x the throughput of GB200 on MiniMax M3 at matched interactivity, measured on AgentX. Inferact’s team has been co-leading this effort since Rubin was announced. We have integrated Rubin optimized MSA prefill kernel and tuned AgentX performance with locality aware optimizations targeted for Rubin GPU. These are early results. 🧵 1/3@SemiAnalysis_HOLY ALERT🚨: NVIDIA vLLM INFERENCE RUBIN IS 3.2x BETTER IN PROFIT💰️ PER GIGAWATT & HAS UP TO 🚀 10x BETTER PERF PER DOLLAR THAN EVEN GB300 NVL72. This is on the widely used production LLM engine called @vllm_project. We explain below.👇️(1/3)🧵@vllm_projectvLLM now supports NVIDIA Vera Rubin. The early results show more than 7.8x the throughput of GB200 on MiniMax M3 on AgentX. @inferact, @NVIDIAAI, @RedHat_AI, and the vLLM community have been bringing vLLM up on Rubin since it was announced. Here is where things stand. 🧵 1/5@inferactvLLM on NVIDIA Vera Rubin NVL72 delivers more than 7.8x the throughput of GB200 on MiniMax M3 at matched interactivity, measured on AgentX. Inferact’s team has been co-leading this effort since Rubin was announced. We have integrated Rubin optimized MSA prefill kernel and tuned AgentX performance with locality aware optimizations targeted for Rubin GPU. These are early results. 🧵 1/3@SemiAnalysis_HOLY ALERT🚨: NVIDIA vLLM INFERENCE RUBIN IS 3.2x BETTER IN PROFIT💰️ PER GIGAWATT & HAS UP TO 🚀 10x BETTER PERF PER DOLLAR THAN EVEN GB300 NVL72. This is on the widely used production LLM engine called @vllm_project. We explain below.👇️(1/3)🧵@vllm_projectvLLM now supports NVIDIA Vera Rubin. The early results show more than 7.8x the throughput of GB200 on MiniMax M3 on AgentX. @inferact, @NVIDIAAI, @RedHat_AI, and the vLLM community have been bringing vLLM up on Rubin since it was announced. Here is where things stand. 🧵 1/5@inferactvLLM on NVIDIA Vera Rubin NVL72 delivers more than 7.8x the throughput of GB200 on MiniMax M3 at matched interactivity, measured on AgentX. Inferact’s team has been co-leading this effort since Rubin was announced. We have integrated Rubin optimized MSA prefill kernel and tuned AgentX performance with locality aware optimizations targeted for Rubin GPU. These are early results. 🧵 1/3@SemiAnalysis_HOLY ALERT🚨: NVIDIA vLLM INFERENCE RUBIN IS 3.2x BETTER IN PROFIT💰️ PER GIGAWATT & HAS UP TO 🚀 10x BETTER PERF PER DOLLAR THAN EVEN GB300 NVL72. This is on the widely used production LLM engine called @vllm_project. We explain below.👇️(1/3)🧵@vllm_projectvLLM now supports NVIDIA Vera Rubin. The early results show more than 7.8x the throughput of GB200 on MiniMax M3 on AgentX. @inferact, @NVIDIAAI, @RedHat_AI, and the vLLM community have been bringing vLLM up on Rubin since it was announced. Here is where things stand. 🧵 1/5@inferactvLLM on NVIDIA Vera Rubin NVL72 delivers more than 7.8x the throughput of GB200 on MiniMax M3 at matched interactivity, measured on AgentX. Inferact’s team has been co-leading this effort since Rubin was announced. We have integrated Rubin optimized MSA prefill kernel and tuned AgentX performance with locality aware optimizations targeted for Rubin GPU. These are early results. 🧵 1/3
