AIvLLM adds GPU video decoding with PyNvVideoCodec
The vLLM project reports 2x-plus throughput on eight H100 GPUs after moving video decoding off the CPU, and says the CPU bottleneck was eliminated.
TLDR
vLLM says it has integrated PyNvVideoCodec to move video decoding from the CPU to the GPU’s NVDEC decoder. The project reports 2x-plus throughput on eight H100 GPUs, with the CPU bottleneck eliminated. It says the integration ships with CUDA vLLM releases and highlights video captioning at scale as a use case.
vLLM adds GPU video decoding with PyNvVideoCodec
The vLLM project reports 2x-plus throughput on eight H100 GPUs after moving video decoding off the CPU, and says the CPU bottleneck was eliminated.
TLDR
vLLM says it has integrated PyNvVideoCodec to move video decoding from the CPU to the GPU’s NVDEC decoder. The project reports 2x-plus throughput on eight H100 GPUs, with the CPU bottleneck eliminated. It says the integration ships with CUDA vLLM releases and highlights video captioning at scale as a use case.