NVIDIA Releases TensorRT Model Connect in Public Preview
Tool converts supported Hugging Face models to TensorRT inference in two commands without ONNX export.
NVIDIA AI posted that the company released TensorRT Model Connect in public preview. The account stated users can take a supported Hugging Face model to end-to-end TensorRT inference in two commands with no intermediate ONNX export and that the resulting bundle runs through native C++ APIs. NVIDIA also said the project was built with OpenAI Codex agents with humans directing and reviewing the work. Tianqi Chen separately noted first-class support for TVM-FFI in the release, enabling custom DSL and agent-generated kernels inside TensorRT inference.
Combined views
96.2K
2 posts, first seen 11d ago
Reactions
2 postsNA 
NVIDIA AI@NVIDIAAIWe just released TensorRT Model Connect in Public Preview. You can take a supported @huggingface model to end-to-end TensorRT inference in just two commands. No intermediate ONNX export, and the resulting bundle can run through native C++ APIs. We also built the entire project…
- likes: 522
- replies: 24
- bookmarks: 186
- reposts: 64
TC Tianqi Chen@tqchenmlOne thing that I am really excited about this TensorRT release is the first-class support for TVM-FFI, so we can bring custom DSL and agent generated kernels to TensorRT inference
- likes: 80
- replies: 1
- bookmarks: 32
- reposts: 15