Together AI Launches Next-Generation Inference Platform
Platform rebuilt from ground up for production use of open and custom models.
Together AI released a rewritten inference platform informed by serving over 400 trillion tokens monthly. The update enables testing changes on live traffic with zero user impact, swapping models behind a stable endpoint, autoscaling based on TTFT, latency, and throughput signals, and using pre-optimized deployment profiles. It supports full control for running open models in mission-critical production environments. The platform is now live following feedback from AI-native companies and enterprises seeking reliable deployment of custom models.
Combined views
17.7K
5 posts, first seen 1d ago
Reactions from ranked influencers
5 posts
The next generation of our inference platform is here, shaped by what we've learned serving over 400 trillion tokens a month. Run open models in production with full control, and test every change on live traffic before users see it.
What's new: -> Test on real traffic, zero user impact -> Swap models behind one stable endpoint -> Autoscale on real signals: TTFT, latency, throughput -> Pre-optimized deployment profiles
Excited to share our new inference platform. We rewrote it from the ground up based on learnings from how ai natives and enterprises want to use open and custom models for mission critical production uses. It’s live and available for use!
The next generation of our inference platform is here, shaped by what we've learned serving over 400 trillion tokens a month. Run open models in production with full control, and test every change on live traffic before users see it.
Lux family co @togethercompute
Excited to share our new inference platform. We rewrote it from the ground up based on learnings from how ai natives and enterprises want to use open and custom models for mission critical production uses. It’s live and available for use! https://twitter.com/togethercompute/status/2080351860213571780
Excited to share our new inference platform. We rewrote it from the ground up based on learnings from how ai natives and enterprises want to use open and custom models for mission critical production uses. It’s live and available for use!
The next generation of our inference platform is here, shaped by what we've learned serving over 400 trillion tokens a month. Run open models in production with full control, and test every change on live traffic before users see it.