@jbhuang0604 finally a model that actually respects open-weight principles while pushing performance. attention residual is a neat trick—less compute, better memory.
@jbhuang0604 Thanks for sharing and I clicked on your profile and realized you work at UMD. Go terps! Definitely will subscribe to your channel
LLM Infra folks are having too much fun these days
Users are excited about open-weight AI models rapidly closing the performance gap with closed systems, praising their advancements and adherence to open principles.
Based on 3 visible X reactions from 4 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@jbhuang0604 finally a model that actually respects open-weight principles while pushing performance. attention residual is a neat trick—less compute, better memory.
@jbhuang0604 Thanks for sharing and I clicked on your profile and realized you work at UMD. Go terps! Definitely will subscribe to your channel
LLM Infra folks are having too much fun these days
The progress is driven by Kimi Delta Attention and MoE.
LLM Infra folks are having too much fun these days
Incredible open-weight model! The gap between open and closed AI has nearly vanished! Interested in the tech behind it? Here are some background topics. Kimi Delta Attention (KDA): https://youtu.be/pUCWwGR5WmQ Attention Residual (AttnRes): https://youtu.be/LSHTkbnmzy4 Mixture of Experts (MoE): https://youtu.be/0QQlYR1r6pQ
Incredible open-weight model! The gap between open and closed AI has nearly vanished! Interested in the tech behind it? Here are some background topics. Kimi Delta Attention (KDA): https://youtu.be/pUCWwGR5WmQ Attention Residual (AttnRes): https://youtu.be/LSHTkbnmzy4 Mixture of Experts (MoE): https://youtu.be/0QQlYR1r6pQ
Introducing Kimi K3: Open Frontier Intelligence 🔹 2.8 Trillion Parameters, 1 Million Context, Native Multimodal 🔹 Kimi Delta Attention enables up to 6.3x faster decoding in million-token contexts 🔹 Attention Residuals deliver ~25% higher training efficiency at <2% additional cost 🔹 Built for long-horizon agentic coding and self-evolving workflows Kimi K3 is now live on on http://Kimi.com, Kimi Work, Kimi Code, and the Kimi API. Open Weights by July 27, 2026. 🔗 API: http://platform.kimi.ai 🔗 Tech blog: http://kimi.com/blog/kimi-k3
Users are excited about open-weight AI models rapidly closing the performance gap with closed systems, praising their advancements and adherence to open principles.
Based on 3 visible X reactions from 4 accounts; directional sample.
Ask a question below.
Published answers will appear here.
