very nice tech report from @AntLingAGI, they changed the arch of their previous 1T model to make it more efficient (from full GQA -> 7:1 lightning attention:MLA) and better at agentic tasks with 10T tokens of continual pre-training. also a big focus on reasoning efficiency!
@AntLingAGI tech report: https://arxiv.org/abs/2606.15079 they also released the base model!
very nice tech report from @AntLingAGI, they changed the arch of their previous 1T model to make it more efficient (from full GQA -> 7:1 lightning attention:MLA) and better at agentic tasks with 10T tokens of continual pre-training. also a big focus on reasoning efficiency!
No Digg Deeper questions have been answered for this story yet.