Cohere releases Command A+, its most powerful large language model to date, as open-source Apache 2.0 software that runs on two H100 GPUs with 30% lower latency
Cohere co-founder Ivan Zhang highlighted efficiency and accessibility design choices.
#68clem 🤗@CLEMENTDELANGUECohere is on such a great open-source trajectory lately. Beautiful Apache 2.0 model! https://huggingface.co/CohereLabs/command-a-plus-05-2026-bf16
Nick Frosst@nickfrosstCommand A+ from @cohere is out now :) its our best model yet and its open source apache 2.0
@nickfrosst @cohere Congratulations on this milestone! Looks great 👍
Nick Frosst@nickfrosstCommand A+ from @cohere is out now :) its our best model yet and its open source apache 2.0
#155Sebastian Raschka@RASBTIt's been *almost* a bit quiet around LLM architecture releases in the past two weeks 😅
Interesting tidbit is the parallel block design. Via the Cmd-A the tech report "equivalent performance but significant improvement in throughput compared to the vanilla transformer block."
Cohere@cohereIntroducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
@eliebakouch It’s interesting. I’ve seen hidden -> wider -> hidden FFNs and hidden -> narrower -> hidden FFNs. But Command A+ seems tu use 4096 -> 4096 -> 4096 for each expert FFN, which I haven’t seen before (as far as I remember)
elie@eliebakouch@rasbt the query head x head dim being super wide is quite different from other model as well afaik
#171Aidan Gomez@AIDANGOMEZOur first fully open source Apache 2 model :)
Cohere@cohereIntroducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
#171Aidan Gomez@AIDANGOMEZNick really championed us going Apache 2 for this release and for Cohere Transcribe. Not an obvious decision and one that required many discussions. Like Nick says, I hope the model is more useful and empowering as a result.
Nick Frosst@nickfrosstCommand A+ from @cohere is out now :) its our best model yet and its open source apache 2.0
Very cool to see that cohere is continuing to use the parallel attention and MLP set up @AiEleuther introduced
Cohere@cohereIntroducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
#216Joelle Pineau@JPINEAU1Check out our latest open-source model, built for efficiency, with a focus on business use-cases, available for all.
Cohere@cohereIntroducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
#258Marc G. Bellemare@MARCGBELLEMAREWhat a week to start at Cohere!
Cohere@cohereIntroducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
@nickfrosst @cohere hell yeah! model looks great
Nick Frosst@nickfrosstCommand A+ from @cohere is out now :) its our best model yet and its open source apache 2.0
Cohere has fallen, DS-MoE shape reigns supreme
Cohere@cohereIntroducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
@nickfrosst @cohere Very cool release. Much needed in the space.
Nick Frosst@nickfrosstCommand A+ from @cohere is out now :) its our best model yet and its open source apache 2.0
#488kache@YACINEMTB@nickfrosst @cohere great work
Nick Frosst@nickfrosstCommand A+ from @cohere is out now :) its our best model yet and its open source apache 2.0
#601Nick Frosst@NICKFROSSTOpen source Command A+ model
This tech can go one of two ways. It can go the way the internet and mobile phones did - in which technological hegemony resulted in a mostly disempowering tech.
Or it can empower the people that use it.
We are working towards that second one.
Cohere@cohereIntroducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
#601Nick Frosst@NICKFROSST> be cohere > join forces with some German companies > immediately open source your best model > life is good
Maziyar PANAHI@MaziyarPanahiwait… did Cohere just release Command A+ models under Apache 2.0 for the first time ever?! 🙊 welcome to Europe! 🤗
Command A+ from @cohere is out now :) its our best model yet and its open source apache 2.0
Cohere dropped Command A+ 🔥
> 25B/219B MoE vision language model > supports 48 languages with efficient tokenizer > tool-calling/agentic + 128k context window > transformers day-0 support 🤗 free license 💗
try their demo and check the models https://huggingface.co/collections/CohereLabs/command-a-plus
merve@mervenoyannCohere dropped Command A+ 🔥 > 25B/219B MoE vision language model > supports 48 languages with efficient tokenizer > tool-calling/agentic + 128k context window > transformers day-0 support 🤗 free license 💗
#715Jay Alammar@JAYALAMMAROut today! Our most capable agentic model: - Runs on one B200 - 48 languages (including العربية, 日本語, 한국어) - Open source (Apache 2.0 ) - Multimodal: text + images - 218B Mixture-of-Experts model, 25B active parameters
Cohere@cohereIntroducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
#716elie@ELIEBAKOUCH@aidangomez congrats on the release, would love to see a tech report here, lot of interesting/unusual choices
elie@eliebakouchinteresting open model by cohere with lots of unusual architecture choices, here is a recap: > parallel transformer, so MoE and attention are computed in parallel. likely doing some kind of MLP/attention disaggregation here? > lots of query heads, query total dim is 4x hidden size > big shared expert, 4x router size > no scaling after normalization of the top k > LayerNorm instead of RMS norm > 32 layer only, no dense layer at the start
#716elie@ELIEBAKOUCH@rasbt the query head x head dim being super wide is quite different from other model as well afaik
Sebastian Raschka@rasbtIt's been *almost* a bit quiet around LLM architecture releases in the past two weeks 😅 Interesting tidbit is the parallel block design. Via the Cmd-A the tech report "equivalent performance but significant improvement in throughput compared to the vanilla transformer block."
#716elie@ELIEBAKOUCHinteresting open model by cohere with lots of unusual architecture choices, here is a recap:
> parallel transformer, so MoE and attention are computed in parallel. likely doing some kind of MLP/attention disaggregation here? > lots of query heads, query total dim is 4x hidden size > big shared expert, 4x router size > no scaling after normalization of the top k > LayerNorm instead of RMS norm > 32 layer only, no dense layer at the start
Cohere@cohereIntroducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
#716elie@ELIEBAKOUCH@torchcompiled yeah actually at training it's pretty nice as well you're right
Ethan@torchcompiled@eliebakouch Have seen a few parallelized MLP and attention cases, I wanna say one of the flux/SD models did that? My understanding is it’s a small perf hit but with proper parallleization can be a cozy inference and training speed up
#716elie@ELIEBAKOUCHalso they have a "use_parallel_embedding" flag in the config? engram next?
elie@eliebakouchinteresting open model by cohere with lots of unusual architecture choices, here is a recap: > parallel transformer, so MoE and attention are computed in parallel. likely doing some kind of MLP/attention disaggregation here? > lots of query heads, query total dim is 4x hidden size > big shared expert, 4x router size > no scaling after normalization of the top k > LayerNorm instead of RMS norm > 32 layer only, no dense layer at the start
#716elie@ELIEBAKOUCH@nrehiew_ yes it's like the "older version" of this basically, the one from palm ect..
wh@nrehiew_@eliebakouch Similar to this? This parallel arch basically halves layer count. The 1/2 weight and router stuff is different though
#716elie@ELIEBAKOUCHmore pretty version of this visualization here if you are seeing this now
Sebastian Raschka@rasbtIt's been *almost* a bit quiet around LLM architecture releases in the past two weeks 😅 Interesting tidbit is the parallel block design. Via the Cmd-A the tech report "equivalent performance but significant improvement in throughput compared to the vanilla transformer block."
@aidangomez A M A Z I N G !!!!!!
Aidan Gomez@aidangomezOur first fully open source Apache 2 model :)
@1vnzh @aidangomez real ones know ✊
Ivan Zhang@1vnzh@aidangomez transcribe erasure bro
@eliebakouch Similar to this? This parallel arch basically halves layer count. The 1/2 weight and router stuff is different though
elie@eliebakouchinteresting open model by cohere with lots of unusual architecture choices, here is a recap: > parallel transformer, so MoE and attention are computed in parallel. likely doing some kind of MLP/attention disaggregation here? > lots of query heads, query total dim is 4x hidden size > big shared expert, 4x router size > no scaling after normalization of the top k > LayerNorm instead of RMS norm > 32 layer only, no dense layer at the start
#1629stochasm@STOCHASTICCHASM4 shared experts with 8 routed experts active? so 12/132, that's crazy, i wonder why. most papers like Towards Greater Leverage would suggest 1 shared expert or minimal (i think we should decouple shared expert size anyway eventually)
also, 128 attention heads with GQA???
#1634Ivan Zhang@1VNZHWe ❤️ Open Source
Cohere@cohereIntroducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
@aidangomez transcribe erasure bro
Aidan Gomez@aidangomezOur first fully open source Apache 2 model :)
@rasbt it’s from gpt-j https://github.com/kingoflolz/mesh-transformer-jax
Sebastian Raschka@rasbtIt's been *almost* a bit quiet around LLM architecture releases in the past two weeks 😅 Interesting tidbit is the parallel block design. Via the Cmd-A the tech report "equivalent performance but significant improvement in throughput compared to the vanilla transformer block."
@eliebakouch Have seen a few parallelized MLP and attention cases, I wanna say one of the flux/SD models did that? My understanding is it’s a small perf hit but with proper parallleization can be a cozy inference and training speed up
elie@eliebakouchinteresting open model by cohere with lots of unusual architecture choices, here is a recap: > parallel transformer, so MoE and attention are computed in parallel. likely doing some kind of MLP/attention disaggregation here? > lots of query heads, query total dim is 4x hidden size > big shared expert, 4x router size > no scaling after normalization of the top k > LayerNorm instead of RMS norm > 32 layer only, no dense layer at the start