Aleph Alpha released Kolibri on Oct. 3, offering the English-German model’s full weights on Hugging Face under Apache 2.0 license terms. The company is pitching the model to enterprise and government customers that want to run it on their own infrastructure.
A sparse model with a long context option
Kolibri is a mixture-of-experts model, an architecture that activates only part of the network for each token. According to Aleph Alpha’s launch article, it has 78.1 billion total parameters but activates 3.46 billion per token across six of its 384 experts.
The company advertises support for contexts up to 1 million tokens. Its technical table lists 262,144 tokens as the longest trained length, and the serving instructions require an extra configuration flag for contexts beyond that point. The same table lists four reasoning settings: none, low, medium and high.
Aleph Alpha says it trained Kolibri on 768 B200 GPUs in three stages. Those included 20 trillion tokens of pretraining at a 16,384-token sequence length, 3.44 trillion tokens of mid-training at 65,536 tokens and 200 billion tokens of long-context adaptation at 262,144 tokens.
Built around local control
Aleph Alpha says its teams built the model in Germany and trained it on infrastructure in Germany and Finland. It presents that development chain, along with support for on-premise deployment, as part of Kolibri’s “sovereign” positioning for regulated industries.
Running Kolibri requires Aleph Alpha’s inference package and a compatible vLLM setup. The company provides a container image, a Python package and serving commands for reasoning and tool use, giving organizations a path to operate the model without sending internal data to a third-party inference service.
Comments