Alessio Serra

@alessio__serra

pre-training @Aleph__Alpha • opinions my own

following
followers

Aleph Alpha releases open-weight Kolibri model under Apache 2.0

Aleph Alpha says the bilingual mixture-of-experts model has 78.1 billion parameters and a context option up to 1 million tokens.

40 Sources, 31d ago, updated 31d ago

EQB and LEI target global and local load balancing in mixture-of-experts training

A paper co-author reports validation on 7.5B mixture-of-experts models trained on up to 500 billion tokens, addressing expert underuse and parallel-execution efficiency.

2 Sources, 40d ago, first seen 40d ago

Featured in

Aleph Alpha releases open-weight Kolibri model under Apache 2.0

EQB and LEI target global and local load balancing in mixture-of-experts training

Featured in

Aleph Alpha releases open-weight Kolibri model under Apache 2.0

EQB and LEI target global and local load balancing in mixture-of-experts training