Announcement
AI model design choices and their costs in parameters, FLOPs and KV cache
Aleph Alpha says two researchers compare open models and share some of the tooling used to design Kolibri.
TLDR
Aleph Alpha says two researchers use first principles to examine how architectural choices allocate model parameters, FLOPs and KV cache. They also compare open models and share some of the tooling used to design Kolibri.
Combined views
10.9K
2 Sources, first seen ago
