Guardrails removed spam, off-topic, unclear, or duplicate replies.
Ask a question below.
Published answers will appear here.
this was a joke (and i was talking about total in my previous post), but tbh i have no idea how many total (or active) parameters mythos or sol has. i think it's easy to know it's more than 1T, probably less than 10T, and in terms of active params i don't know what sparsity is optimal for frontier model (current oss frontier is around 3%)
@eliebakouch @robert_mchardy I don't think Mythos is 500B active
@eliebakouch @robert_mchardy my pre-training guy is crazy
Guardrails removed spam, off-topic, unclear, or duplicate replies.
Ask a question below.
Published answers will appear here.