Reactions from ranked influencers
3 posts
the more i dig into k3 choices, the more elegant they seem. latentmoe combines badly with strong expert imbalance (as a tiny amount of experts are overused and becomes critical noisy chokepoints with compression), which is exactly what quantile balancing fixes at the root.
the more i dig into k3 choices, the more elegant they seem. latentmoe combines badly with strong expert imbalance (as a tiny amount of experts are overused and becomes critical noisy chokepoints with compression), which is exactly what quantile balancing fixes at the root.
ok this might have led me to stumble on something cool. we’ll see.
the more i dig into k3 choices, the more elegant they seem. latentmoe combines badly with strong expert imbalance (as a tiny amount of experts are overused and becomes critical noisy chokepoints with compression), which is exactly what quantile balancing fixes at the root.