Hipkittens Go Distributed with Moey

Subha, a Stanford math and CS student working with GPUs at AMD and HazyResearch, posted on X that the hipkittens have gone distributed. The post introduces Moey with a cartoon diagram showing a Mixture-of-Experts model and a gating network routing activations. The message ends with "more soon" and links to an attached image of the diagram. No further details on the architecture or hardware appear in the post itself.