Awni Hannun#995
RDMA over Thunderbolt mentioned in WWDC Platforms State of the Union.
Proud to have played a small part in bringing this technology to the world with Apple / MLX through @exolabs.
The main characters are @angeloskath @awnihannun and Jack, who made this all happen.
Users praised Apple's RDMA over Thunderbolt highlight at WWDC for enabling impressive deep systems engineering and low-latency benefits like tensor parallelism.
For reference, RDMA over Thunderbolt is a capability on macOS that enables single digit microsecond latency clustering.
This enables tensor parallelism which makes models run faster as you cluster more devices.
@alexocheema @alexocheema any news on new Mac Studio yet?
@DngrSkipPermiss I would guess October
@alexocheema @exolabs @angeloskath @awnihannun It'd be nice if USB and thunderbolt could just share a common feature set already. I don't think anyone would disagree just how bad USB 3's naming convention became. USB 3.2 gen 2 2x2? What?
@alexocheema Any news on sparks and studios working together in exo? Congrats on those sparks and dgx station..
@alexocheema @exolabs @angeloskath @awnihannun great work, cool to see this kind of deep systems engineering making it into Apple platforms.
@alexocheema @exolabs @angeloskath @awnihannun great Insight,tensor parallelism really benefits from this level of low latency communication.
@alexocheema Impressive
@alexocheema Strong idea capturing failure modes alongside success patterns is especially valuable.
Awni Hannun#995RDMA over Thunderbolt mentioned in WWDC Platforms State of the Union.
Proud to have played a small part in bringing this technology to the world with Apple / MLX through @exolabs.
The main characters are @angeloskath @awnihannun and Jack, who made this all happen.