/Tech2h ago

Google DeepMind's Zachary Nado highlights a comparison arguing the Shampoo optimizer is fundamentally a variant of KFAC

Rohan Anil shared documentation comparing their mathematical curvature approximations.

2372144.9K
Zachary Nado
rohan anil#86
Original post
Zachary Nado@zacharynado#589inTech

this is the real spicy take people aren't ready for

Thomas Ahle@thomasahle

@JamesWhate89993 Wasn't shampoo just a variant of KFAC?

8:06 PM · Jun 10, 2026 · 3.6K Views
Sentiment
Sentiment building, check back later.
Cluster Engagement
Posts from X
Most Activity
Most Activity
VIEWS1.2KBOOKMARKS9LIKES18RETWEETS1
rohan anil@_arohan_

@zacharynado This question is what led to this section which naturally leads to the linalg for E-KAC which is parallel to SOAP for Shampoo. See end of the section.

Zachary Nado@zacharynado

this is the real spicy take people aren't ready for

2hViews 1.2KLikes 18Bookmarks 9
Google DeepMind's Zachary Nado highlights a comparison arguing the Shampoo optimizer is fundamentally a variant of KFAC · Digg