/Tech1d ago

Rohan Anil proposes pseudoinverse optimization for Modded GPT, drawing verification request from Muon creator Keller Jordan

Jordan requested the automatic run logfile to verify results.

211422015.5K
Original post
rohan anil@_arohan_#86inTech

@kellerjordan0 The right thing is to give a heads up to the library owner to see if there is any alternative options to use. So I filed a bug here: https://github.com/facebookresearch/optimizers/issues/265

Keller Jordan@kellerjordan0

@_arohan_ Any update on this?

4:21 PM · Jun 9, 2026 · 6.3K Views
Sentiment
Sentiment building, check back later.
Cluster Engagement
Posts from X
Most Activity
Most Activity
VIEWS7.3KLIKES59RETWEETS1REPLIES1
Keller Jordan@kellerjordan0

@_arohan_ Are you planning on releasing the reproducible logfile that was automatically generated by your run?

rohan anil@_arohan_

@kellerjordan0 The right thing is to give a heads up to the library owner to see if there is any alternative options to use. So I filed a bug here: https://github.com/facebookresearch/optimizers/issues/265

1dViews 7.3KLikes 59Bookmarks 5
BOOKMARKS5
rohan anil@_arohan_

@kellerjordan0 Yes I am planning to. Why would have a different impression.

Looks like the library already supports a PseudoInverse when matrices are rank deficient. Let me see if it reproduces, so we should get a clean log.

Keller Jordan@kellerjordan0

@_arohan_ Are you planning on releasing the reproducible logfile that was automatically generated by your run?

1dViews 1.9KLikes 19Bookmarks 5