Users expressed gratitude for the collaborative development process behind the SPS Transformer and approval that splitting state and prediction roles moves beyond single linear streams.
Based on 2 visible X reactions from 3 accounts; directional sample.
Ask a question below.
Published answers will appear here.
This project started with a very different intent, and we had a lot of fun along the way. I'm incredibly grateful to @nthngdy for showing me the ways of pretraining and our frequent discussions, and to my amazing advisor @yoavartzi and @xkianteb for their invaluable mentoring 🤝.
@giomonea single linear stream is going away! https://x.com/michalwols/status/2076671376694321212
New paper with a nifty hypothesis and set of results about state preparation and transformers, and a super simple modification that buys a lot in pre training https://twitter.com/giomonea/status/2076699240517210535
This project started with a very different intent, and we had a lot of fun along the way. I'm incredibly grateful to @nthngdy for showing me the ways of pretraining and our frequent discussions, and to my amazing advisor @yoavartzi and @xkianteb for their invaluable mentoring 🤝.
New paper with a nifty hypothesis and set of results about state preparation and transformers, and a super simple modification that buys a lot in pre training https://twitter.com/giomonea/status/2076699240517210535
Users expressed gratitude for the collaborative development process behind the SPS Transformer and approval that splitting state and prediction roles moves beyond single linear streams.
Based on 2 visible X reactions from 3 accounts; directional sample.
Ask a question below.
Published answers will appear here.