Oliver Sieberling

@osieberling

PhD student @MIT, pretraining large neural networks, rnn enjoyer

following
followers

A proposed learning method uses a growing gradient-signal cache instead of per-step linear-layer updates

A researcher says the paper derives the method from a duality between SGD and linear attention.

2 Sources, 10d ago, first seen 10d ago

Triadic linear attention proposes a three-dimensional state

A user proposes using the outer product of three vectors to move from linear attention’s matrix state to a three-dimensional one.

8 Sources, 15d ago, updated 15d ago

MIT club plans to restart its talks, with Alok speaking Thursday

An organizer says the club will host talks on various topics. Alok, scheduled for October 1, is known for a blog post and video on group theory and positional encodings.

5 Sources, 22d ago, updated 22d ago

Featured in

A proposed learning method uses a growing gradient-signal cache instead of per-step linear-layer updates

Triadic linear attention proposes a three-dimensional state

MIT club plans to restart its talks, with Alok speaking Thursday

Featured in

A proposed learning method uses a growing gradient-signal cache instead of per-step linear-layer updates

Triadic linear attention proposes a three-dimensional state

MIT club plans to restart its talks, with Alok speaking Thursday