Nathan Godey @COLM 🌉

@nthngdy

Post-doc at Cornell Tech

following
followers

Copy-Paste Language Models aim to improve token copying

The announcement claims gains in downstream accuracy, long-context performance and retrieval-augmented generation memory efficiency at under 3% overhead.

8 Sources, 3d ago, updated 3d ago

Shared memory is claimed to improve looped Transformers while cutting memory use

The researchers say later recursions read the first recursion’s key-value cache while keeping a short window of their own.

4 Sources, 14d ago, updated 14d ago

Softmax bottleneck paper planned for an October 8 presentation at COLM

The presenter also calls it the “gradient bottleneck” and says the talk is scheduled for 4:40 p.m.

2 Sources, 15d ago, first seen 15d ago

Matryoshka LM Suites Train Model Families in One Run

New approach trains multiple model sizes jointly inside one architecture.

10 Sources, 62d ago, first seen 62d ago

Featured in

Copy-Paste Language Models aim to improve token copying

Shared memory is claimed to improve looped Transformers while cutting memory use

Softmax bottleneck paper planned for an October 8 presentation at COLM

Matryoshka LM Suites Train Model Families in One Run

Featured in

Copy-Paste Language Models aim to improve token copying

Shared memory is claimed to improve looped Transformers while cutting memory use

Softmax bottleneck paper planned for an October 8 presentation at COLM

Matryoshka LM Suites Train Model Families in One Run