/Tech1h ago

TAPA Decouples Magnitude and Angle in Positional Encoding for Better Long-Context Performance

0250154.9K

Original post

We introduce TAPA that decouples magnitude from angular contributions in position encoding. TAPA yields better OOD (long-context) performance than vanilla RoPE approach. We also provide theoretical analysis why it works.

Thanks @yusidwang and the colleagues for the great work!

Yu Wang@yusidwang

We’d like to introduce our paper on long-context positional encoding, centered on a simple principle:

9:30 AM · Jun 26, 2026 · 694 Views

Sentiment

Sentiment building, check back later.

Cluster Engagement

Digg Deeper

No Digg Deeper questions have been answered for this story yet.

ARXIV.ORGVia

#187

Posts from X

Most Activity

VIEWS3.2KBOOKMARKS8LIKES14

Yuandong Tian@tydsh

We introduce TAPA (https://arxiv.org/abs/2509.12635) that decouples magnitude from angular contributions in position encoding. TAPA yields better OOD (long-context) performance than vanilla RoPE approach. We also provide theoretical analysis why it works.

Thanks @yusidwang and the colleagues for the great work!

Yu Wang@yusidwang

We’d like to introduce our paper on long-context positional encoding, centered on a simple principle:

1h3.2K148