Report
A 1991 positional encoding reportedly used a 1/t time-decay bias
A reply says the bias encodes sequence order and is still widely used.
TLDR
A reply describes the 1991 positional encoding as using a 1/t (or 1/Δt) relationship to encode sequence order. It says human memory and many physical systems forget information following a power law, and attributes to Google the point that a 1/t relationship prioritizes immediate context over long-range dependencies.
Combined views
1 Source, first seen ago