Researcher Pushes Back On Information Bottleneck Framing For Residual Streams

For example, in our attention-sinks work we showed which kinds of tasks benefit from intermediate vs. final-layer representations (embedding/compression-heavy tasks vs. generation). That task-dependence is the missing axis here

Anyway, great work @xuanmingzhangai Go to read it!

Entropy is not information. The whole method reads predictive entropy off the logit lens and treats the valley as the "representational zenith." But an approximation of the entropy of the output distribution ≠ the mutual information that the IB story actually invokes. The real question, and it's a very hard one, is how to measure the information.

@ziv_ravid Sincerely thanks for the reading and all the valuable thoughts! Looking forward to have exciting conversations or cooperations in the future!

@xuanmingzhangai Thabk you!