Users mocked the idea of blocking attention gradients into past tokens by calling it the transformer with no transformer.
Based on 1 visible X reactions from 5 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@willdepue > preventing attention gradients into past tokens The transformer with no transformer
Maybe, but my guess is the optimization pressure just finds another communication channel. If a shared workspace is genuinely useful for multi-step reasoning, I’d expect it to re-emerge through residual streams or other circuits even if you block one path. Let’s do a paper on it?
can't you prevent J-space from forming just by preventing attention gradients into past tokens? i assume that is the major reason why models do lookahead computation, without it would be really hard to form from just circuit sharing
@willdepue > preventing attention gradients into past tokens The transformer with no transformer
Maybe, but my guess is the optimization pressure just finds another communication channel. If a shared workspace is genuinely useful for multi-step reasoning, I’d expect it to re-emerge through residual streams or other circuits even if you block one path. Let’s do a paper on it?
can't you prevent J-space from forming just by preventing attention gradients into past tokens? i assume that is the major reason why models do lookahead computation, without it would be really hard to form from just circuit sharing
Users mocked the idea of blocking attention gradients into past tokens by calling it the transformer with no transformer.
Based on 1 visible X reactions from 5 accounts; directional sample.
Ask a question below.
Published answers will appear here.