Rohan Pandey argues transformers underutilize residual streams which could enable speculative decoding from early layers
Ashwinee Panda noted that intermediate residual speculative decoding already exists.
Story Brief
Ashwinee Panda noted that intermediate residual speculative decoding already exists.
Source Posts on X
Cluster sources
Rohan Pandey@khoomeikTECH
twitter is talking abt logit lens, the precursor to “j-space”, again friendly reminder that the property enabling these lenses still remains completely unleveraged in architecture research one simple idea for example could be to spec dec from early layer residual stream https://twitter.com/khoomeik/status/1920620258353610896
Ashwinee Panda @ICML2026@PandaAshwinee