Report
Looped CED paper reports big cache savings—but the loops still cost compute
Researchers say Looped CED reuses the final encoder-loop cache instead of stockpiling one for every round.
TLDR
The authors report a 36.8-fold KV-cache reduction in eight-loop tests on 552B-parameter DeepSeek-V4.1-Flash, versus caching every layer and round, plus 3.6× throughput on 8K-token prompts. Their large-model tests measured serving efficiency, not quality; those weights weren't trained to loop, and quality evaluations reached only 340M parameters. As teortaxesTex cautions, a smaller cache doesn't erase the repeated encoder computation.
Combined views
4.3K
3 Sources, first seen ago
Comments