ML researcher @kalomaze says a KV cache bug caused transformer inference routing errors that exposed model outputs to other users

GO

Guilherme O'Tina@guilhermeotinaTECH

yeah this is the radix attention nightmare. shared prefix caches partition blocks by request ids, and crash recovery scrambles that mapping. the worst part: it generates fluent output from the wrong context, so you need a user noticing 'hey this is someone else's conversation' to catch it