AI technologist @deepfates and founder Danielle Fong debate feasibility of preserving KV cache during live parameter adjustments
The discussion centers on protecting residual streams during token streaming.
Story Brief
The discussion centers on protecting residual streams during token streaming.
Source Posts on X
Cluster sources
Danielle Fong 🔆@DanielleFongTECH
@deepfates i'm not sure what the blocker is to preserving the kv cache vs the effort level to be honest. like i don't understand what the limitation is, i would have thought effort would be implemented in a way that can protect the residual stream, but maybe i'm missing something
🎭@deepfatesTECH