KronLSTM is out, using the Kronecker Delta Rule for memory revisions
The announcement says KronLSTM uses the rule for minimally invasive memory revisions compared with the delta rule.
TLDR
A post announces KronLSTM’s release and says it uses the Kronecker Delta Rule for minimally invasive memory revisions compared with the delta rule. The author claims to have proved that the expected norm of the memory matrix is uniformly bounded and says KronLSTM is particularly well suited for long-context language modeling.
Combined views
649
1 Source, first seen ago