MIT Co-Author Promises LSTM Comparisons for RNN Pretraining Paper
Phillip Isola responds to Christopher Manning's praise of the arXiv paper.
Phillip Isola, an MIT associate professor, replied to Stanford professor Christopher Manning. Manning had called the paper Pretraining Recurrent Networks without Recurrence a great work by Akarsh Kumar and Isola. He noted it uses a transformer teacher to learn predictive state representations and a supervised memory transition function. Isola thanked Manning for sharing and the comments. He stated the authors are preparing comparisons to LSTMs along with analysis of the GRU failure and expect to post an update soon.
Combined views
79.6K
2 posts, first seen 2d ago
MIT Co-Author Promises LSTM Comparisons for RNN Pretraining Paper
Phillip Isola responds to Christopher Manning's praise of the arXiv paper.
Phillip Isola, an MIT associate professor, replied to Stanford professor Christopher Manning. Manning had called the paper Pretraining Recurrent Networks without Recurrence a great work by Akarsh Kumar and Isola. He noted it uses a transformer teacher to learn predictive state representations and a supervised memory transition function. Isola thanked Manning for sharing and the comments. He stated the authors are preparing comparisons to LSTMs along with analysis of the GRU failure and expect to post an update soon.
Reactions
2 postsCM 
Christopher Manning@chrmanningPretraining Recurrent Networks without Recurrence by @akarshkumar0101 & @phillip_isola is a great paper! It makes an end-run around the problems of RNNs via a transformer teacher to learn good predictive state representations & supervised learning of a memory transition…
- likes: 725
- replies: 12
- bookmarks: 651
- reposts: 88
PI Phillip Isola@phillip_isola@chrmanning Thakns for sharing and the comments! We are working on comparisons to LSTMs and analysis of the GRU failure, hopefully will update soon!
- likes: 1
- replies: 1
- bookmarks: 0
- reposts: 0
