Ofir Press Shares Excerpt From Review He Says Rejected RNN Paper From EMNLP 2016

Ofir Press posted the first half of a review that he says rejected a paper on RNN language models from EMNLP 2016. The review states that tying input and output embeddings plus an extra projection matrix improved perplexity in most experiments. It criticizes the lack of justification for these modeling choices and asks whether gains would carry over to realistic tasks, noting many parameterizations can boost perplexity without clear insight.