AllenAI finds pure transformers excel at copying tasks while hybrid RNN models better model meaning-bearing words

@giffmana in retrospect i realized this post sounds hilariously biased which was not intentional, i was mostly quoting the original 😂

Transformers are better at copying, while RNNs are better at modeling "meaning-bearing words—the nouns, verbs, & adjectives that say what a sentence is about"

@_albertgu Interesting