Will Depue claimed transfer in pretraining lacks meaningful effect. Jack Rae replied that cross-lingual transfer is evident though weaker than hoped five years ago, citing lower-resource languages like Welsh. Boris Power pushed back by noting strong transfer visible in the initial GPT-3 model before instruction tuning. Depue added that even similar data such as English and Welsh global news shows low-efficiency transfer, and very different domains fare worse. Peter Welinder suggested growing model sizes reduce pressure toward unified representations. Victor Taelin described models as massive neural lookup tables.