A proposed test of nanoGPT improvements on less language-like data

A user proposes investigating which historical nanoGPT improvements fail to carry over when the dataset changes radically. The suggested test case is causal next-token prediction over discrete cosine transform coefficients—a less language-like basis for the task.