Announcement
N-gram count approach reportedly matches 57% more compute at 1B scale
The author says their team has been testing whether n-gram counts in read-only CPU RAM help language models.
TLDR
The team claims its COUNT approach scales to 1B and matches 57% more compute. The author says the work tests whether n-gram counts in read-only CPU RAM help language models, and notes that others broke nanoGPT speedrun records at small scale with a similar trick.
Combined views
13.5K
5 Sources, first seen ago
