Announcement
N-gram count approach reportedly matches 57% more compute at 1B scale
The author says their team has been testing whether n-gram counts in read-only CPU RAM help language models.
TLDR
The team claims its COUNT approach scales to 1B and matches 57% more compute. The author says the work tests whether n-gram counts in read-only CPU RAM help language models, and notes that others broke nanoGPT speedrun records at small scale with a similar trick.
Combined views
1.8K
3 Sources, first seen ago
42 likes4 comments24 saves9 reposts
