/AI6h ago

Debate Questions If Next-Token Prediction Requires Understanding Reality

7343912.5K
Original post
Greg Kamradt@GregKamradt

Do n-gram models also require an understanding of the underlying reality?

If not, which architecture level of next-word predictors does?

If so, then imo the “understanding” is inherit in the data, not the transformation of it

6:26 AM · Jun 1, 2025 · 6.7K Views
Sentiment
Sentiment building, check back later.
Cluster Engagement
Posts from X
Most Activity
Most Activity
RETWEETS1
Greg Kamradt@GregKamradt

Does a n-gram model count as a world model?

233dViews 5.9KLikes 11Bookmarks 3