Jon Barron argues chain-of-thought is standard next-token prediction, countering CMU's David Held on LLM planning

@davheld @alexandertmai @KostasPenn @GeorgiaChal @vincesitzmann @anand_bhattad Chain-of-thought is just a regular roll out of next-token predictions, just like the usual chat

@jon_barron @alexandertmai @KostasPenn @GeorgiaChal @vincesitzmann @anand_bhattad Right but it's an intermediate rollout of tokens before the final "answer" is produced. Kind of like "planning".

@davheld @alexandertmai @KostasPenn @GeorgiaChal @vincesitzmann @anand_bhattad Yeah philosophically I agree it's like a plan but critically unlike a traditional plan in AI the state space of it isn't the Cartesian product of n states, it's a greedy picking of a state n times.