Reaction
The missing data commons for open-model posttraining
A user argues that posttraining consumes tasks rather than documents, leaving it without pretraining's internet commons.
TLDR
A user asks where the next generation of open models will get its data, arguing that the internet supplied a commons for pretraining but posttraining consumes tasks, not documents, and has no comparable commons yet. Another user agrees that reinforcement-learning data is an issue for open development and notes that companies have contributed to open-source projects they used heavily, while saying the situations aren't necessarily similar.
Combined views
559
1 Source, first seen ago