Why do papers report post-training statistics in number of examples and not number of tokens?
@yanaiela Because a huge % of AI researchers have never done serious comparisons between their work and that of others and don’t realize that they’re fucking up.
@BlancheMinerva @yanaiela But for post training, the # of rollouts arguably matters more than the # of tokens, esp. since the model itself creates those tokens?
Users criticize AI papers for reporting post-training stats in examples rather than tokens, blaming researchers for failing to compare their work with prior studies.
Based on 1 visible X reactions from 1 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@BlancheMinerva @yanaiela But for post training, the # of rollouts arguably matters more than the # of tokens, esp. since the model itself creates those tokens?
Why do papers report post-training statistics in number of examples and not number of tokens?
Users criticize AI papers for reporting post-training stats in examples rather than tokens, blaming researchers for failing to compare their work with prior studies.
Based on 1 visible X reactions from 1 accounts; directional sample.
Ask a question below.
Published answers will appear here.
