ORQA working paper proposes a framework for assessing LLM knowledge across occupations
One of the paper’s authors presents ORQA as a way to tackle the difficult, costly task of evaluating large language models’ capabilities across the full range of occupations.
TLDR
Understanding LLMs’ impact on work first requires examining their capabilities across occupations, an author of ORQA argues. The author introduces the working paper as an occupation-realistic question-and-answer framework for professional knowledge, aimed at tackling that evaluation challenge at scale. The September 14 announcement includes links to the paper and a dashboard.
Combined views
9.6K
2 Sources, first seen ago