That feeling when you realize practitioners say “verifiable tasks” when they actually mean “easy tasks.”
This shortcut limits benchmarks to simple automated tests.
That feeling when you realize practitioners say “verifiable tasks” when they actually mean “easy tasks.”
Many users found the claim that verifiable tasks often mean easy benchmarks relatable to sprint planning experiences, while some dismissed the terminology as typical rebranding.

@MillionInt ngl this is too real
reminds me of every sprint planning ive ever sat through

@MillionInt Word. The real hard tasks begin when even your verifiers need to evolve with the policy.

@MillionInt Most verifiable tasks rapidly become easy if you throw machine learning at them.

@MillionInt oohhh snap.

@MillionInt Classic rebrand for stuff that doesn't actually scale. Typical.

@MillionInt real talk. if you can't verify the output in 2 seconds it's not an easy task, it's a research project

@MillionInt honestly verifiable is just a fancy word for "i can write a unit test for this in 2 seconds"

@MillionInt Verifiable just means we can check if the answer is right and easy tasks are the ones where we can check.

@MillionInt Too soon!

@MillionInt No. See: Lean.

@MillionInt

@MillionInt haha !