Positive users endorse stressing infrastructure needs before AI hallucination benchmarks as useful, while negative users highlight frustrations with web APIs, harnesses, and limits.
Based on 3 visible X reactions from 2 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@xeophon yeah that’s for sure useful. for our benchmarks/evals, we’ve been just playing an ad-hoc whack-a-mole game (which i think is fine for the current level of capabilities, but won’t be fine for much more capable agents).
@maksym_andr well and also not all harnesses/web APIs allow allow/blocklists and/or have some limits web stuff is so annoying to get working 😭
@xeophon you’re absolutely right!
@maksym_andr need infra first before you can think of building on top of it!
@xeophon yeah that’s for sure useful. for our benchmarks/evals, we’ve been just playing an ad-hoc whack-a-mole game (which i think is fine for the current level of capabilities, but won’t be fine for much more capable agents).
@maksym_andr well and also not all harnesses/web APIs allow allow/blocklists and/or have some limits web stuff is so annoying to get working 😭
Positive users endorse stressing infrastructure needs before AI hallucination benchmarks as useful, while negative users highlight frustrations with web APIs, harnesses, and limits.
Based on 3 visible X reactions from 2 accounts; directional sample.
Ask a question below.
Published answers will appear here.