Surge AI Releases Chartography Benchmark For Professional Chart Understanding
Source Posts on X
the existing chart benchmarks (chartqa, charxiv, etc) are basically solved: 80-90% on bars, lines, pies. but the charts a cardiologist or a process engineer runs into are a different problem entirely and nobody was measuring them, until now.
Posts from X
2 poststhe existing chart benchmarks (chartqa, charxiv, etc) are basically solved: 80-90% on bars, lines, pies. but the charts a cardiologist or a process engineer runs into are a different problem entirely and nobody was measuring them, until now.
Today we rolled out Chartography: a benchmark for the charts professionals actually work from. Kaplan-Meier curves, contour maps, candlesticks, Sankey diagrams, Bode plots. 100 tasks, each written and reviewed by working domain experts. No frontier model reaches 45%.