KimptonAI says results pages now show each model’s task outcomes alongside scores and their intervals.
KimptonAI says EvalRouter now charts every benchmark run to show how models with the same score can fail different tasks. Results pages also compare scores against cost and latency, draw a frontier, and show every model on every benchmark and each model’s profile across benchmarks.
5.2K
2 Sources, first seen ago
—
Not ranked yet
—
Not ranked yet
@KimptonAIEvalRouter now charts every benchmark run. Two models can post the same score and fail completely different tasks. A single number cannot show that. Each results page now opens with: - the outcome of every task, model by model - each score with its interval - score against cost and latency, with the frontier drawn - every model on every benchmark - each model's profile across benchmarks
@ycombinatorRT @KimptonAI: EvalRouter now charts every benchmark run. Two models can post the same score and fail completely different tasks. A single…@KimptonAIEvalRouter now charts every benchmark run. Two models can post the same score and fail completely different tasks. A single number cannot show that. Each results page now opens with: - the outcome of every task, model by model - each score with its interval - score against cost and latency, with the frontier drawn - every model on every benchmark - each model's profile across benchmarks@ycombinatorRT @KimptonAI: EvalRouter now charts every benchmark run. Two models can post the same score and fail completely different tasks. A single…@KimptonAIEvalRouter now charts every benchmark run. Two models can post the same score and fail completely different tasks. A single number cannot show that. Each results page now opens with: - the outcome of every task, model by model - each score with its interval - score against cost and latency, with the frontier drawn - every model on every benchmark - each model's profile across benchmarks@ycombinatorRT @KimptonAI: EvalRouter now charts every benchmark run. Two models can post the same score and fail completely different tasks. A single…@KimptonAIEvalRouter now charts every benchmark run. Two models can post the same score and fail completely different tasks. A single number cannot show that. Each results page now opens with: - the outcome of every task, model by model - each score with its interval - score against cost and latency, with the frontier drawn - every model on every benchmark - each model's profile across benchmarks@ycombinatorRT @KimptonAI: EvalRouter now charts every benchmark run. Two models can post the same score and fail completely different tasks. A single…
