Guardrails removed spam, off-topic, unclear, or duplicate replies.
Ask a question below.
Published answers will appear here.
GPT 5.6 Sol eval results on FrontierFinance We released FrontierFinance, the hardest and largest public benchmark for financial intelligence, at the same time as GPT 5.6 Sol was released by OpenAI. We just evaluated Sol, results and takeaways below: Results: GPT 5.6 Sol achieves 46.8%, lower than Fable 5’s 49.2% in quality. (Opus 4.8 comes in at 45%) But Sol wins in cost per query over Fable. Samaya’s System (light) still sits at the Pareto frontier of best quality (50.8%) with lowest cost (4x cheaper). Analysis: We share some analysis below of the types of examples found challenging by the different models. We see differences in qualitative analysis capabilities vs pulling financial data. We'll be continuing to evaluate and share results of leading models and AI systems on FrontierFinance!
GPT 5.6 Sol eval results on FrontierFinance We released FrontierFinance, the hardest and largest public benchmark for financial intelligence, at the same time as GPT 5.6 Sol was released by OpenAI. We just evaluated Sol, results and takeaways below: Results: GPT 5.6 Sol achieves 46.8%, lower than Fable 5’s 49.2% in quality. (Opus 4.8 comes in at 45%) But Sol wins in cost per query over Fable. Samaya’s System (light) still sits at the Pareto frontier of best quality (50.8%) with lowest cost (4x cheaper). Analysis: We share some analysis below of the types of examples found challenging by the different models. We see differences in qualitative analysis capabilities vs pulling financial data.
See full eval results, dataset, methodology and code at https://research.samaya.ai/benchmarks/frontier-finance/system-performance
GPT 5.6 Sol eval results on FrontierFinance We released FrontierFinance, the hardest and largest public benchmark for financial intelligence, at the same time as GPT 5.6 Sol was released by OpenAI. We just evaluated Sol, results and takeaways below: Results: GPT 5.6 Sol achieves 46.8%, lower than Fable 5’s 49.2% in quality. (Opus 4.8 comes in at 45%) But Sol wins in cost per query over Fable. Samaya’s System (light) still sits at the Pareto frontier of best quality (50.8%) with lowest cost (4x cheaper). Analysis: We share some analysis below of the types of examples found challenging by the different models. We see differences in qualitative analysis capabilities vs pulling financial data. We'll be continuing to evaluate and share results of leading models and AI systems on FrontierFinance!
GPT 5.6 Sol eval results on FrontierFinance We released FrontierFinance, the hardest and largest public benchmark for financial intelligence, at the same time as GPT 5.6 Sol was released by OpenAI. We just evaluated Sol, results and takeaways below: Results: GPT 5.6 Sol achieves 46.8%, lower than Fable 5’s 49.2% in quality. (Opus 4.8 comes in at 45%) But Sol wins in cost per query over Fable. Samaya’s System (light) still sits at the Pareto frontier of best quality (50.8%) with lowest cost (4x cheaper). Analysis: We share some analysis below of the types of examples found challenging by the different models. We see differences in qualitative analysis capabilities vs pulling financial data.
See full eval results, dataset, methodology and code at https://research.samaya.ai/benchmarks/frontier-finance/system-performance
Guardrails removed spam, off-topic, unclear, or duplicate replies.
Ask a question below.
Published answers will appear here.