Reactions from ranked influencers
2 posts
Muse Spark 1.1 is a better model than people realize!
APEX-Agents score update for Muse Spark 1.1. When Meta first released Muse Spark 1.1, the model scored 37.1% Pass@1 on APEX-Agents, our agentic benchmark built for long-horizon professional services tasks in banking, law, and consulting. For roughly 10% of tasks, false positives in Meta's content moderation logic blocked trajectory completion, and as a result, those tasks failed and received 0-scores. We worked with @AIatMeta to tune content filters, which have since been updated on the publicly available model checkpoint. The new result: 41.9% Pass@1, placing Muse Spark 1.1 at #2 overall, behind only Claude Fable 5 (43.3%).
APEX-Agents score update for Muse Spark 1.1. When Meta first released Muse Spark 1.1, the model scored 37.1% Pass@1 on APEX-Agents, our agentic benchmark built for long-horizon professional services tasks in banking, law, and consulting. For roughly 10% of tasks, false positives in Meta's content moderation logic blocked trajectory completion, and as a result, those tasks failed and received 0-scores. We worked with @AIatMeta to tune content filters, which have since been updated on the publicly available model checkpoint. The new result: 41.9% Pass@1, placing Muse Spark 1.1 at #2 overall, behind only Claude Fable 5 (43.3%).