/Tech6h ago

SkillsBench 1.1 launches to evaluate AI agent skill usage, showing top model resolution rates climb to 67%

The OpenHands agent currently outperforms all other evaluated models

7503164.8K

#89

Original post

Xiangyi Li@xdotli

In our SkillsBench 1.1 launch event, I talked about 'Community Is All You Need'. Kudos to all authors, contributors, and partners who made this project possible 🙏❤️

Authors and contributors: @xdotli @Yimin1010 @wenbochen8 @bingran_bry @ElegantLin21 @yfhe @shenghan_zheng @kywch500 @JiankaiSun @KookieMaster77 @chujun_tao69443 @li91889 @xuandongzhao @hejia0530 Xiaojun Wu, Junwei Zhou, @kobe0938, @harvenx01 @li_yubo66512 Qunhong Zeng, Di Wang, Yuanli Wang, @roeybc Penghao Jiang, Haotian Shen, @Chris61643492, Xinyi Liu, Runhui Wang, Xuanqing Liu, JIachen Li, Xin Lan, @YueqianL, Wengao Ye, Junwei He, Songlin Li, Yue Zhang, Yipeng Gao, @Williamiumli, Ze Ma, Liqiang Jing, Tianyu Wang, @kxli_2000, Yiqi Xue, Haoran Lyu, Yizhuo He, Yuchen Tian, @terryyuezhuo, Tinghan Ye, Qi Qi, Miao Li, Longtai Liao, Zelin Tan, Chang Shi, Xilin Tang, Srinath Tankasala, Boqin Yuan, Yaoyao Qian, Jianhong Tu, Chenguang Wang, Yizhou Sun, Wei Wang, Aaron Taylor, Ziyue Yang, Changkun Guan, Zhikang Dong, Xinyu Zhang, @junxian_he @May_F1_, @StevenDillmann, @HanchungLee, @dawnsongtweets

Credits supporters: @dkundel @gabrielchua @OpenAI @ZixuanLi_ @Zai_org @Young_AGI @Kimi_Moonshot @ShunyuYao12 @TencentHunyuan @XiaomiMiMo @xai @ivanburazin @daytonaio @gneubig @OpenHandsDev @johannes_hage @xeophon @PrimeIntellect @TheZachMueller @LambdaAPI @charles_irl @modal

3:10 PM · Jun 16, 2026 · 579 Views

Sentiment

Users appreciate SkillsBench 1.1 launching as an audited benchmark for AI agent skills because it focuses on real usage rather than leaderboard farming.

Pos

100.0%

Neg

0.0%

1 comments with sentiment.

Cluster Engagement

Digg Deeper

No Digg Deeper questions have been answered for this story yet.

Posts from X

Most Activity

VIEWS4.3KBOOKMARKS15LIKES34RETWEETS2REPLIES4

Graham Neubig@gneubig

SkillsBench is a great benchmark, and I'm not just saying that because @OpenHandsDev beats everyone else on it 😀

Skills are now the defacto standard for how we customize agents, so figuring out models/agents ability to use them well is super-important.

Xiangyi Li@xdotli

A big pain point in using AI benchmarks is encountering errors after its first release. Today, we're releasing SkillsBench 1.1, the first benchmark for how well AI agents use skills, now audited end to end and verified error-free. Prof. @dawnsongtweets joins 1.1 as advising author.

We worked through every task with several frontier labs to eliminate the errors in the previous version. We also added new tasks, moved the ones with external dependencies into a separate set so the core suite runs clean, and expanded coverage to more models.

Capability is climbing fast. The best with-skills resolution rate rose from ~36% (Claude Sonnet 4.5, Sep 2025) to 67% (GPT-5.5, May 2026), about +1.9 points per month. The frontier is hill-climbing SkillsBench fast.

The right skills still matter. Across the fleet, curated skills lift resolution rate by +16.6 points on average (33.9% → 50.5%), and by as much as +25.7 points for a single model. The top configuration is GPT-5.5 on OpenHands at 67.3%.

By popular demand (thx Nate @cursor_ai), we're now tracking skills invocation: how often an agent actually uses the skills it's given. Recent flagship configurations invoke them 90–99% of the time (Codex 99%, OpenHands + GPT-5.5 92%, Gemini CLI 90%), versus roughly 50% for older setups.

Also new in 1.1: @OpenHands joins as a fourth harness, alongside Claude Code, Codex, and Gemini CLI; a rebuilt leaderboard with refined categories, subdomain skill rankings, and Skill Lift; and native task . md on BenchFlow, with multi-scene environments and rollout branching. We also partnered with @k_dense_ai to add scientific skills to some science tasks.

One implication for deployment: skills can substitute for scale. GLM 5.1 with skills (58.4%) outperforms Opus 4.8 without (45.7%). A smaller model with the right procedural knowledge can beat a larger one running without it.

Huge thanks to @nick_kango @ivanleomk @kaggle @GoogleDeepMind for hosting a launch event with us. Thanks for everyone who's come on May 27!

Also thanks to our partners @gneubig @OpenHandsDev @ivanburazin @daytonaio @jackminong @johannes_hage @PrimeIntellect @TimothyKassis @k_dense_ai for providing support in credits, compute, and skills.

SkillsBench live leaderboard will also come to @ValsAI. Many people have told us they use SkillsBench as an index to measure models' agentic capability over diverse and high GDP value domains. Great work on Valkyrie as well! @ Jarett @nikilravi @langstonnashold @RayanKrishnan

SkillsBench is fully open-source. Explore the leaderboard and tasks, read the docs, or contribute your own skill set or harness and join the leaderboard. 🧵

6h4.3K3415

Xiangyi Li@xdotli

Website: https://skillsbench.ai Paper: https://arxiv.org/pdf/2602.12670 HuggingFace https://huggingface.co/datasets/benchflow/skillsbench PrimeIntellect: https://app.primeintellect.ai/dashboard/environments/xiangyi/skillsbench AgentBeats: https://agentbeats.dev/Yiminnn/skillsbench Harbor: https://hub.harborframework.com/datasets/benchflow/skillsbench GitHub: https://github.com/benchflow-ai/skillsbench

6h10432

Shangyin Tan@ShangyinT

@gneubig @OpenHandsDev I love the work but I think it’s missing a critical part: how generalizable are the skills?

5h332

Qiuyang Mang@MangQiuyang

@ShangyinT @gneubig @OpenHandsDev so GEPA for skill is all you need instead of mountains of static prompts, which will be garbage soon once Frontier Lab finishes the distillation

5h133

Saylor@seylorra

@gneubig @OpenHandsDev good to see benchmarks actually focusing on real usage instead of leaderboard farming

which models surprised u the most on skill retention?

5h2

Entropy Fighters@EntropyFighters

@gneubig @OpenHandsDev Skills are the standard now... but the harder benchmark is curation, not use. A skill library grows faster than anyone maintains it, and the real failure isn't "agent can't use the skill," it's "the skill went stale and the agent trusted it anyway." Restraint beats coverage.