NVIDIA AI Skill Benchmarks Show Agent Gains
Nous Research integrates the SkillEvaluator tool into skill installations.
TLDR
NVIDIA AI reported benchmarking over 300 verified skills on identical tasks and models. Agents using the skills showed gains of 41 points in correctness, 39 in effectiveness, and 35 in efficiency. Nous Research stated it now runs SkillEvaluator on skill installs to check for PII, leaked secrets, Unicode smuggling, licensing, and security issues. The company applied the tool to its own bundled skills and improved 11 of them.
Combined views
318.7K
2 Sources, first seen ago
Sources
NA 
NVIDIA AI@NVIDIAAIWe benchmarked 300+ NVIDIA verified skills to see how much they actually help agents on real tasks. Same task, same model, same setup. The only difference was whether the agent had the skill. Across the benchmarks, skills improved correctness by 41 points, effectiveness by 39,…
- likes: 835
- replies: 29
- bookmarks: 715
- reposts: 91
NR Nous Research@NousResearchHermes now leverages NVIDIA's SkillEvaluator on skill installs, checking for PII, leaked secrets, Unicode smuggling, licensing and security issues before you confirm. We pointed it at our own bundled skills first, and used what it found to improve 11 of them.…
- likes: 1.9K
- replies: 48
- bookmarks: 844
- reposts: 125
MR mr-r0b0t@mr_r0b0t@NousResearch my two favorite companies in one post, teaming up to give me something quite valuable, just because they want to make sure people are getting the best product experience possible. absolute art
XL Xiangyi Li@xdotli@NVIDIAAI amazing work! would love to integrate this to SkillsBench & BenchFlow
LP Lorenzo Price@lorenzo_madeit@NVIDIAAI 41 points of lift just for installing SkillEvaluator is absurdly good.
FU Fueloncker@fueloncker@NVIDIAAI the evil boss of semi