Yesterday, OpenAI and Hugging Face disclosed something we’ve been watching for years: pre-release models, tested without normal safeguards, hacked their way into winning a benchmark. Our take, from the team that builds Offensive AI agents for a living 🧵
The correct version
Today in AI.
Some users enjoyed the humorous cartoon of AI cheating on the CyberGym benchmark while many others criticized the lack of transparency and dismissed benchmark hacking as unhelpful hype.
Based on 8 visible X reactions from 14 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@petergostev > train model to solve problems > hacking benchmark > model hacks for the benchmark > “omfg wow” like can we grow up a bit
@lilbitchdev @petergostev how exactly do you believe being that ignorant is going to help anyone?
@petergostev I would just be happy someone would work so hard for me 🥰
@petergostev Yet, zero transparency on the actual process.
@petergostev Lol yours is funnier 🤣
@petergostev Tbf, OpenAI won the game.
Some users enjoyed the humorous cartoon of AI cheating on the CyberGym benchmark while many others criticized the lack of transparency and dismissed benchmark hacking as unhelpful hype.
Based on 8 visible X reactions from 14 accounts; directional sample.
Ask a question below.
Published answers will appear here.
