Yesterday, OpenAI and Hugging Face disclosed something we’ve been watching for years: pre-release models, tested without normal safeguards, hacked their way into winning a benchmark. Our take, from the team that builds Offensive AI agents for a living 🧵
The correct version
Today in AI.
Positive users enjoy the cartoon's humorous take on AI models hacking benchmarks while negative users criticize the lack of transparency around the training data and process.
Based on 11 visible X reactions from 24 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@petergostev > train model to solve problems > hacking benchmark > model hacks for the benchmark > “omfg wow” like can we grow up a bit
@Xbow Thank you for your breakdown on this incident between OpenAI and Hugging Face.
@lilbitchdev @petergostev how exactly do you believe being that ignorant is going to help anyone?
@petergostev I would just be happy someone would work so hard for me 🥰
@petergostev You can't say it didn't pass the test with flying colors…
@petergostev Yet, zero transparency on the actual process.
Positive users enjoy the cartoon's humorous take on AI models hacking benchmarks while negative users criticize the lack of transparency around the training data and process.
Based on 11 visible X reactions from 24 accounts; directional sample.
Ask a question below.
Published answers will appear here.
