Google DeepMind announces double-blind Gemini Flash Lite evaluation pilot

Google DeepMind announced a double-blind evaluation pilot on Aug. 27, 2026, partnering with the Singapore AI Safety Institute, OpenMined, AVERI and MLCommons to test a Gemini Flash Lite model against confidential benchmarks. DeepMind says Google cannot see the evaluators' test prompts, while the evaluators cannot see the model's weights.

DeepMind calls advance exposure to test questions “benchmark contamination” and says it can skew results. The company says high-stakes external evaluations have historically involved a trade-off: Evaluators could hand over their prompts, or model providers could hand over their weights. Its pilot is intended to avoid that exchange.

DeepMind says it uses Confidential Space, part of Google Cloud's Confidential Computing portfolio, to cryptographically verify that the evaluation data and proprietary model remain private to their respective owners. The company says this evidence helps prevent benchmark contamination. Its announcement describes the safeguards but provides no benchmark scores.