Summary
Google DeepMind announced a pilot of double-blind evaluations for a proprietary frontier model, running Gemini Flash Lite against confidential benchmarks in Google Cloud Confidential Space. Google says cryptographic separation keeps model weights private from evaluators and test prompts private from Google, with the Singapore AI Safety Institute, OpenMined, AVERI and MLCommons participating.
Why it matters
The approach addresses the longstanding confidentiality conflict that makes independent testing of closed models difficult and can improve confidence in safety and capability benchmarks.
Related coverage
- NSA seeks access to commercial AI models for pre-release cyber testing
- OpenAI, Anthropic and Google join 100-company call for AI cyber defences
- Google releases Gemini Omni 1.1 Flash for controllable AI video