Original post · AI translation
这张图展示了 AI 的工作成果是如何受到检查的。
基本上，你永远不必信任 AI。
AI 智能体编写证明，但严格的数学检查器会剔除任何错误内容，因此你只需信任人类批准的安全规则，而不必信任 AI 本身。https://t.co/Ha174MKW4k
Original
This diagram shows how the AI’s work gets checked. Basically, you never have to trust the AI. The AI agents write the proof, but a strict math checker throws out anything wrong, so you only have to trust the security rules humans approved, not the AI itself. https://t.co/Ha174MKW4k
Google DeepMind 取得了一项重大成就：展示了一个 AI 系统能够在无需信任 AI 的情况下，以数学方式证明某个软件是安全的。
> 他们刚刚利用 AI 智能体，以数学方式证明了 Signal 实际运行的代码能让消息保持机密。
Google 的 AI 智能体编写了 650 万行数学证明，证明 Signal 编译后的代码能保护聊天内容的隐私。
检查深入到处理器实际执行的指令层面。该实验室表示，此前还没有人针对任何广泛使用的协议做到这一点。
> 常规测试会尝试许多输入，并希望碰巧遇到那个会出问题的输入。证明则表明安全规则在所有情况下都成立，而 Lean 证明检查器会拒绝 AI 写出的任何错误步骤。
> 真正的漏洞往往在设计转化为代码时悄然出现，因此检查处理器层面的代码可以弥合这一差距。Rust 编译器也不需要被信任，所以编译器犯的错误同样会被发现。
> 这项工作耗时 4 周，而过去专家团队通常需要数年。由 40 至 80 个 Gemini Carbon 智能体组成的团队编写了约 650 万行 Lean 代码，人类则设定安全目标，并审查了智能体标记出的 1%。
> 为检验该系统是否会无差别地认可任何代码，研究人员将 Signal 先前已修复的 2 个旧漏洞重新放入一个副本中。两次测试中，系统都生成了经过机器检查的证明，证明代码是不安全的。
> 更大的变化在于成本：如今 AI 可以在数周内生成经过机器检查的证明，不再需要耗时数年的项目，这种证明有望成为设备固件、电网软件和其他关键代码的常规检查。
Original
Big achievement by Google DeepMind for showing that an AI system that proves a software is secure mathematically, with no need to trust the AI. > They just used AI agents to mathematically prove that Signal’s real running code keeps messages secret. Google's AI agents wrote 6.5M lines of math proving Signal’s compiled code keeps chats private. checked at the level of the instructions a processor actually runs. The lab says no one had done this before for any widely used protocol. > Normal testing tries many inputs and hopes to hit the bad one. A proof shows the security rules hold in every case, and the Lean proof checker rejects any wrong step the AI writes. > Real bugs often creep in when a design gets turned into code, so checking the processor-level code closes that gap. The Rust compiler isn’t trusted either, so a mistake by the compiler would also get caught. > The work took 4 weeks, where expert teams used to need years. Teams of 40 to 80 Gemini Carbon agents wrote about 6.5M lines of Lean, while humans set the security goals and reviewed the 1% the agents flagged. > To check that the system can’t just approve any code, the researchers put 2 old, already fixed Signal bugs back into a copy. Both times it produced a machine-checked proof that the code was insecure. > The bigger shift is cost, now that AI can produce machine-checked proofs in weeks, rather than years-long projects and become routine checks for device firmware, power grid software and other critical code.