Original post · AI translation
标题：“一颗心意味着递归：探索视觉语言模型中的视觉意义与交流”
真有意思。🤔
Google DeepMind 和斯坦福大学的一篇新论文发现，提示中的一张红色爱心图片可以决定 AI 如何编写代码；图像中的简单形状能够引导 AI 完成无关任务，而且模型往往不会说出这一点。
提示中的纯色形状会改变 AI 模型编写代码的方式，尽管这些形状与代码毫无关系。
视觉语言模型如今在日常工作中会读取截图和图像。如果一张无关图片改变了模型的选择，开发者就需要察觉到这一点，最好能从模型自身的推理中看出来。
他们在编程提示中加入一个纯色形状，例如一颗红色爱心，并让模型看它。他们测试了 288 种形状，追踪 Claude Opus 4.7、Gemini 3.1 Pro 和 GPT-5.4 选择递归的频率。
在一道二叉树问题中，一颗红色爱心使 Claude Opus 4.7 和 GPT-5.4 给出递归解法的比例从 0% 升至 100%。当某个形状确实影响 Gemini 3.1 Pro 的选择时，其思考轨迹中有 71.5% 称该图像无关紧要。
如果你的智能体会看到图像，就要分别在有图像和无图像的情况下测试它的决策，不要指望推理轨迹会指出视觉影响。
Original
so interesting. 🤔 New Google DeepMind and Stanford paper finds that a red heart picture in the prompt can decide how AI write code, that simple shapes in images can steer AI on unrelated tasks, often without the models saying so. plain colored shapes in a prompt change how AI models write code, though the shapes have nothing to do with it. Vision-language models now read screenshots and images as part of normal work. If an unrelated picture changes the model's choice, builders need to see it, ideally in the model's own reasoning. They added a plain colored shape, like a red heart, to a coding prompt and told the model to look at it. Across 288 shapes, they tracked how often Claude Opus 4.7, Gemini 3.1 Pro, and GPT-5.4 chose recursion. A red heart moved Claude Opus 4.7 and GPT-5.4 from 0% recursive solutions to 100% on a binary tree problem. When a shape did steer Gemini 3.1 Pro, 71.5% of its thinking traces called the image irrelevant. If your agent sees images, test its decisions with and without them, and do not count on reasoning traces to flag visual influence.