aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

一项视觉语言模型研究发现:无关图形可改变代码写法

AI 导读

一则介绍 Google DeepMind 与 Stanford 论文的帖子称,研究者在编程提示中加入红心等纯色图形,测试了三款模型对递归解法的选择。在一道二叉树题中,红心使 Claude Opus 4.7 和 GPT-5.4 采用递归的比例从 0% 变为 100%;当图形影响 Gemini 3.1 Pro 时,其 71.5% 的思考记录仍称图像无关。

原帖引用 · 原文

@rohanpaul_ai

– https://t.co/8HP5321hAM

Title: "A Heart Means Recursion: Exploring Visual Meaning and Communication in VLMs"

回复Rohan Paul@rohanpaul_ai
so interesting. 🤔 New Google DeepMind and Stanford paper finds that a red heart picture in the prompt can decide how AI write code, that simple shapes in images can steer AI on unrelated tasks, often without the models saying so. plain colored shapes in a prompt change how AI models write code, though the shapes have nothing to do with it. Vision-language models now read screenshots and images as part of normal work. If an unrelated picture changes the model's choice, builders need to see it, ideally in the model's own reasoning. They added a plain colored shape, like a red heart, to a coding prompt and told the model to look at it. Across 288 shapes, they tracked how often Claude Opus 4.7, Gemini 3.1 Pro, and GPT-5.4 chose recursion. A red heart moved Claude Opus 4.7 and GPT-5.4 from 0% recursive solutions to 100% on a binary tree problem. When a shape did steer Gemini 3.1 Pro, 71.5% of its thinking traces called the image irrelevant. If your agent sees images, test its decisions with and without them, and do not count on reasoning traces to flag visual influence.
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错