Ai2 study finds LLM judgments of research novelty sensitive to prompt wordingMachine translation
Ai2 发布研究：大模型判断科研想法新颖性易受提示词影响
Automatically verified and published · Generated and evidence-checked automatically; not reviewed by a human.
Ai2 tested six LLMs on their ability to judge the novelty of research ideas. In one comparison of human-authored and AI-generated ideas, a change in prompt wording lowered GPT-5.4’s accuracy from 93.5% to 40.9%. The team also released a benchmark, data and code; the lower novelty of the generated ideas was an assumption, not a verified judgment for each idea.
This source permits summary display only.
Read at the original source