aifollow.news
Search
Ai2 has released a study alongside a benchmark, data, and code. In one test pairing human-authored and AI-generated ideas, a prompt change reduced GPT-5.4’s accuracy from 93.5% to 40.9%. The researchers note that the labels treating generated ideas as less novel were not verified for every idea.
Coverage of the same occurrence shares a node; subsequent developments have their own nodes.
Ai2 tested six LLMs on their ability to judge the novelty of research ideas. In one comparison of human-authored and AI-generated ideas, a change in prompt wording lowered GPT-5.4’s accuracy from 93.5% to 40.9%. The team also released a benchmark, data and code; the lower novelty of the generated ideas was an assumption, not a verified judgment for each idea.
By original publication time, with each report's bookmarks and feedback preserved.
Ai2 has released a study alongside a benchmark, data, and code. In one test pairing human-authored and AI-generated ideas, a prompt change reduced GPT-5.4’s accuracy from 93.5% to 40.9%. The researchers note that the labels treating generated ideas as less novel were not verified for every idea.
Ai2 tested six LLMs on their ability to judge the novelty of research ideas. In one comparison of human-authored and AI-generated ideas, a change in prompt wording lowered GPT-5.4’s accuracy from 93.5% to 40.9%. The team also released a benchmark, data and code; the lower novelty of the generated ideas was an assumption, not a verified judgment for each idea.
Name a person, company or topic. We find the sources and gather every new development in your own tab.