aifollow.news Search
Back Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · Original publication time AI score54

Post says Harvard study compared AI and expert paper reviewsMachine translation

Automatically verified and published · Generated and evidence-checked automatically; not reviewed by a human.

AI introduction

A post says Claude Opus 4.8 reviewed 1,124 papers and its reviews were compared with expert reviews from rapidPeer. In 132 blind comparisons with a decision, GPT-5.5 chose the AI review every time, while three former journal editors chose it 87% of the time. The post says 5.5% of the AI reviews’ claims about the papers were wrong, versus 15.9% for the expert reviews.

Original post · Original

The article text is unavailable in this language; an existing version is shown.

@rohanpaul_ai

@VinVomero yes, so very much

ReplyVin Vomero@VinVomero
@rohanpaul_ai Worth knowing the AI reviews were written by Opus 4.8 and judged by GPT-5.5. Opus 5.5 is already out, so the gap is probably even wider now. I'd like to see them rerun it on current models.
View this post on X ↗
Related contextRohan Paul@rohanpaul_ai
Massive. New Harvard study finds that AI-written paper reviews now beat the best human peer reviews on every measure the authors tested. Peer review is slow and often misses errors. Earlier studies concluded AI should only assist human reviewers, not replace them. They used 134 reviews from rapidPeer, a Harvard service that pays expert reviewers well. Claude Opus 4.8 reviewed the same 1124 papers, and GPT-5.5 plus 3 former journal editors judged the pairs blind. GPT-5.5 picked the AI review in all 132 decided pairs, and the editors picked it 87% of the time. Only 5.5% of the AI's claims about the paper were wrong, versus 15.9% for the human reviews. Teams that depend on expert critique can now let AI do the first full review pass, and keep a human for the final decision.
View this post on X ↗

来源:Rohan Paul · x.com

Research
Found an error? Send a correction