aifollow.news Search
Back Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · Original publication time AI score54

Post says GPT-6-Astra solved 49 of 50 unpublished physics research problems with web searchMachine translation

Automatically verified and published · Generated and evidence-checked automatically; not reviewed by a human.

AI introduction

A post describing a new paper says experts supplied 50 unpublished theoretical physics research problems, each roughly a day's work for a specialist. GPT-6-Astra in Codex solved 49 with web search; without search, both it and Opus 5.5 in Claude Code solved 46. Grading was based only on final numerical answers.

Original post · Original

@rohanpaul_ai

@EaglesPerch360 🫡🫡

ReplyEaglesPerch360@EaglesPerch360
#fascinating … thank you @rohanpaul_ai
View this post on X ↗
Related contextRohan Paul@rohanpaul_ai
New paper from Google Deep Mind, and many top Universities finds the best AI agent tested solved 49 of 50 unpublished research-level theoretical physics problems, each roughly a day's work for a world expert. at under $3 each. Benchmarks for AI in physics often use textbook or contest problems, or block the tools real researchers use. So it was unclear how agents handle the calculations that fill daily research. Experts contributed 50 unpublished problems from their own research, each about a day's work for a specialist. Agents worked alone, with and without web search, and were graded only on final numerical answers. GPT-6-Astra in Codex solved 49 of 50 with web search, and both it and Opus 5.5 in Claude Code solved 46 of 50 without it. Accuracy barely changed without search, which suggests the models have memorized much of the physics literature.
View this post on X ↗

Source: Rohan Paul · x.com

Research
Found an error? Send a correction