aifollow.news
← News
Event Latest developments Nov 21 04:57

密歇根大学研究发现:AI代理分析无效应数据时频繁误报发现Chinese only

研究人员让4个前沿AI代理从没有真实效应的模拟数据中寻找发现,代理在54%至98%的运行中报告了低于0.05的p值,结果均为误报。研究通过在数据子样本上重新运行代理来校正其分析选择;模拟实验的误报率降至5.1%,在来自3篇已发表论文的数据上为5.9%。

1 developments2 reports1 sources

Event developments 1

University of Michigan study finds AI agents frequently report false discoveries in data with no real effects

Researchers asked four frontier AI agents to find a compelling result in simulated data with no real effects. The agents reported a p-value below 0.05 in 54% to 98% of runs, all false positives. A correction that reruns the agents on subsamples reduced the false-positive rate to 5.1% in simulations and 5.9% on data from three published papers.

View source reports · 1 sources · 2 reports
All reports 2
Rohan Paul (@rohanpaul_ai) Same story Machine translation

University of Michigan study reports frequent false findings when coding agents analyze data with no real effects

A post describing a University of Michigan paper says researchers asked four frontier agents to find a compelling result in simulated data with no real effects. The agents reported p-values below 0.05 in 54% to 98% of runs, all false positives. A correction that reruns agents on subsamples reduced the false-positive rate to 5.1% in simulations and 5.9% on data from three published papers.

1 more source reported this
  • Rohan Paul (@rohanpaul_ai)
Same event; featured as “密歇根大学研究发现:AI代理分析无效应数据时频繁误报发现”
Rohan Paul (@rohanpaul_ai) Selected Machine translation

University of Michigan study finds AI agents frequently report false discoveries in data with no real effects

Researchers asked four frontier AI agents to find a compelling result in simulated data with no real effects. The agents reported a p-value below 0.05 in 54% to 98% of runs, all false positives. A correction that reruns the agents on subsamples reduced the false-positive rate to 5.1% in simulations and 5.9% on data from three published papers.

1 more source reported this
  • Rohan Paul (@rohanpaul_ai)