aifollow.news
返回 Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · 原发布时间 AI 评分54

研究发现:用户对未变化的 AI 聊天机器人评价逐月趋严

AI 导读

一项基于 Chatbot Arena 1260967 次盲测投票的研究,只比较版本固定的模型。对同一模型组合,认为“两者都差”的投票每月增加 1.71 个百分点,平均占比为 17.1%;但在 2024 年,较强模型赢得明确偏好投票的比例大致稳定,排名未因此改变。

原帖引用 · 原文

@rohanpaul_ai

– https://t.co/8rnwRijPP3

Title: "AI Isn't Getting Dumber, We're Getting Pickier: Rising Standards in Human Evaluation of Large Language Models"

回复Rohan Paul@rohanpaul_ai
People rate the exact same AI chatbots more harshly each month, so a chatbot that never changes starts to look worse. They studied 1,260,967 blind votes on Chatbot Arena, where people compare answers from 2 anonymous models. They kept only pairs of frozen models, such as dated snapshots and fixed open weights, so the models never changed. On the same pair, "both are bad" votes rose 1.71 percentage points a month, on a 17.1% average. Yet in 2024 the stronger model won about the same share of decisive votes, so rankings held.
在 X 查看这条帖子 ↗

来源:Rohan Paul · x.com

论文
发现内容有误?提交纠错