aifollow.news
返回 IT之家 科技新闻
IT之家 科技新闻· · 原发布时间 AI 评分47

匿名模型 Gestalt 登顶 Contra 设计盲测榜单

AI 辅助摘要

Contra Labs 公布的设计评测中,匿名模型 Gestalt 以 1931 分的 Elo 评分居首,专业设计师在与人类参考作品的 28 次直接比较中有 20 次选择了它。评测涵盖 33 项任务、35 名设计师和 4311 张投票;但 Gestalt 在另一项设计质量评价中得分 83 分,低于两款得分 84 分的 Claude 模型,其开发团队和架构仍未公开。

正文 · AI 翻译

Thanks to IT Home readers 汪淼 and StrangeAi for the tips!

IT Home reported on November 21 that Contra Labs released its AI design capability benchmark, Frontier Design Benchmark, on November 18 local time.

Among the models, an anonymous AI model codenamed Gestalt, currently in stealth development, emerged as the biggest surprise. It topped a blind test by professional designers by a clear margin, surpassing mainstream models including Anthropic’s Claude Fable 5.2 and OpenAI’s GPT-6.1 Sol, and even scored higher on designer preference than reference work by human designers.

According to the evaluation results officially published by Contra Labs, Gestalt achieved an Elo rating of 1931 and a win rate of 90%, placing first on the leaderboard.

The human designer reference work, Human golden, ranked second with an Elo rating of 1739 and a win rate of 74%; Claude Fable 5.2 ranked third with 1713, while GPT-6.1 Sol ranked fifth with 1668.

The benchmark was launched jointly by Contra Labs and BenchFlow and focuses on real commercial design tasks rather than simple prompt generation tests. It includes 33 tasks: 27 adapted from real design commissions on the Contra platform and another 6 from startup collaborations. The evaluation involved 35 professional designers and collected 4311 votes across 5 rounds of blind testing.

The evaluation uses 3 measures: double-blind preference voting by professional designers, the Design Critique Rubric, and the Slop Detector. The Elo rating is calculated from designers’ pairwise comparisons of different works, covering color, typography, layout, usability, and adherence to task requirements.

In direct comparisons between Gestalt and the human designer reference work, designers chose Gestalt in 20 of 28 comparisons. However, this does not mean Gestalt surpassed human work across every design quality measure.

On the Design Critique Rubric, Gestalt scored 83, Claude Fable 5.2 and Claude Opus 5.5 each scored 84, and the human designer reference work scored 74. Contra Labs noted that designers’ aesthetic preferences do not entirely match the quality of execution: although the human reference work had some identifiable execution issues, it still received a relatively high number of preference votes in the blind test.

Gestalt’s work also performed well in AI artifact detection, with its generated results receiving an AI artifact score close to that of the human reference work. The detector uses a classification model to assess how many typical characteristics of AI-generated work a design exhibits; a lower score means the work looks less AI-generated.

Contra Labs has not yet disclosed Gestalt’s development team, model architecture, or specific identity, labeling it only as Stealth. The model’s actual origin and the team behind it remain to be confirmed.

Related reading:

“The world’s most powerful AI model: Claude Fable 5.2 arrives, surpasses GPT-6 Astra in coding and tops multiple benchmarks”

发现内容有误?提交纠错