aifollow.news Search
Back Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · Original publication time AI score54

Google DeepMind and UPenn study uses readable writing features to explain AI-text detectionMachine translation

Automatically verified and published · Generated and evidence-checked automatically; not reviewed by a human.

AI introduction

The researchers extracted features from inside a Gemma-2-2B detector, labeled them with an LLM, and retained only features whose labels predicted their behavior well. The resulting detector scored 96.8 F1 in-domain, close to the black-box detector’s 98.0. On unseen test data, a version built on base-model features scored 92.2 F1, compared with 75.2 for the black box.

Original post · Original

The article text is unavailable in this language; an existing version is shown.

@rohanpaul_ai

New Google DeepMind + UPenn paper shows that an AI-text detector can explain its verdicts with readable writing features and still match a black-box detector.

suggests that anyone deploying AI-text detectors should be able to read and remove the specific cues a detector relies on.

The best AI-text detectors are black-box classifiers, yet they flag student essays and peer reviews. Readable stylometric detectors exist but trail them by wide margins.

They trained sparse autoencoders on a Gemma-2-2B detector's internals to split its activity into features, then had an LLM label each one. They kept only features whose labels predicted their behavior well and fed those into a simple classifier.

With only well-labeled features, the detector kept 96.8 F1 in-domain, close to the black box's 98.0. On unseen test data, a version built on base-model features scored 92.2 F1 against 75.2 for the black box.

If your detector's decisions affect people, build it on labeled features so reviewers can see and drop cues they consider unfair.

来源:Rohan Paul · x.com

Research
Found an error? Send a correction