正文 · AI 翻译
IT Home reported on November 21 that OpenAI issued an announcement on November 20 local time describing its defenses against “adversarial distillation.”
OpenAI said it has strengthened protections for the internal chain of thought of GPT-6.2 Astra and GPT-6.2 Sol to reduce the risk of other models acquiring their capabilities by extracting their reasoning processes.
OpenAI said adversarial distillation refers to the unauthorized use of outputs from frontier AI models to train other models, giving those models similar capabilities without the original models’ corresponding safety protections. This could transfer high-risk capabilities in areas such as cybersecurity and biology to models with weaker safety restrictions.
In internal adaptive black-box red-team tests, OpenAI measured how often attackers succeeded in extracting protected reasoning content from a model’s previous and current requests.
The results showed that the success rate for extracting reasoning content from previous requests fell from 54.0% under GPT-6 Astra’s original safety measures to 0.2% under GPT-6.2 Astra’s updated measures. For reasoning content from the current request, the rate fell from 22.9% to 0.9%. The results combine data from 6 attacking models.
To provide these protections, OpenAI adopted multiple measures, including adversarial training to teach models to recognize and reject requests to extract chains of thought; input and output classifiers to detect related attacks and leaks of reasoning content; and API restrictions on reusing reasoning content across different sessions, models, and organizations. An image provided by IT Home follows:
GPT-6.2 Astra and GPT-6.2 Sol link protected reasoning content to the conversation history that generated it. If an earlier message is modified, the API discards subsequent reasoning content generated based on that message. Older models with weaker safety protections also cannot read the protected reasoning content of these 2 new models.
Regarding enforcement against such activity, OpenAI said it has taken action each week this year against accounts involved in unauthorized distillation, ranging from thousands to tens of thousands of accounts, with enforcement increasing markedly since this summer. The company also investigates networks of accounts coordinating large-scale extraction and shares attack indicators with other model providers to help identify related abuse.
OpenAI also described its research into the safety risks of distillation. Internal tests found that student models fine-tuned under supervision using a teacher model’s complete reasoning traces generally achieved better transfer of cybersecurity capabilities than student models trained using only the model’s final answers, reasoning summaries, and records of tool interactions.
Related reading:
“Fields Medal winner Deng Yu comments on AI companies’ opposition to distillation: You cannot support distillation only when you are distilling all of humanity’s knowledge”
“Nvidia CEO Jensen Huang responds to controversy over AI model distillation: It is market competition”
“Ministry of Commerce responds to alleged U.S. accusations of AI distillation: Baseless in fact and law, a typical double standard”
“Reports say Zhang Yiming issued a firm internal order: Even if its models temporarily lag behind, ByteDance will not rely on AI distillation technology”
“Microsoft CEO Nadella criticizes AI model companies’ ‘double standards’: Advocating ‘fair use’ of data while restricting others from distilling their models”