正文 · AI 翻译
On November 9, Baidu ERNIE disclosed the latest progress in RSI for the first time
On November 9, Baidu ERNIE disclosed the latest progress in RSI for the first time, unveiling ERNIE Evolve, an automated research system involved in developing ERNIE 5.5 Flash. Given goals and constraints, it can independently propose research directions, conduct experiments, validate results, and accumulate experience. It can continuously improve existing approaches and explore new directions based on actual results. It achieved the best publicly verified SOTA performance to date on all three evaluation metrics across the NanoChat AutoResearch and NanoGPT Speedrun tasks. ERNIE Evolve has already been applied to inference optimization for the forthcoming ERNIE 5.5 Flash, including performance optimization for low-latency scenarios on the Kunlunxin P800, deterministic inference throughput optimization, and maximum decoding speed optimization.
ERNIE Evolve benchmark performance
Recursive Self-Improvement (RSI) is one of the major areas of exploration in AI worldwide today. Its core idea is to have AI participate directly in its own research and development, continually improving its capabilities and forming a closed loop of self-improvement. OpenAI, Anthropic, Tencent, Zhipu, and other companies are exploring the use of RSI in model development.
ERNIE Evolve organizes research through a two-level loop. The outer loop combines past experimental results with existing knowledge to propose new research directions and arrange for multiple research tracks to be explored in parallel. The inner loop continuously modifies code, evaluates performance, and records lessons learned for a specific direction. Multiple agents within the system divide the work and collaborate: Director chooses research directions, Selector determines each experiment’s objective and starting code, and Executor modifies the code and generates a new version. After each experiment, a separate evaluation program, Evaluator, assesses code performance and experimental results, while Reviewer further checks whether the research has strayed from its objective or whether there are flaws in the evaluation. Only improvements that pass evaluation and review are retained. The code changes, results, and lessons from each experiment are recorded for use in subsequent research.
ERNIE Evolve architecture
After ERNIE Evolve was integrated into real-world inference optimization for ERNIE 5.5 Flash, the system continued to deliver significant gains in the production environment.
For optimization of low-latency scenarios on the Kunlunxin P800, ERNIE Evolve spent approximately 168 hours exploring 4 directions. It first integrated native fused operators and enabled speculative decoding, then optimized host scheduling and synchronization mechanisms and communication paths. The best version increased generation speed to 623.6% of the baseline.
For deterministic inference throughput optimization, ERNIE Evolve spent 3 days exploring 3 directions involving deterministic sampling, deterministic switches, and operator parallelization strategies. While strictly preserving determinism, it ultimately raised throughput to 133.7% of the baseline, reduced average generation time per token by 26%, and cut first-token latency by 12%.
For maximum decoding speed optimization, ERNIE Evolve explored a total of 7 research directions. Of these, 3 produced effective improvements through optimizations to MoE and mHC operators, matrix multiplication and scheduling strategies, and the critical path of the draft model. Ultimately, cumulative inference performance rose to 114.9% of the baseline, while accuracy remained unchanged from the baseline.
All of the above changes have passed the ERNIE team’s stress tests and correctness checks and completed production-level validation.
Performance optimization process for low-latency scenarios on the Kunlunxin P800
Optimization process for deterministic inference throughput
Optimization process for maximum decoding speed
The ERNIE technical blog states that inference infrastructure optimization is RSI’s first application in real-world research and development. In the future, the same research loop will gradually extend to training system optimization, research into model capabilities, and iteration of evaluation systems, enabling AI to independently identify problems, propose hypotheses, design experiments, and verify results. This expansion will take place within clearly defined boundaries for goals, permissions, and resources, with ongoing safety assessments and human oversight.
On November 5, Baidu officially open-sourced its latest foundation model, ERNIE 5.5 Flash Base. The model achieved SOTA performance among models of the same size across multiple benchmarks covering knowledge, reasoning, mathematics, code, multilingual capabilities, and long context. A person close to Baidu said a new ERNIE model may be released soon.
Technical blog: https://ernie.baidu.com/zh/research/posts/ernie-evolve/
This article was provided by Baidu ERNIE and republished by QbitAI with permission. The views expressed belong to the original author.