原帖引用 · AI 翻译
– https://arxiv.org/abs/2611.09891
标题:“CPU-ARC:用于在 CPU 上复用推理的智能体式推理缓存”
英伟达的一篇新论文发现,一旦大语言模型对某个问题完成推理,其解法就可以作为 CPU 代码运行,用于该问题的改写版本和更换数字的版本。
这样回答类似问题的成本可降低至原来的 1/16 到 1/291。
大语言模型每次都会从头解决问题,即使变化的只是措辞或数字。智能体用新数据发送相同指令时,每次都要为这种重复推理付费。
他们的方法让 Claude Opus 4.5 将推理过程转化为小型 Python 函数,并把可变输入设为参数。运行时,这些函数在 CPU 上执行,只有提取数值等小型文本任务才会调用一个经过微调的 1B 模型。
在 8 项数学和科学基准测试中,这种缓存有 7 项的准确率达到或超过源模型;在 AIME '25 上得分为 93.3%,而源模型为 83.3%。它至少能处理 90% 的改写或更换数字的变体。
如果你的智能体整天用新输入运行同一个提示词,就把推理过程缓存为代码,只在缓存未命中时使用大语言模型。
原文
New NVIDIA paper finds that once an LLM has reasoned through a problem, the solution can run as CPU code for rephrased and re-numbered versions of it. that answers similar questions 16x to 291x cheaper. LLMs re-solve every problem from scratch, even when only wording or numbers change. Agents sending the same instructions with new data pay for that repeated reasoning each time. Their method has Claude Opus 4.5 turn a reasoning trace into small Python functions, with changeable inputs as parameters. At run time they execute on a CPU, calling a fine-tuned 1B model only for small text jobs like extracting values. The cache matched or beat the source model's accuracy on 7 of 8 math and science benchmarks, scoring 93.3% versus 83.3% on AIME '25. It handled at least 90% of rephrased or re-numbered variants. If your agent runs the same prompt on new inputs all day, cache the reasoning as code and use the LLM only on misses.