原帖引用 · AI 翻译
@thesimaximalist 😄😀
@rohanpaul_ai 你把“594B”和“Small”放在同一句话里，说得那么轻描淡写 😂
原文
@rohanpaul_ai You slipped “594B” and “Small” into the same sentence so casually 😂
Thinking Machines Lab 兑现了承诺，将 Inkling 2 Small 开源。这是一个拥有 5940 亿参数、采用 4 位前向传播训练的模型。
> 现已在 HuggingFace 上提供下载，采用 Apache 2.0 许可证
> 4 位训练使预训练时间缩短 8%，强化学习时间缩短 20%，效率损失约为 2%。
> 运行该模型时，采用 NVFP4 格式至少需要 500GB GPU 内存，采用 MXFP8 格式至少需要 800GB；在评测中，MXFP8 比 NVFP4 高出“微乎其微的 1–2 个百分点”。
> 由于从预训练到最终的强化学习推演阶段都保持相同的数值格式，发布的 4 位模型检查点无需单独进行校准。
> 该实验室最初尝试在前向传播和反向传播中都使用 NVFP4，但这使计算效率下降过多，因此只有前向传播保留了 4 位格式。
原文
Thinking Machines Lab kept its promise and open-sourced Inkling 2 Small, a 594B-parameter model trained with 4-bit forward passes. > now available for download on HuggingFace under an Apache 2.0 licens > Training in 4-bit cut pretraining time 8% and RL time 20%, for about 2% efficiency loss. > Running it takes at least 500GB of GPU memory in NVFP4 and at least 800GB in MXFP8, and MXFP8 gains “a very marginal 1–2 percentage points” over NVFP4 on evals. > Because training kept the same numerics from pretraining through the final RL rollout, the released 4-bit checkpoint needed no separate calibration step. > The lab first tried NVFP4 in both forward and backward passes, but that hurt compute efficiency too much, so only the forward pass stays in 4-bit.