Singularity-Bench Tests Whether AI Agents Can Build Training and Inference SystemsMachine translation

New Carnegie Mellon paper finds that Opus 5.5 + Claude Code built a pre-training pipeline, an inference engine, and a cluster orchestrator that matched or beat production tools like SGLang and NVIDIA Dynamo on the same GPUs. Singularity-Bench gives agents 6 end-to-end jobs, from pre-training a model to building a serving engine. Each result is scored against a production system, such as SGLang or Claude Code, on the same GPUs. Opus 5.5 scored a Singularity-Index of 1.25, where 1 equals the human baselines, while the other 10 agents averaged 0.58. Its serving engine earned 11.52 USD per GPU-hour against 8.18 for SGLang.