Article · AI translation
IT Home reported on October 15 that StepFun officially released the Step5Preview weights today. This release includes BF16 and FP8 model weights, a technical blog, and accompanying open-source development tools.
According to the company, Step 5 Preview is designed for long-horizon agent tasks. It uses a sparse MoE architecture with 600B total parameters and only 27B active parameters per token, supports a 1 million-token context window, and natively supports text and visual input. It performs particularly well in AI coding, software engineering, professional knowledge work, finance, and other areas. Recently, Step 5 Preview ranked second globally on OpenRouter's weekly and daily call-volume charts.
The model weights are now available on Hugging Face and ModelScope, with support for multiple chip platforms and inference frameworks. Developers can deploy an OpenAI-compatible API service through vLLM or SGLang, or use Transformers for offline inference, connecting the model to their own data, tools, and systems to build agents, applications, workflows, and services.
IT Home noted that the technical blog published at the same time explains how Step 5 Preview supports longer and more complex agent tasks through coordinated optimization of its architecture and training. Key design choices include:
A narrow, deep 92-layer architecture: This provides a deeper computation path for implicit multi-hop reasoning while keeping parameter count and computational cost under control.
Sparse GQA: By combining block-level indexing with sparse computation, it reduces the computational cost of a million-token context. In particular, 8-token block-level indexing can reduce the cost of indexing and top-k selection to 1/8 of its original level.
More stable training: It retains FP32 precision for key states and parameters and combines this with fine-grained gradient clipping to mitigate abnormal activation growth and gradient spikes in deep networks.
A million-scale set of verifiable environments: These cover software engineering, scientific reasoning, machine learning research and development, and professional knowledge work, with ongoing calibration of task difficulty, diversity, and evaluation reliability.
Reinforcement learning for long-horizon tasks: Context compression is incorporated into end-to-end policy training, allowing the agent to learn to organize task state and continue execution. More granular reward attribution is also used to improve learning from long trajectories.