Stepfun Releases Step-5-Preview: A 600B MoE Model with 1M Context
Stepfun has introduced Step-5-Preview, a flagship sparse Mixture-of-Experts (MoE) model featuring 600 billion total parameters and 27 billion active parameters. The model natively supports text, image, and video inputs while providing a massive 1-million-token context window. Designed specifically for agentic workflows, Step-5-Preview delivers frontier-level performance across software engineering and professional knowledge tasks, with exceptional strength in finance. This release represents a significant advancement in the capabilities of large-scale AI models geared toward complex, multi-step real-world operations. The model utilizes a sparse Mixture-of-Experts architecture to maintain high intelligence while keeping computational costs manageable during inference. It is optimized to handle large amounts of information, continuous tool calls, and multimodal data streams seamlessly.
## BACKGROUND
A Mixture of Experts (MoE) is a machine learning architecture where only a subset of neural networks (experts) are activated for any given input, allowing models to scale up total parameter count without a proportional increase in computing cost. A context window refers to the maximum number of tokens—words or sub-words—a language model can process and retain in its working memory at a single time.