DiffuSpace Releases Clover-35B-A3B-Preview Open-Source Diffusion Language Model
DiffuSpace has released Clover-35B-A3B-Preview, marking its first open-source diffusion language model. The model features a 35-billion parameter mixture-of-experts architecture with 3 billion active parameters per token and a 128K context window. This release represents a significant technical milestone by combining large-scale mixture-of-experts routing with block diffusion for text generation. It challenges the dominant autoregressive paradigm by demonstrating competitive open-source diffusion performance. Clover utilizes block diffusion generation alongside a 35B MoE backbone to balance efficiency and generative capacity. At launch, it delivers state-of-the-art performance among open-source diffusion language models on the benchmarks reported in its release blog.
## BACKGROUND
Unlike traditional autoregressive models that generate text strictly token by token from left to right, diffusion language models generate tokens iteratively through a denoising process, allowing for parallel generation and bidirectional context. Mixture-of-experts (MoE) architectures partition computation across multiple sub-networks or experts, activating only a subset for each input to scale model capacity efficiently without increasing active compute.