Moonshot AI to Retire Trillion-Parameter Kimi K2.5 Model at Month's End
Moonshot AI has announced that its first-generation trillion-parameter multimodal model, Kimi K2.5, will be retired at the end of August. This transition follows the release of its successor, the 2.8-trillion-parameter Kimi K3 model, which was open-sourced in July. This retirement marks a routine transition in Moonshot AI's product lifecycle as they shift focus to their newer, larger, and more efficient Kimi K3 model. It highlights the rapid pace of development in large language models, where even trillion-parameter models are quickly replaced by superior architectures. Kimi K2.5 was launched in January as a general-purpose multimodal model supporting vision, text, and agent tasks. Its successor, Kimi K3, features a 2.8 trillion parameter scale, a 1-million-token context window, and utilizes the Kimi Delta Attention (KDA) hybrid linear attention mechanism.
## BACKGROUND
Moonshot AI is a prominent Chinese AI startup known for its Kimi smart assistant and long-context capabilities. The newer Kimi K3 model utilizes Kimi Delta Attention (KDA), an expressive linear attention module that extends Gated DeltaNet to optimize the use of finite-state RNN memory, allowing for highly efficient processing of long contexts.