Moonshot AI to Release Open-Weight Kimi K3 Model
Moonshot AI is set to release the open-weight version of its flagship Kimi K3 model. The model features 2.8 trillion parameters, native multimodality, and a 1-million-token context window. Releasing a massive 2.8-trillion-parameter model under open weights provides the AI community with access to state-of-the-art reasoning and coding capabilities. It also paves the way for third-party inference providers to host the model, making advanced AI more accessible. Kimi K3 is built on Kimi Delta Attention (KDA), a hybrid linear attention mechanism, and Attention Residuals. Due to its massive size, running the model locally is impractical for most individual users, making cloud-based inference providers highly anticipated.
## BACKGROUND
Moonshot AI is an AI startup known for its Kimi chatbot and focus on long-context models. "Open-weight" models provide the public with the trained parameters of the neural network, allowing users to run or fine-tune them, unlike fully closed-source models which are only accessible via proprietary APIs.