Arena Releases Inkling-Small, a 276B Parameter Open-Weights MoE Model
Arena (Thinking Machines Lab) has released Inkling-Small, a Mixture-of-Experts (MoE) model with 276 billion total parameters and 12 billion active parameters. The full weights of the model have been made publicly available, allowing developers to run it locally or fine-tune it. Inkling-Small delivers comparable performance to the original Inkling model at just a quarter of its size, making high-performance reasoning more accessible. By releasing the open weights, Arena enables researchers and developers to run and customize a massive MoE model efficiently. The model uses a Mixture-of-Experts architecture to activate only 12 billion of its 276 billion parameters per token, optimizing computational efficiency. Developers can fine-tune Inkling-Small using Tinker, a flexible training API that utilizes LoRA for resource-efficient adaptation.
## BACKGROUND
Mixture of Experts (MoE) is a machine learning technique that routes inputs to specialized sub-networks, allowing models to scale in capacity without a proportional increase in computing cost. Tinker is a training API developed by Thinking Machines that simplifies model fine-tuning using Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA.