KRAFTON AI Previews Raon-1 mini Open-Source MoE Model
Korean gaming company KRAFTON AI previewed Raon-1 mini, a 30B-A4B open-source Mixture of Experts (MoE) model featuring linear attention and speculative decoding. Released under the Apache 2.0 license, the model focuses primarily on Korean while maintaining strong English performance and offering natural conversational abilities. The release offers a competitive regional alternative in the open-source LLM landscape, rivaling Meta's Muse Glimmer 30B in coding and agent tasks while providing faster speeds and higher memory efficiency. Its availability under a permissive license empowers developers to build customized AI applications with optimized resource usage. The Raon-1 mini model utilizes a 30B-A4B configuration and incorporates linear attention alongside speculative decoding to optimize inference speed and memory footprint. KRAFTON AI has made the preview version available with a limited-time free API trial on their official website, ahead of their larger upcoming models.
## BACKGROUND
Mixture of Experts (MoE) is a neural network architecture where multiple specialized sub-models (experts) are used, but only a subset is activated for any given input to reduce computational costs. Linear attention modifies standard transformer attention mechanisms to scale linearly with sequence length, while speculative decoding accelerates text generation by using a smaller draft model to propose tokens that a larger model verifies in parallel.