Sabi-Noema: Capability and Efficiency Focused Fine-Tune of Qwen 27B
Developers have released Sabi-Noema and Sabi-Noema-MLA, two post-trained versions of the Qwen 27B model that enhance coding and tool-calling capabilities while reducing token verbosity by 50%. The MLA variant converts Qwen's GQA attention post-hoc to dramatically reduce KV cache size and memory requirements. These models push the boundaries of sub-30B open-weights models by offering significantly higher reasoning efficiency and much larger context lengths on standard consumer GPUs like the RTX 3090 or 4090. This development enables developers to run sophisticated agentic workflows locally with reduced costs and faster response times. Sabi-Noema-MLA utilizes a calibration dataset without extra training to achieve up to 8.4x smaller total KV cache sizes when using FP8 quantization, allowing a 24GB GPU to reach approximately 534K context length. Additionally, the models exhibit a significantly lower hallucination rate compared to competing options.
## BACKGROUND
Fine-tuning involves taking a pre-trained machine learning model and adapting it further on specific datasets to improve performance in designated domains like coding or tool use. KV cache optimization, such as Multi-head Latent Attention (MLA), reduces the memory footprint required during long-context inference by compressing key-value pairs.