Transferring Pretrained N-Gram Memory to Qwen3.5-0.8B Reduces Validation Perplexity by 5.05%

Independent developer Ninnix created Qwengram-0.8B, a technique that transfers the ~51B-parameter Predictive Language Enhancement (PLE) n-gram memory from Qwen3.8-Flash-Next into a frozen Qwen3.5-0.8B backbone. By training a lightweight reader and a dynamic token-level gate without fine-tuning the base model, the setup achieved a 5.05% reduction in validation perplexity. This project demonstrates that compact language models can be significantly augmented by attaching external, large-scale n-gram memory sidecars without modifying the base model's weights. It offers a resource-efficient path to improve small LLM accuracy on low-end hardware such as free Kaggle GPU instances. The setup pairs the frozen 0.8B backbone with an external 51B PLE sidecar, training an R=1 reader module at decoder layers 3 and 9 alongside a dynamic linear gate using 15 million tokens. The inference pipeline was implemented in a custom llama.cpp fork, where 8-bit quantization (Q8_0) retained 99.1% of the full-precision reader's loss reduction.