Zagreus-0.4B-por: A Small Open-Source Portuguese-English Language Model
Open-source AI lab mii-llm has released Zagreus-0.4B-por, a compact 400-million-parameter bilingual Portuguese-English language model pretrained from scratch. The model outperforms the larger Qwen3-0.6B-Base on Portuguese benchmarks, scoring 0.3230 compared to Qwen3's 0.2569. It demonstrates the viability of training highly efficient, small language models (SLMs) for specific non-English languages, making AI more accessible for edge and on-device applications. It also provides a customizable foundation for researchers and developers focusing on Portuguese NLP. The model was trained on open datasets including FineWeb Portuguese, FineWeb2 Portuguese, FinePDFs, and StarCoderData, with compute sponsored by Seeweb and Regolo.ai. It is released as a base model intended for further instruction tuning and research rather than a finished assistant.
## BACKGROUND
Small language models (SLMs) are increasingly popular because they require fewer computational resources to run, enabling on-device deployment. Benchmarks like ARC, HellaSwag, and MMLU, often run via frameworks like EleutherAI's LM Evaluation Harness, are standard tools used to measure a model's reasoning, knowledge, and language understanding capabilities.