SupraLabs Releases Supra2-Medium-Base, a Tiny 25M Parameter Model
SupraLabs has released Supra2-Medium-Base, a 25-million parameter base model trained from scratch using the Qwen3 architecture. The developers claim it competes with their previous 50-million parameter model, though its initial sample output exhibits highly repetitive text. This release highlights the ongoing interest in TinyML and extremely small language models designed to run on resource-constrained hardware. However, the low quality of the generated text underscores the challenges of training functional LLMs at such a small scale. The model is currently a base model only, with an instruction-tuned version planned for the future. Additionally, the developers claimed to train the model on unreleased hardware (RTX 5060 series GPUs), raising doubts about the post's accuracy.
## BACKGROUND
Large Language Models (LLMs) typically contain billions of parameters, requiring significant computational power to run. TinyML is a field focused on deploying machine learning models on low-power, resource-constrained edge devices, which often requires shrinking models to millions of parameters. Base models are trained on raw text data to predict the next token and usually require instruction tuning to become useful conversational assistants.