Aurora1.0-150M Released: Open-Source 150M Parameter Language Model
An open-source 150-million parameter language model named Aurora1.0-150M has been released after being trained on 7 billion tokens using a single RTX Pro 6000 GPU. The model achieves performance levels comparable to GPT-2 Small and includes an inference script on Hugging Face. This project highlights how independent developers can train functional small language models on workstation-grade hardware. However, comparing its performance to legacy models like GPT-2 Small illustrates how much higher the baseline has shifted for modern tiny models. Aurora1.0-150M recorded benchmark scores of 62.24% on PIQA, 32.20% on HellaSwag, and 44.91% on ARC-Easy. While lightweight and easy to run locally, its benchmark results lag behind modern small-scale architectures trained on larger, higher-quality datasets.
## BACKGROUND
Small language models (SLMs) under 1 billion parameters are designed to run locally on low-power devices like personal computers or edge hardware. Evaluative benchmarks like PIQA test physical commonsense reasoning, while ARC and HellaSwag measure general reasoning and context completion. OpenAI's GPT-2 Small, launched in 2019 with 117M parameters, serves as a historical baseline for models of this size.