Developer Releases BetterGPT-150M, a Compact 152M Parameter Language Model
A developer has released BetterGPT-150M, a lightweight 152-million parameter causal language model trained on 15 billion tokens. The model is open-source, features open weights, and includes a live demo hosted on Hugging Face Spaces. This model offers a highly efficient alternative to older small models like GPT-2, making it ideal for edge computing, fast CPU inference, and lightweight LLM experimentation. It demonstrates how modern datasets and training techniques can maximize token efficiency in tiny architectures. BetterGPT-150M is a base text completion model rather than an instruction-tuned one, trained using stable and annealing phases on datasets like FineWeb-Edu and Cosmopedia. It features a very low RAM and vRAM footprint, allowing it to run instantly on standard CPUs.
## BACKGROUND
In LLM training, the annealing phase is a critical stage (often near the end of training) where the learning rate is decayed or data is reweighted to help the model converge and improve performance. Additionally, Hugging Face ZeroGPU is a shared infrastructure that dynamically allocates GPU resources to host AI demos efficiently.