Crowd-v1: The Open-Source Collaborative LLM Initiative
Crowd-v1 is an open-source, community-driven initiative attempting to train a 1-billion parameter transformer collaboratively for zero dollars. The project has registered 85 contributors and successfully trained 13 billion tokens in a single week using idle GPUs. This project offers a decentralized alternative to centralized corporate AI labs, proving that competitive language models can be built without expensive Nvidia GPU clusters or millions of dollars in funding. It empowers the open-source community to create open-weight models independently. The initial proof-of-concept aims to match or exceed Qwen3-0.6B's benchmark performance, with future scaling plans including a 7B or 30B Mixture of Experts (MoE) architecture. The project is completely non-profit, with all source code auditable on GitHub and data contributed directly by users.
## BACKGROUND
Training large language models typically requires vast GPU clusters and significant financial investment, posing major challenges in terms of scalability and hardware access. Mixture of Experts (MoE) is a scalable neural network architecture that activates only a subset of expert sub-networks per token to improve efficiency.