CrowdGPT v2.6 Released for Collaborative Open-Source LLM Training
CrowdGPT v2.6 introduces major security upgrades, robust server-side validation, and client cryptographic signing to prevent tampering and arbitrary code execution. The update also enabled processing 4.5 billion new tokens in six days and added a collaborative dataset submission pipeline. This project demonstrates a viable proof-of-concept for training language models collaboratively on consumer hardware without expensive data centers. It empowers the open-source community to pool resources and participate in foundational AI development securely. The update implements byzantine cross-checks between clients to mitigate data poisoning and malicious backdoors, limiting malicious contributions to a small percentage per round. Additionally, the project introduced CrowdBench, an open-source collaborative benchmark for testing LLMs.
## BACKGROUND
Distributed machine learning is a computational paradigm that enables training models across multiple decentralized devices or nodes. Mixture-of-Experts (MoE) architectures use a gating network to dynamically activate specialized subnetworks, allowing scalable and efficient learning across complex tasks.