Developer Upgrades Homelab to 36-Node DGX Spark Cluster for Local AI Inference
A developer has upgraded their homelab cluster from 16 to 36 NVIDIA DGX Spark nodes, achieving 4.6TB of unified memory and 200Gbps networking. This setup is designed to run state-of-the-art AI models and a custom multi-agent inference system locally. This project demonstrates the feasibility of running massive, state-of-the-art models like the 2.8-trillion-parameter Kimi K3 entirely on-premise without relying on cloud datacenters. It highlights a growing trend of high-performance, sovereign AI infrastructure built within home environments. The cluster utilizes a 200Gbps FS switch with QSFP56 DAC and breakout cables to handle high-speed networking, allocating 16 nodes specifically for heavy LLM inference while reserving others for auxiliary tasks like embeddings and image generation. The developer opted for DGX Sparks over B200/B300 GPUs due to the latter's extreme cooling, power requirements, and high costs in a home environment.
## BACKGROUND
NVIDIA's DGX Spark is a hardware platform designed for distributed AI workloads, allowing multiple units to be clustered together via high-speed networking to run models that exceed the memory of a single machine. QSFP56 is a network transceiver standard supporting speeds up to 200 Gbps, which is crucial for minimizing latency in distributed parallel inference. Kimi K3 is a massive 2.8-trillion-parameter open-weights LLM developed by Moonshot AI.