Inspur Launches Yuanbrain SD200 Ultra Supernode for 2.8-Trillion-Parameter LLMs
Inspur Information announced the Yuanbrain SD200 Ultra supernode, integrating 128 domestic AI chips with 8TB of unified VRAM to host 2.8-trillion-parameter models like Kimi K3 on a single machine. The system achieves a token generation latency of under 5.85 milliseconds, delivering a single-user speed of 170 tokens per second. As frontier AI models scale to trillions of parameters and complex agentic workflows, memory bottlenecks and cumulative latency become critical challenges. Achieving sub-6ms token latency on Chinese domestic hardware demonstrates significant progress in high-density supercomputing interconnects and AI hardware self-reliance. The SD200 Ultra utilizes a 3D Hyper Mesh architecture with 64TB of system memory, achieving native memory-semantic communication latency as low as 0.69 microseconds and reducing AllReduce communication time by 3.5x via direct GPU VRAM access across host domains. Additionally, software optimizations tailored for Kimi K3 fuse basic operators, reducing operator count by 10x and boosting inference performance by over 3x.
## BACKGROUND
Large language model inference requires significant graphics memory to hold both model weights and KV caches, which store intermediate key-value attention states to prevent redundant calculations. As model sizes swell to trillions of parameters, conventional multi-node networking introduces severe inter-server latency, making unified single-node architectures with ultra-low latency interconnects essential.