China Announces Domestic GPU and Neuromorphic Chip Hybrid LLM Inference System
Chinese researchers and enterprise partners have launched the country's first heterogeneous hybrid LLM inference system combining domestic GPUs with neuromorphic chips. Tested on DeepSeek models, the architecture doubles cost-effectiveness and reduces operational costs by more than 40% compared to standard domestic GPU clusters. This heterogeneous computing approach addresses severe memory bandwidth bottlenecks in LLM inference by offloading memory-intensive modules to specialized processing-in-memory hardware. It offers a viable, fully domestic hardware stack for latency-sensitive AI workloads like multi-agent collaboration and real-time code generation. The system assigns compute-heavy Attention layers to domestic GPUs while offloading memory-bound Feed-Forward Network (FFN/MoE) layers to neuromorphic chips featuring large on-chip SRAM. Task decomposition, cross-chip scheduling, and result aggregation are driven by a custom model compiler, high-speed interconnect protocol, and a unified inference engine.
## BACKGROUND
LLM inference involves two main compute patterns: Attention mechanisms require high floating-point processing capability, whereas Feed-Forward Networks (FFN) rely heavily on high memory bandwidth to load weight parameters. Neuromorphic processing-in-memory chips merge computation and data storage into the same hardware, eliminating the energy and latency overhead caused by shuffling data between traditional CPUs/GPUs and external memory.