Acrab Unveils GΞLIX 1 Edge AI SoC Supporting Local 100B Models
Startup Acrab has officially released its first chip, the GΞLIX 1, a 5nm heterogeneous edge AI SoC designed to run large language models with up to 100 billion parameters locally. This chip enables high-performance, low-latency local execution of massive AI models on edge devices, reducing reliance on cloud computing and enhancing privacy. Its hardware-accelerated attention mechanism addresses key bottlenecks in LLM inference, potentially shifting how AI agents are deployed. The GΞLIX 1 features a 20-core Arm CPU, 3TFLOPS GPU, NPU delivering 650TOPS of AI compute, and 256-bit LPDDR5X shared memory. It achieves a prefill rate of 1,416.8 tokens/s on a Gemma 26B configuration, which is 7.5 times faster than Apple's M4 Pro Mac Mini.
## BACKGROUND
LLM inference consists of two main phases: the prefill phase, which processes the initial prompt in parallel, and the decode phase, which generates subsequent tokens sequentially. To avoid redundant calculations during decoding, a Key-Value (KV) cache is used to store intermediate attention states, though it requires significant memory bandwidth.