Majestic Labs Announces Prometheus AI Server with Up to 128TB LPDDR6 Memory
Startup Majestic Labs has announced "Prometheus," an AI server that replaces traditional GPUs with Ignite AI Processing Units (AIUs) combining Arm cores and RISC-V engines. The server supports between 8TB and 128TB of next-generation LPDDR6 memory to challenge Nvidia's GPU and HBM dominance. By utilizing LPDDR6 memory instead of expensive and supply-constrained High Bandwidth Memory (HBM), Majestic aims to offer massive memory capacity at lower power consumption. This architecture could provide a cost-effective alternative for running large language models (LLMs) that are heavily memory-bound. A single Prometheus server can house up to 12 Ignite AIUs, and a standard 40U rack can accommodate four servers with a total liquid-cooled power draw of 120kW. The system is compliant with Open Compute Project (OCP) standards and supports popular AI frameworks like PyTorch, vLLM, and OpenAI Triton.
## BACKGROUND
Modern AI workloads, especially Large Language Models (LLMs), require massive memory capacity and bandwidth, typically provided by High Bandwidth Memory (HBM) stacked directly on GPUs. However, HBM is expensive and faces severe supply shortages, prompting hardware startups to explore alternative architectures using low-power DDR (LPDDR) memory and custom accelerators like RISC-V.