Cactus Releases Needle 2, a 14MB Agentic LLM for Edge Devices
Cactus has released Needle 2, a 45-million parameter agentic LLM compressed to 2-bit that fits in a 14MB binary and runs within 28MB of RAM. It is optimized for tool calling, device use, and structured extraction on resource-constrained hardware like budget phones and microcontrollers. This model enables high-speed, local AI execution on billions of low-end IoT and edge devices without NPUs, offering a highly energy-efficient alternative to larger models. By drastically reducing memory and compute requirements, it makes always-on, on-device agentic capabilities practical and cost-effective. Needle 2 achieves up to 500 tokens/second on a Raspberry Pi 5 and uses only 70 MFLOPs per token by utilizing Simple Attention Networks that eliminate multilayer perceptrons (MLPs). It also features a confidence scoring system to escalate complex queries to larger cloud models when necessary.
## BACKGROUND
Traditional Large Language Models (LLMs) require gigabytes of memory and significant computational power, making them difficult to run locally on edge devices like smart home appliances or budget smartphones. Agentic LLMs focus on tool calling and function mapping, which allows them to perform specific tasks by interacting with external APIs rather than relying on internal world knowledge.