Local LLM Developer Contrasts Personal Setup With Enterprise Scale
A developer shared a lighthearted post describing their personal local LLM setup running two Strata engines at 70 tokens per second and consuming 12 million tokens daily. They contrasted this with enterprise-scale usage, noting a developer handling 34 million tokens per minute. This post highlights the vast operational gap between individual hobbyist or developer setups and massive industrial-grade AI workloads. It provides a humorous perspective on the modern AI landscape where token consumption scales exponentially for enterprise developers. The user mentioned running engines that achieve 70 tokens per second locally, while the enterprise workload referenced belongs to a former Cursor developer contributing thousands of pull requests monthly to AI tools like GrokBot.
## BACKGROUND
Local LLM inference servers allow users to run large language models on personal hardware for tasks like coding and chat, ensuring privacy without relying entirely on cloud APIs. Tokens represent the fundamental units of text that LLMs process, and tokens per second (tps) is a primary metric measuring inference speed.