DeepSeek-V4-Flash Benchmarked on M2 Ultra Mac Using DwarfStar Inference Engine

A user shared performance benchmarks for running the DeepSeek-V4-Flash model locally on an M2 Ultra Mac with 192GB RAM using the DwarfStar inference engine. The setup achieved decode speeds ranging from 28 tokens per second (t/s) at the start down to 18 t/s at a 192k context depth. This demonstrates the viability of running massive, state-of-the-art models like DeepSeek V4 Flash locally on high-end consumer hardware like Apple Silicon. It highlights how specialized inference engines and quantization can make large-scale local AI assistant setups practical without relying on cloud APIs. The benchmark showed that the decode speed of 18-28 t/s was maintained even with an output of 8k tokens at various context depths. DwarfStar achieves this by using selective quantization to compress the model size, making it compatible with high-memory Mac systems.