ExllamaV3 and EXL3 Deliver High Benchmark Speeds for Local LLM Flash Next Inference

A local AI practitioner shared benchmarks demonstrating high performance using ExllamaV3 and the EXL3 quantization format on the Qwen3.8-Flash-Next model. A setup with three RTX 3090 GPUs achieved a prefill speed of 1,500 tokens per second and a decode speed of 80 tokens per second at a 262k context length. These benchmarks demonstrate that ExllamaV3 can significantly outperform popular inference engines like llama.cpp and vLLM for long-context multimodal inference on consumer hardware. This highlights rapid progress in open-source quantization techniques, enabling users to efficiently run large modern models on consumer GPUs. Testing with a 3-bit per weight (3bpw) EXL3 quantization included active vision processing and speculative decoding at a full 262k token context length. A single RTX 5090 system matched the 1,500 token/s prefill rate but achieved a slower decode speed of 29 tokens per second compared to the multi-3090 configuration.