Local AI User Requests Faster Qwen 3.8 35B MoE Variant for Apple Silicon
A community discussion on Reddit highlights a user's request for a Qwen 3.8 35B model with 3B active parameters (A3B) to balance speed and intelligence on consumer hardware like the Apple M1 Max. The user notes that the current Qwen 3.8 27B model is too slow for practical local use due to its long reasoning/thinking times. This highlights the ongoing challenge for local LLM users who must balance model intelligence (often scaled via test-time compute) with hardware limitations. It underscores the demand for Mixture-of-Experts (MoE) architectures that reduce active parameters to make advanced models runnable on consumer-grade hardware. The user proposes a "35B A3B" model, which refers to a Mixture of Experts (MoE) setup with 35 billion total parameters but only 3 billion active parameters per token, significantly speeding up inference. Running reasoning models locally on Apple Silicon (like the M1 Max) without dedicated GPU hardware like Nvidia RTX cards often results in impractical generation times.
## BACKGROUND
Mixture of Experts (MoE) is an LLM architecture that uses only a subset of its total parameters (active parameters) for each token, allowing larger models to run faster. Test-time compute (or inference-time compute) allows reasoning models to generate internal chains of thought to solve complex tasks, which increases accuracy but significantly slows down generation speed.