Challenging Neural Network Superposition with Direct Neuron Analysis
An influential essay by former OpenAI researcher Nick Cammarata challenges the dogma of neural network superposition, demonstrating through interactive visualizations that modern models can be understood by directly studying individual neurons. If valid, this discovery suggests that AI interpretability is far more tractable than previously thought, potentially allowing researchers to monitor model safety and detect deceptive reasoning by directly reading internal activations. The study analyzes modern large language models like Qwen3 32B and Kimi K3, estimating that roughly 90% of their neurons have a single primary function rather than being hopelessly entangled.
## BACKGROUND
Superposition is the hypothesis that neural networks pack multiple unrelated concepts into the same individual neurons to save dimensional space. Historically, this polysemantic behavior led researchers to rely heavily on auxiliary methods like sparse autoencoders to untangle models rather than analyzing neurons directly.