Discovering Gemma-31B Latent Neurons and Exporting Steering Vectors for Llama.cpp
A researcher discovered that caring and contempt behaviors in the Gemma-31B model are controlled by the exact same 3 neurons within its latent space. They also shared a practical methodology to find custom steering recipes and export them into GGUF steering vectors compatible with llama.cpp. This discovery offers valuable insights into mechanistic interpretability, demonstrating how specific internal components of large language models directly govern complex psychological or behavioral traits. Furthermore, enabling custom steering vectors in llama.cpp empowers local AI enthusiasts to easily manipulate model behaviors without expensive fine-tuning. The post details how to isolate precise latent directions and convert them into control vectors that can be directly applied during local model inference. This bridges the gap between advanced mechanistic interpretability research and accessible local deployment tools like GGUF and llama.cpp.
## BACKGROUND
Mechanistic interpretability is a subfield of AI safety and research that aims to reverse-engineer neural networks by understanding what individual neurons or circuits do. Steering vectors are learned or discovered additive directions injected into a model's hidden states to reliably alter its output behavior without retraining the weights.