Intern-S2-Mobius: A Foundation Model Decoupling Knowledge and Reasoning
Researchers have introduced Mobius-v0, a novel model architecture that decouples knowledge storage (using a globally shared Feed-Forward Network memory) from reasoning (using multiple Self-Attention reasoners). The resulting Intern-S2-Mobius model, continually pretrained from Qwen3.5-35B, achieves a 4x end-to-end inference speedup while maintaining downstream performance. By separating knowledge from reasoning, this architecture significantly improves training data efficiency and inference speeds, addressing key bottlenecks in scaling large language models. It offers a promising path toward more resource-efficient AI systems that do not sacrifice reasoning capabilities. The architecture uses hidden states as a cache and carrier, allowing reasoners to repeatedly query the shared memory for knowledge vectors. A 7B model trained from scratch using Mobius-v0 matched the performance of a standard 7B Transformer baseline while requiring only 62.6% of the training data.
## BACKGROUND
In traditional Transformer architectures, self-attention layers (which handle context and reasoning) and feed-forward networks (FFNs, which store factual knowledge as key-value memories) are tightly coupled and interleaved. This entanglement often leads to inefficiencies, as scaling the model's knowledge base requires scaling the entire network, including the computationally expensive reasoning components.