Microsoft Briefly Leaks OpenAI's Use of Looped Transformers in GPT-6 Models
A briefly accessible Microsoft web page revealed that OpenAI is using looped transformers with multiple inference passes in its unreleased GPT-6 series models. The post specifically referenced a model variant named GPT-6.1 Sol that uses two inference passes instead of three, before Microsoft subsequently removed the content. If accurate, this confirms a major architectural shift toward parameter-efficient designs, allowing next-generation language models to achieve deeper reasoning capabilities by reusing network layers rather than just expanding parameter counts. It also provides empirical evidence backing long-standing rumors about OpenAI's architectural changes for future foundation models. Looped transformers repeatedly pass intermediate data through weight-tied transformer blocks to mimic deeper network reasoning with fewer parameters. According to the leaked post, variants like GPT-6 Sol and GPT-6.1 Sol share the same pre-trained base model weights but differ in post-training optimizations and the number of inference passes.
## BACKGROUND
Standard transformer architectures process input data through a fixed sequence of distinct layers in a single forward pass. In contrast, looped transformers repeatedly apply a shared set of transformer layers across depth, making the model significantly more parameter-efficient. This iterative design allows models to dynamically scale compute depth for complex algorithmic reasoning tasks without enlarging the model's footprint.