SenseNova-Loop-9B-Preview: A 9B Multimodal LLM with Looped Decoder Architecture
SenseNova has released SenseNova-Loop-9B-Preview, a 9B parameter multimodal language model featuring a looped decoder architecture, dual-stream manifold-constrained hyper-connections (mHC), and a 262k token context window. The model executes a contiguous block of 24 layers twice in sequence to achieve deeper computation without duplicating model parameters. This release represents a significant architectural exploration for open-weight models by combining efficient deep computation through parameter sharing with ultra-long context multimodal handling. Innovations like manifold-constrained hyper-connections help stabilize training and information flow in unconventional recurrent or looped transformer designs. The model uses hybrid attention interleaving linear and full attention in a four-layer pattern and supports multimodal inputs including text, images, code, and agentic tool trajectories. Its 56 total layer executions are derived from 32 distinct layers using shared looped blocks.
## BACKGROUND
Transformer-based language models traditionally rely on a straight-through stack of decoder layers, where adding depth requires adding unique parameters. Looped architectures instead reuse existing layers multiple times during a single forward pass, while manifold-constrained hyper-connections (mHC) manage parallel residual streams to prevent signal explosion or degradation.