Xiaomi Open-Sources MiDashengLM-Spatial for End-to-End Spatial Audio Understanding
Xiaomi has open-sourced MiDashengLM-Spatial under the Apache 2.0 license, introducing the industry's first end-to-end spatial audio understanding large language model. Built on top of the MiDashengLM-7B base model, it integrates a hierarchical semantic-to-spatial conditioning (HSSC) module to perceive both audio content and its spatial distribution. This release marks a shift in audio AI from merely understanding semantic content to perceiving the physical environment and location of sounds. Such spatial intelligence is crucial for enhancing interactive experiences in smart home ecosystems and smart vehicle cockpits. The model fully releases model weights and inference code, featuring capabilities built through a hierarchical semantic-to-spatial conditioning module that separately captures and integrates semantic and spatial data. It is designed to integrate into Xiaomi's 'Human-Car-Home' ecosystem.
## BACKGROUND
Spatial audio understanding extends traditional audio AI by processing multi-channel signals to determine not just what is being said or played, but precisely where the sound originates in a 3D space. Multimodal large language models combine text, audio, and visual inputs to reason about complex real-world environments.