Latent Space Podcast Features Inference Engineering Masterclass with Baseten
Latent Space has released an in-depth interview and masterclass featuring Philip Kiely and Ali Taha from Baseten, focusing on the systems engineering, optimization, and scaling challenges of AI inference. The session covers the technical complexities of running both autoregressive and diffusion models in production. As generative AI adoption grows, inference engineering has emerged as a critical discipline for reducing latency and infrastructure costs. Understanding these optimization techniques is essential for developers looking to deploy large-scale AI models efficiently. The masterclass details the hardware, software, and infrastructure required for production AI, highlighting the differences in serving autoregressive models versus diffusion models. It also highlights Baseten's role in the ecosystem, following their recent growth and focus on inference engineering.
## BACKGROUND
AI inference is the process of running live data through a trained machine learning model to generate outputs, such as text or images. Autoregressive models, like LLMs, generate outputs sequentially (one token at a time), while diffusion models generate data through a step-by-step denoising process, both requiring distinct optimization strategies.