NVIDIA Open-Sources Switchyard to Route and Translate LLM Traffic Across Providers
NVIDIA's Switchyard repository has gained rapid popularity on GitHub, offering an open-source Rust proxy and library under the NeMo ecosystem that routes LLM traffic across multiple AI providers. The tool dynamically translates requests between OpenAI and Anthropic API formats while maintaining full compatibility for client applications. As developers build complex AI agents, Switchyard prevents vendor lock-in and eliminates the need to rewrite integration code when switching models. It enables engineering teams to dynamically balance model capability, latency, and API costs across different LLM backends. Switchyard provides both tuning-free and tunable routers to direct agent workloads based on operational metrics. Built in Rust with Python interoperability, it supports OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages protocols.
## BACKGROUND
Modern generative AI applications often rely on multiple large language models to handle distinct tasks based on quality, speed, or cost constraints. NVIDIA NeMo is an enterprise framework designed for building and deploying generative AI, and Switchyard expands this ecosystem by acting as an intelligent API translation layer and traffic router.