SGLang v0.5.20 Released with New Models and Inference Enhancements
SGLang v0.5.20 has been released, incorporating 713 pull requests from 237 contributors to add support for new architectures like GLM-5.3-Flash, Qwen3.8-Flash-Next, and FastH3. This update introduces sampling masks for reinforcement learning rollouts, unified radix tree branching-point caching, and a CPU-only simulation tool. The release improves inference efficiency for long-context and multi-turn workloads while accelerating RL training pipelines. Features like sliding-window attention branching-point caching substantially decrease time-to-first-token (TTFT) and improve memory reuse in large-scale deployments. Sampling masks with overlap scheduling boost Qwen3-8B decode throughput by up to 52% at batch size 64, while radix tree optimizations increase token hit rate from 43.8% to 60.8% on DeepSeek-V4-Flash. Additionally, in-memory response retention for `/v1/responses` is now opt-in via `--enable-response-store` to prevent unnecessary memory consumption.
## BACKGROUND
SGLang is an open-source, high-performance LLM serving engine designed for low-latency and high-throughput inference across individual GPUs and distributed clusters. It relies on advanced KV cache sharing algorithms like RadixAttention to accelerate prompt processing across complex generation workflows.