TAG
#serving
Stories connected through a shared idea, actor, technology or theme.
SGLang 0.5.22 delivers faster model loading and a critical state-correctness fixSGLang 0.5.22 introduces an InstantTensor loader that maintainers say makes models load up to ten times faster, adds new model and diffusion support, and improves DeepSeek-V4.1 throughput on H200 GPUs. It also fixes a bug that could resume hybrid-linear models from incorrect Mamba states when speculative decoding and structured output were combined.→vLLM 0.31 adds faster restarts and broader large-scale serving supportvLLM 0.31 introduces a weight-cache daemon and snapshot create/restore workflows for faster engine recovery. The release also expands speculative decoding, large-scale expert-parallel serving and DeepSeek-V4.1-Flash optimisations.→