TAG

#throughput

Stories connected through a shared idea, actor, technology or theme.

20 OctSGLang 0.5.22 delivers faster model loading and a critical state-correctness fixSGLang 0.5.22 introduces an InstantTensor loader that maintainers say makes models load up to ten times faster, adds new model and diffusion support, and improves DeepSeek-V4.1 throughput on H200 GPUs. It also fixes a bug that could resume hybrid-linear models from incorrect Mamba states when speculative decoding and structured output were combined.→