Local runtime - SGLang project (LMSYS) - verified
SGLang: install, API, platforms and features
SGLang is a local LLM runtime from SGLang project (LMSYS). As its docs describe it: A high-performance serving framework for large language and multimodal models, designed for low-latency, high-throughput inference from a single GPU up to large distributed clusters. The latest release is v0.5.21, published October 2, 2026. License: Apache-2.0 (open source).
- Latest release
- v0.5.21
- Released
- 2026-10-02
- License
- Open source
- OpenAI API
- Yes
Install
Copied verbatim from the page linked under each command. Labels are worded as the source words them.
linux (NVIDIA CUDA 13, uv)
uv pip install --prerelease=allow sglangdocker (Linux, NVIDIA GPU)
docker pull lmsysorg/sglang:latestRun a model
The first-run example from the official docs. OpenAI-compatible endpoint: http://localhost:30000/v1 (port 30000 as used in the official quickstart: sglang serve MODEL_PATH --host 0.0.0.0 --port 30000)
sglang serve MODEL_PATH --host 0.0.0.0 --port 30000Features
What the SGLang docs say it supports. Not stated means the docs do not say either way.
| Feature | Value | Source |
|---|---|---|
| Interface | CLI, Server, Docker | See sources below |
| Platforms | Linux, macOS | docs.sglang.io/docs/hardware-platforms/apple_metal |
| Backends | CUDA, ROCm, CPU, TPU, NPU, Metal, Other | docs.sglang.io/docs/hardware-platforms/overview |
| Model formats | Safetensors, GGUF, MLX | docs.sglang.io/docs/advanced_features/server_arguments |
| OpenAI-compatible API | Yeshttp://localhost:30000/v1 (port 30000 as used in the official quickstart: sglang serve MODEL_PATH --host 0.0.0.0 --port 30000) | docs.sglang.io/docs/basic_usage/openai_api_completions |
| Anthropic-compatible API | Yes | docs.sglang.io/docs/basic_usage/anthropic_api |
| Tool calling | Yes | docs.sglang.io/docs/advanced_features/tool_parser |
| Structured outputs | Yes | docs.sglang.io/docs/advanced_features/structured_outputs |
| Vision (image input) | Yes | docs.sglang.io/docs/basic_usage/openai_api_vision |
| Multi-GPU | Yes | docs.sglang.io/docs/advanced_features/server_arguments |
| MCP client | Not stated | Docs do not say |
| License | Apache-2.0Open source | github.com/sgl-project/sglang/blob/main/LICENSE |
| Latest release | v0.5.21October 2, 2026 | github.com/sgl-project/sglang/releases/tag/v0.5.21 |
Models to run with SGLang
19 of the open-weight models in our local models directory list a command for it on their official pages:
Compare SGLang
FAQ
Does SGLang have an OpenAI-compatible API?
Yes. Endpoint, per its docs: http://localhost:30000/v1 (port 30000 as used in the official quickstart: sglang serve MODEL_PATH --host 0.0.0.0 --port 30000). Source: https://docs.sglang.io/docs/basic_usage/openai_api_completions
Does SGLang have an Anthropic-compatible API?
Yes, per https://docs.sglang.io/docs/basic_usage/anthropic_api
Which platforms and GPUs does SGLang support?
Its docs list Linux and macOS, with these backends: CUDA, ROCm, CPU, TPU, NPU, Metal and Other.
How do I install SGLang?
On linux (NVIDIA CUDA 13, uv): "uv pip install --prerelease=allow sglang", as shown at https://docs.sglang.io/docs/get-started/install. This page lists 2 install commands: linux (NVIDIA CUDA 13, uv) and docker (Linux, NVIDIA GPU).
Is SGLang open source?
Yes. It is released under Apache-2.0; the license is at https://github.com/sgl-project/sglang/blob/main/LICENSE.
What is the latest version of SGLang?
v0.5.21, published October 2, 2026 (https://github.com/sgl-project/sglang/releases/tag/v0.5.21), as of October 18, 2026.
Which model formats does SGLang run?
Safetensors, GGUF and MLX, per https://docs.sglang.io/docs/advanced_features/server_arguments
Sources
Last verified from the pages below.
- GitHub README - github.com
- GitHub Releases - github.com
- Docs home - docs.sglang.io
- Installation - docs.sglang.io
- Quickstart - docs.sglang.io
- OpenAI APIs - Completions - docs.sglang.io
- OpenAI APIs - Vision - docs.sglang.io
- Anthropic-Compatible API - docs.sglang.io
- Tool Parser - docs.sglang.io
- Structured Outputs - docs.sglang.io
- Server Arguments - docs.sglang.io
- Hardware Platforms - docs.sglang.io
- Apple Silicon with Metal - docs.sglang.io
Newsletter
Local AI, every week
One email per week. New model releases, runtime updates and what they mean for what you build.
Keep exploring
Open-weight models you can run
Download sizes, licenses and the run command for each model.
All local LLM runtimes
Every runtime in one feature matrix, with sources.
Picking a runtime for coding agents
Ollama, LM Studio, vLLM and llama.cpp for agent workloads.
Setup guides
Point your coding tools at the model you picked.