NVIDIA - workstation GPU - verified
Local LLMs that run on NVIDIA RTX PRO 5000 Blackwell (up to 72 GB)
By this directory's rule of thumb, 33 of 63 open-weight models fit in the 72 GB of VRAM on the RTX PRO 5000 Blackwell in its largest configuration. The largest by total parameters is Qwen3-Coder-Next (79.7B), whose smallest non-MLX download is 48.4 GB, the Q4_K_M GGUF build from Qwen/Qwen3-Coder-Next-GGUF.
- Models that fit
- 33 of 63
- Memory
- Up to 72 GB
- Memory type
- GDDR7 ECC
- Bandwidth
- 1,344 GB/s
Spec sheet
- Memory
- Up to 72 GB
- Configurations
- 48 or 72 GB
- Memory type
- GDDR7 with ECC
- Unified memory
- No
- Bandwidth
- 1,344 GB/s
Offered in 48 GB and 72 GB GDDR7 with ECC configurations, with 1,344 GB/sec memory bandwidth listed on the spec page.
Source: NVIDIA spec page (nvidia.com)
33 models that fit in 72 GB
Total params, largest first| Model | Total params | Smallest download | Quant | Commercial |
|---|---|---|---|---|
| Qwen3-Coder-NextAlibaba Cloud (Qwen) | 79.7B3B active | 48.4 GB | Q4_K_MGGUF | Commercial use: Yes |
| NVIDIA Nemotron 3.5 Bolt 74B-A9BNVIDIA | 74.9B9B active | 22.0 GB | 24GB (native 2-bit)GGUF | Commercial use: Yes |
| AI21 Jamba2 MiniAI21 Labs | 51.6B12B active | 31.2 GB | Q4_K_MGGUF | Commercial use: Yes |
| Qwen3.6-35B-A3BAlibaba Cloud (Qwen) | 36B3B active | 22.1 GB | UD-Q4_K_MGGUF | Commercial use: Yes |
| Cohere Aya Expanse 32BCohere | 32.3B | 19.8 GB | Q4_K_MGGUF | Commercial use: No |
| Ai2 Olmo 3.1 32B ThinkAi2 | 32.2B | 19.5 GB | Q4_K_MGGUF | Commercial use: Yes |
| NVIDIA Nemotron 3.5 Lightning 30B-A3BNVIDIA | 31.6B3B active | 18.9 GB | Q4_0GGUF | Commercial use: Yes |
| Gemma 4 31BGoogle | 31.3B | 17.7 GB | Q4_0 (QAT)GGUF | Commercial use: Yes |
| IBM Granite 4.2 30BIBM | 29.3B | 17.7 GB | Q4_K_MGGUF | Commercial use: Yes |
| Qwen3.8-27BAlibaba Cloud (Qwen) | 27.8B | 16.5 GB | UD-Q4_K_MGGUF | Commercial use: Yes |
| Gemma 4 26B A4BGoogle | 25.8B3.8B active | 14.4 GB | Q4_0 (QAT)GGUF | Commercial use: Yes |
| Devstral Small 2 24B Instruct 2512Mistral AI | 24B | 14.3 GB | Q4_K_MGGUF | Commercial use: Yes |
| Liquid AI LFM2.5-24B-A2BLiquid AI | 24B2.5B active | 14.5 GB | Q4_K_MGGUF | Commercial use: Conditions |
| gpt-oss-20bOpenAI | 20.9B3.6B active | 11.6 GB | Q4_K_MGGUF | Commercial use: Yes |
| Phi-4-reasoning-vision-15BMicrosoft | 15.1B | 30.2 GB | BF16Official weights | Commercial use: Yes |
| Phi-4Microsoft | 14.7B | 9.1 GB | Q4_KGGUF | Commercial use: Yes |
| Phi-4-reasoning-plusMicrosoft | 14.7B | 9.1 GB | Q4_K_MGGUF | Commercial use: Yes |
| Ministral 3 14B Instruct 2512Mistral AI | 13.9B | 8.2 GB | Q4_K_MGGUF | Commercial use: Yes |
| Gemma 4 12BGoogle | 12B | 7.0 GB | Q4_0 (QAT)GGUF | Commercial use: Yes |
| Qwen3.5-9BAlibaba Cloud (Qwen) | 9.7B | 5.7 GB | Q4_K_MGGUF | Commercial use: Yes |
| Ministral 3 8B Instruct 2512Mistral AI | 8.9B | 5.2 GB | Q4_K_MGGUF | Commercial use: Yes |
| IBM Granite 4.2 8BIBM | 8.8B | 5.3 GB | Q4_K_MGGUF | Commercial use: Yes |
| Cohere Command R7B (12-2024)Cohere | 8B | 5.1 GB | Q4_K_MGGUF | Commercial use: No |
| Gemma 4 E4BGoogle | 8B | 5.0 GB | Q4_K_MGGUF | Commercial use: Yes |
| Ai2 Olmo Hybrid Instruct DPO 7BAi2 | 7.4B | 14.9 GB | BF16Official weights | Commercial use: Yes |
| Gemma 4 E2BGoogle | 5.1B | 3.1 GB | Q4_K_MGGUF | Commercial use: Yes |
| Qwen3.5-4BAlibaba Cloud (Qwen) | 4.7B | 2.7 GB | Q4_K_MGGUF | Commercial use: Yes |
| Ministral 3 3B Instruct 2512Mistral AI | 3.8B | 2.1 GB | Q4_K_MGGUF | Commercial use: Yes |
| Phi-4-mini-instructMicrosoft | 3.8B | 2.5 GB | Q4_K_MGGUF | Commercial use: Yes |
| Phi-4-mini-reasoningMicrosoft | 3.8B | 2.5 GB | Q4_K_MGGUF | Commercial use: Yes |
| Liquid AI LFM2.5-2.6BLiquid AI | 2.7B | 1.7 GB | Q4_K_MGGUF | Commercial use: Conditions |
| Qwen3.5-2BAlibaba Cloud (Qwen) | 2.3B | 1.3 GB | Q4_K_MGGUF | Commercial use: Yes |
| Qwen3.5-0.8BAlibaba Cloud (Qwen) | 0.9B | 0.5 GB | Q4_K_MGGUF | Commercial use: Yes |
2 more models fit in 72 GB only as an MLX build, which runs on Apple silicon only: GLM-4.5-Air and Llama 4 Scout 17B-16E Instruct.
The rest of the 63 need more than 72 GB. Browse every local model.
By memory configuration
NVIDIA sells the RTX PRO 5000 Blackwell with 48 or 72 GB. The same rule applied to each configuration, and the models each step up adds.
| Configuration | Models that fit | Largest that fits | New at this size |
|---|---|---|---|
| 48 GB | 32 / 63 | NVIDIA Nemotron 3.5 Bolt 74B-A9B | Base configuration: 32 models |
| 72 GB | 33 / 63 | Qwen3-Coder-Next | Qwen3-Coder-Next |
Compare all 33 GPUs and Macs by memory, bandwidth and how many models fit.
FAQ
How many local LLMs can the RTX PRO 5000 Blackwell run?
33 of the 63 open-weight models in this directory fit in its largest configuration, 72 GB, by this site's rule of thumb that the smallest non-MLX download is at most 85% of the memory. By configuration: 32 at 48 GB and 33 at 72 GB. MLX builds do not count here: they run on Apple silicon only.
Can the RTX PRO 5000 Blackwell run Qwen3-Coder-Next?
Yes, by this directory's rule of thumb: its smallest non-MLX download, 48.4 GB, is at most 85% of 72 GB (61.2 GB), the Q4_K_M GGUF build from Qwen/Qwen3-Coder-Next-GGUF. That leaves room for the context. It is a memory check, not a speed estimate.
Can the RTX PRO 5000 Blackwell run NVIDIA Nemotron 3.5 Bolt 74B-A9B?
Yes, by this directory's rule of thumb: its smallest non-MLX download, 22.0 GB, is at most 85% of 72 GB (61.2 GB), the 24GB (native 2-bit) GGUF build from nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF. That leaves room for the context. It is a memory check, not a speed estimate.
Can the RTX PRO 5000 Blackwell run gpt-oss-120b?
Not by this directory's rule of thumb: its smallest non-MLX download, 62.8 GB, is more than 85% of 72 GB (61.2 GB). It needs the 128 GB tier or more: https://www.developersdigest.tech/local-models/fits/128.
Can the RTX PRO 5000 Blackwell use system RAM for bigger models?
This page only counts models that fit entirely in the 72 GB of VRAM. Runtimes can offload layers to system RAM, but this directory does not list or measure that.
What memory bandwidth does the RTX PRO 5000 Blackwell have?
NVIDIA states 1,344 GB/s of memory bandwidth (GDDR7 with ECC). From the spec notes: Offered in 48 GB and 72 GB GDDR7 with ECC configurations, with 1,344 GB/sec memory bandwidth listed on the spec page. This page does not turn bandwidth into a speed estimate.
Newsletter
New open weights, every week
One email per week. New model releases, open-weight drops and what they mean for what you build.
Keep exploring
Best local AI models
Our curated picks by hardware tier, with benchmarks and the reasoning.
Model API pricing and specs
First-party prices per 1M tokens when you would rather call an API.
AI benchmark leaderboards
Sourced scores, for when you need to compare quality.
Setup guides
Point your coding tools at the model you picked.