NVIDIA - consumer GPU - verified
Local LLMs that run on NVIDIA GeForce RTX 5060 Ti 8GB (8 GB)
By this directory's rule of thumb, 13 of 63 open-weight models fit in the 8 GB of VRAM on the RTX 5060 Ti 8GB. The largest by total parameters is Qwen3.5-9B (9.7B), whose smallest non-MLX download is 5.7 GB, the Q4_K_M GGUF build from unsloth/Qwen3.5-9B-GGUF.
- Models that fit
- 13 of 63
- Memory
- 8 GB
- Memory type
- GDDR7
- Bandwidth
- 448 GB/s
Spec sheet
- Memory
- 8 GB
- Memory type
- GDDR7
- Unified memory
- No
- Bandwidth
- 448 GB/s
- Released
- 2025
128-bit memory interface; NVIDIA lists one 448 GB/sec bandwidth figure covering both the 16 GB and 8 GB versions.
Source: NVIDIA spec page (nvidia.com)
13 models that fit in 8 GB
Total params, largest first| Model | Total params | Smallest download | Quant | Commercial |
|---|---|---|---|---|
| Qwen3.5-9BAlibaba Cloud (Qwen) | 9.7B | 5.7 GB | Q4_K_MGGUF | Commercial use: Yes |
| Ministral 3 8B Instruct 2512Mistral AI | 8.9B | 5.2 GB | Q4_K_MGGUF | Commercial use: Yes |
| IBM Granite 4.2 8BIBM | 8.8B | 5.3 GB | Q4_K_MGGUF | Commercial use: Yes |
| Cohere Command R7B (12-2024)Cohere | 8B | 5.1 GB | Q4_K_MGGUF | Commercial use: No |
| Gemma 4 E4BGoogle | 8B | 5.0 GB | Q4_K_MGGUF | Commercial use: Yes |
| Gemma 4 E2BGoogle | 5.1B | 3.1 GB | Q4_K_MGGUF | Commercial use: Yes |
| Qwen3.5-4BAlibaba Cloud (Qwen) | 4.7B | 2.7 GB | Q4_K_MGGUF | Commercial use: Yes |
| Ministral 3 3B Instruct 2512Mistral AI | 3.8B | 2.1 GB | Q4_K_MGGUF | Commercial use: Yes |
| Phi-4-mini-instructMicrosoft | 3.8B | 2.5 GB | Q4_K_MGGUF | Commercial use: Yes |
| Phi-4-mini-reasoningMicrosoft | 3.8B | 2.5 GB | Q4_K_MGGUF | Commercial use: Yes |
| Liquid AI LFM2.5-2.6BLiquid AI | 2.7B | 1.7 GB | Q4_K_MGGUF | Commercial use: Conditions |
| Qwen3.5-2BAlibaba Cloud (Qwen) | 2.3B | 1.3 GB | Q4_K_MGGUF | Commercial use: Yes |
| Qwen3.5-0.8BAlibaba Cloud (Qwen) | 0.9B | 0.5 GB | Q4_K_MGGUF | Commercial use: Yes |
1 more model fits in 8 GB only as an MLX build, which runs on Apple silicon only: Gemma 4 12B.
The rest of the 63 need more than 8 GB. Browse every local model.
Compare all 33 GPUs and Macs by memory, bandwidth and how many models fit.
FAQ
How many local LLMs can the RTX 5060 Ti 8GB run?
13 of the 63 open-weight models in this directory fit, with 8 GB of VRAM, by this site's rule of thumb that the smallest non-MLX download is at most 85% of the memory. MLX builds do not count here: they run on Apple silicon only.
Can the RTX 5060 Ti 8GB run Qwen3.5-9B?
Yes, by this directory's rule of thumb: its smallest non-MLX download, 5.7 GB, is at most 85% of 8 GB (6.8 GB), the Q4_K_M GGUF build from unsloth/Qwen3.5-9B-GGUF. That leaves room for the context. It is a memory check, not a speed estimate.
Can the RTX 5060 Ti 8GB run IBM Granite 4.2 8B?
Yes, by this directory's rule of thumb: its smallest non-MLX download, 5.3 GB, is at most 85% of 8 GB (6.8 GB), the Q4_K_M GGUF build from ibm-granite/granite-4.2-8b-GGUF. That leaves room for the context. It is a memory check, not a speed estimate.
Can the RTX 5060 Ti 8GB run Gemma 4 12B?
Not by this directory's rule of thumb: its smallest non-MLX download, 7.0 GB, is more than 85% of 8 GB (6.8 GB). It needs the 16 GB tier or more: https://www.developersdigest.tech/local-models/fits/16.
Can the RTX 5060 Ti 8GB use system RAM for bigger models?
This page only counts models that fit entirely in the 8 GB of VRAM. Runtimes can offload layers to system RAM, but this directory does not list or measure that.
What memory bandwidth does the RTX 5060 Ti 8GB have?
NVIDIA states 448 GB/s of memory bandwidth (GDDR7). From the spec notes: 128-bit memory interface; NVIDIA lists one 448 GB/sec bandwidth figure covering both the 16 GB and 8 GB versions. This page does not turn bandwidth into a speed estimate.
Newsletter
New open weights, every week
One email per week. New model releases, open-weight drops and what they mean for what you build.
Keep exploring
Best local AI models
Our curated picks by hardware tier, with benchmarks and the reasoning.
Model API pricing and specs
First-party prices per 1M tokens when you would rather call an API.
AI benchmark leaderboards
Sourced scores, for when you need to compare quality.
Setup guides
Point your coding tools at the model you picked.