Skip to main content
Watch: I Built an AI App Without Leaving Databricks

AMD - data center GPU - verified

Local LLMs that run on AMD Instinct MI325X (256 GB)

By this directory's rule of thumb, 48 of 63 open-weight models fit in the 256 GB of GPU memory on the Instinct MI325X. The largest by total parameters is GLM-4.7 (358.3B), whose smallest non-MLX download is 216.5 GB, the Q4_K_M GGUF build from unsloth/GLM-4.7-GGUF.

Models that fit
48 of 63
Memory
256 GB
Memory type
HBM3E
Bandwidth
6,000 GB/s

Spec sheet

Memory
256 GB
Memory type
HBM3E
Unified memory
No
Bandwidth
6,000 GB/s
Released
2024

AMD states peak memory bandwidth as 6 TB/s (converted to GB/s by x1000) with an 8192-bit interface and launch date 10/10/2024.

Source: AMD spec page (amd.com)

48 models that fit in 256 GB

Total params, largest first
48 open-weight models that fit in the 256 GB Instinct MI325X, by total parameters, largest first
ModelTotal paramsSmallest downloadQuantCommercial
GLM-4.7Z.ai358.3B216.5 GBQ4_K_MGGUFCommercial use: Yes
GLM-4.6Z.ai356.8B215.6 GBQ4_K_MGGUFCommercial use: Yes
GLM-5.3-FlashZ.ai321.3B18B active199.7 GBUD-Q4_K_XLGGUFCommercial use: Yes
MiMo-V2.6-Flash-RLXiaomi MiMo310.8B15B active167.4 GBMXFP4GGUFCommercial use: Yes
Hy3Tencent298.8B21B active182.3 GBQ4_K_MGGUFCommercial use: Yes
Step 3.7 FlashStepFun201.4B11B active111.5 GBQ4_K_SGGUFCommercial use: Yes
Mistral Medium 3.5 128BMistral AI127.7B74.9 GBQ4_K_MGGUFCommercial use: Conditions
Devstral 2 123B Instruct 2512Mistral AI125B74.9 GBQ4_K_MGGUFCommercial use: Conditions
NVIDIA Nemotron 3.5 Super VL 120B-A12BNVIDIA124.5B12B active71.8 GBQ4_0GGUFCommercial use: Yes
Mistral Small 4 119B (2603)Mistral AI119.4B6.5B active73.8 GBUD-Q4_K_MGGUFCommercial use: Yes
gpt-oss-120bOpenAI116.8B5.1B active62.8 GBQ4_K_MGGUFCommercial use: Yes
Cohere Command A (03-2025)Cohere111.1B67.1 GBQ4_K_MGGUFCommercial use: No
GLM-4.5-AirZ.ai110.5B12B active73.0 GBQ4_K_MGGUFCommercial use: Yes
Llama 4 Scout 17B-16E InstructMeta108.6B17B active65.4 GBQ4_K_MGGUFCommercial use: Conditions
GLM-4.6VZ.ai107.7B70.6 GBQ4_K_MGGUFCommercial use: Yes
Qwen3-Coder-NextAlibaba Cloud (Qwen)79.7B3B active48.4 GBQ4_K_MGGUFCommercial use: Yes
NVIDIA Nemotron 3.5 Bolt 74B-A9BNVIDIA74.9B9B active22.0 GB24GB (native 2-bit)GGUFCommercial use: Yes
AI21 Jamba2 MiniAI21 Labs51.6B12B active31.2 GBQ4_K_MGGUFCommercial use: Yes
Qwen3.6-35B-A3BAlibaba Cloud (Qwen)36B3B active22.1 GBUD-Q4_K_MGGUFCommercial use: Yes
Cohere Aya Expanse 32BCohere32.3B19.8 GBQ4_K_MGGUFCommercial use: No
Ai2 Olmo 3.1 32B ThinkAi232.2B19.5 GBQ4_K_MGGUFCommercial use: Yes
NVIDIA Nemotron 3.5 Lightning 30B-A3BNVIDIA31.6B3B active18.9 GBQ4_0GGUFCommercial use: Yes
Gemma 4 31BGoogle31.3B17.7 GBQ4_0 (QAT)GGUFCommercial use: Yes
IBM Granite 4.2 30BIBM29.3B17.7 GBQ4_K_MGGUFCommercial use: Yes
Qwen3.8-27BAlibaba Cloud (Qwen)27.8B16.5 GBUD-Q4_K_MGGUFCommercial use: Yes
Gemma 4 26B A4BGoogle25.8B3.8B active14.4 GBQ4_0 (QAT)GGUFCommercial use: Yes
Devstral Small 2 24B Instruct 2512Mistral AI24B14.3 GBQ4_K_MGGUFCommercial use: Yes
Liquid AI LFM2.5-24B-A2BLiquid AI24B2.5B active14.5 GBQ4_K_MGGUFCommercial use: Conditions
gpt-oss-20bOpenAI20.9B3.6B active11.6 GBQ4_K_MGGUFCommercial use: Yes
Phi-4-reasoning-vision-15BMicrosoft15.1B30.2 GBBF16Official weightsCommercial use: Yes
Phi-4Microsoft14.7B9.1 GBQ4_KGGUFCommercial use: Yes
Phi-4-reasoning-plusMicrosoft14.7B9.1 GBQ4_K_MGGUFCommercial use: Yes
Ministral 3 14B Instruct 2512Mistral AI13.9B8.2 GBQ4_K_MGGUFCommercial use: Yes
Gemma 4 12BGoogle12B7.0 GBQ4_0 (QAT)GGUFCommercial use: Yes
Qwen3.5-9BAlibaba Cloud (Qwen)9.7B5.7 GBQ4_K_MGGUFCommercial use: Yes
Ministral 3 8B Instruct 2512Mistral AI8.9B5.2 GBQ4_K_MGGUFCommercial use: Yes
IBM Granite 4.2 8BIBM8.8B5.3 GBQ4_K_MGGUFCommercial use: Yes
Cohere Command R7B (12-2024)Cohere8B5.1 GBQ4_K_MGGUFCommercial use: No
Gemma 4 E4BGoogle8B5.0 GBQ4_K_MGGUFCommercial use: Yes
Ai2 Olmo Hybrid Instruct DPO 7BAi27.4B14.9 GBBF16Official weightsCommercial use: Yes
Gemma 4 E2BGoogle5.1B3.1 GBQ4_K_MGGUFCommercial use: Yes
Qwen3.5-4BAlibaba Cloud (Qwen)4.7B2.7 GBQ4_K_MGGUFCommercial use: Yes
Ministral 3 3B Instruct 2512Mistral AI3.8B2.1 GBQ4_K_MGGUFCommercial use: Yes
Phi-4-mini-instructMicrosoft3.8B2.5 GBQ4_K_MGGUFCommercial use: Yes
Phi-4-mini-reasoningMicrosoft3.8B2.5 GBQ4_K_MGGUFCommercial use: Yes
Liquid AI LFM2.5-2.6BLiquid AI2.7B1.7 GBQ4_K_MGGUFCommercial use: Conditions
Qwen3.5-2BAlibaba Cloud (Qwen)2.3B1.3 GBQ4_K_MGGUFCommercial use: Yes
Qwen3.5-0.8BAlibaba Cloud (Qwen)0.9B0.5 GBQ4_K_MGGUFCommercial use: Yes

The rest of the 63 need more than 256 GB. Browse every local model.

Compare all 33 GPUs and Macs by memory, bandwidth and how many models fit.

FAQ

How many local LLMs can the Instinct MI325X run?

48 of the 63 open-weight models in this directory fit, with 256 GB of GPU memory, by this site's rule of thumb that the smallest non-MLX download is at most 85% of the memory. MLX builds do not count here: they run on Apple silicon only.

Can the Instinct MI325X run Mistral Medium 3.5 128B?

Yes, by this directory's rule of thumb: its smallest non-MLX download, 74.9 GB, is at most 85% of 256 GB (217.6 GB), the Q4_K_M GGUF build from unsloth/Mistral-Medium-3.5-128B-GGUF. That leaves room for the context. It is a memory check, not a speed estimate.

Can the Instinct MI325X run NVIDIA Nemotron 3.5 Super VL 120B-A12B?

Yes, by this directory's rule of thumb: its smallest non-MLX download, 71.8 GB, is at most 85% of 256 GB (217.6 GB), the Q4_0 GGUF build from ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF. That leaves room for the context. It is a memory check, not a speed estimate.

Can the Instinct MI325X run Llama 4 Maverick 17B-128E Instruct?

Not by this directory's rule of thumb: its smallest non-MLX download, 242.8 GB, is more than 85% of 256 GB (217.6 GB). It needs more memory than any single device in this list.

Can the Instinct MI325X use system RAM for bigger models?

This page only counts models that fit entirely in the 256 GB of GPU memory. Runtimes can offload layers to system RAM, but this directory does not list or measure that.

What memory bandwidth does the Instinct MI325X have?

AMD states 6,000 GB/s of memory bandwidth (HBM3E). From the spec notes: AMD states peak memory bandwidth as 6 TB/s (converted to GB/s by x1000) with an 8192-bit interface and launch date 10/10/2024. This page does not turn bandwidth into a speed estimate.

Newsletter

New open weights, every week

One email per week. New model releases, open-weight drops and what they mean for what you build.

Keep exploring

Best local AI models

Our curated picks by hardware tier, with benchmarks and the reasoning.

Model API pricing and specs

First-party prices per 1M tokens when you would rather call an API.

AI benchmark leaderboards

Sourced scores, for when you need to compare quality.

Setup guides

Point your coding tools at the model you picked.