Skip to main content
Watch: I Built an AI App Without Leaving Databricks

Mistral AI - Dense - verified

Run Mistral Medium 3.5 128B locally

Mistral Medium 3.5 128B is a 127.7B dense model from Mistral AI with a 262K-token context window, released under Modified MIT License (commercial use is allowed with conditions set out in the license); the smallest listed download is 74.9 GB, the Q4_K_M GGUF build from unsloth/Mistral-Medium-3.5-128B-GGUF.

Official weights: mistralai/Mistral-Medium-3.5-128B - modalities: text, image

Total params
127.7B
Architecture
Dense
Context
262K
Smallest download
74.9 GB

Fits in

Smallest download at most 85% of memory

Downloads

The official weights and every quantization listed for this model, with file sizes from Hugging Face.

Mistral Medium 3.5 128B downloads and file sizes
DownloadFormatSizeRepo
Official weightsFP8133.6 GBmistralai/Mistral-Medium-3.5-128B
Q4_K_MSmallest listedGGUF74.9 GBunsloth/Mistral-Medium-3.5-128B-GGUF
Q8_0GGUF132.9 GBunsloth/Mistral-Medium-3.5-128B-GGUF
4bitMLX78.0 GBmlx-community/Mistral-Medium-3.5-128B-4bit

GB is decimal: 1 GB = 10^9 bytes. Add memory for the context and the operating system on top of the file size.

Run it

Copied verbatim from the page linked under each command. Server commands from model cards often assume a multi-GPU node; check the flags before you run them.

Ollama

Terminal
ollama run mistral-medium-3.5

Source: ollama.com/library/mistral-medium-3.5

vLLM

Terminal
vllm serve mistralai/Mistral-Medium-3.5-128B --tensor-parallel-size 8 \
  --tool-call-parser mistral --enable-auto-tool-choice --reasoning-parser mistral --max_num_batched_tokens 16384 --max_num_seqs 128 \
  --gpu_memory_utilization 0.8

Source: huggingface.co/mistralai/Mistral-Medium-3.5-128B

SGLang

Terminal
python -m sglang.launch_server --model-path mistralai/Mistral-Medium-3.5-128B \
  --tp 8 --tool-call-parser mistral --reasoning-parser mistral

Source: huggingface.co/mistralai/Mistral-Medium-3.5-128B

Not sure which runtime to use? Compare local LLM runtimes by platform, API and features.

License

Commercial use
Commercial use: ConditionsCommercial use is allowed with conditions set out in the license.
Gated download
No. The files download directly.

Worth knowing

The modified MIT license withholds its rights from companies with global monthly revenue above 20 million USD, so larger businesses need a separate agreement.

Released (per the announcement).

API pricing and specsHosted prices per 1M tokens for Mistral Medium 3.5 128B, if you would rather not run it yourself.

FAQ

How big is the Mistral Medium 3.5 128B download?

The official FP8 weights in mistralai/Mistral-Medium-3.5-128B are 133.6 GB. The smallest listed download is 74.9 GB, the Q4_K_M GGUF build from unsloth/Mistral-Medium-3.5-128B-GGUF. Plan for more memory than the file size: the context (KV cache) and the operating system need room too.

Can I use Mistral Medium 3.5 128B commercially?

Mistral Medium 3.5 128B is released under Modified MIT License, and commercial use is allowed with conditions set out in the license. The license text is at https://huggingface.co/mistralai/Mistral-Medium-3.5-128B/blob/main/LICENSE.

How much memory do I need for Mistral Medium 3.5 128B?

By this directory's rule of thumb it fits in 128 GB and up: the smallest listed download, 74.9 GB, is at most 85% of 128 GB, which leaves room for the context and the operating system.

How do I run Mistral Medium 3.5 128B with Ollama?

Run "ollama run mistral-medium-3.5", as shown at https://ollama.com/library/mistral-medium-3.5. The Ollama library lists the mistral-medium-3.5:128b tag at 80GB.

What is the context length of Mistral Medium 3.5 128B?

262,144 tokens. A longer context needs more memory for the KV cache, so you can run with a shorter one when memory is tight.

Sources

Last verified from the pages below.

Newsletter

New open weights, every week

One email per week. New model releases, open-weight drops and what they mean for what you build.

Keep exploring

Best local AI models

Our curated picks by hardware tier, with benchmarks and the reasoning.

Model API pricing and specs

First-party prices per 1M tokens when you would rather call an API.

AI benchmark leaderboards

Sourced scores, for when you need to compare quality.

Setup guides

Point your coding tools at the model you picked.