Skip to main content
Watch: I Built an AI App Without Leaving Databricks

NVIDIA - Mixture of experts - verified

Run NVIDIA Nemotron 3.5 Bolt 74B-A9B locally

NVIDIA Nemotron 3.5 Bolt 74B-A9B is a 74.9B mixture-of-experts (9B active) model from NVIDIA with a 262K-token context window, released under OpenMDW License 1.1 (commercial use is allowed); the smallest listed download is 22.0 GB, the 24GB (native 2-bit) GGUF build from nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF.

Official weights: nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 - modalities: text, image, video

Total params
74.9B
Active params
9B
Context
262K
Smallest download
22.0 GB

Fits in

Smallest download at most 85% of memory

Downloads

The official weights and every quantization listed for this model, with file sizes from Hugging Face.

NVIDIA Nemotron 3.5 Bolt 74B-A9B downloads and file sizes
DownloadFormatSizeRepo
Official weightsBF16149.8 GBnvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16
24GB (native 2-bit)Smallest listedGGUF22.0 GBnvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF
Q8_0GGUF78.9 GBunsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF

GB is decimal: 1 GB = 10^9 bytes. Add memory for the context and the operating system on top of the file size.

Run it

Copied verbatim from the page linked under each command. Server commands from model cards often assume a multi-GPU node; check the flags before you run them.

Ollama

Terminal
ollama run nemotron-3.5-bolt

Source: ollama.com/library/nemotron-3.5-bolt

vLLM

Terminal
wget https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16/resolve/main/ultra_v3_reasoning_parser.py

export VLLM_USE_V2_MODEL_RUNNER=1

vllm serve nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 \
  --trust-remote-code \
  --load-format auto \
  --tensor-parallel-size 1 \
  --max-model-len 262144 \
  --max-num-seqs 1 \
  --enable-chunked-prefill \
  --attention-backend FLASHINFER \
  --mamba-backend flashinfer \
  --mamba-cache-mode align \
  --seed 42 \
  --allowed-local-media-path / \
  --limit-mm-per-prompt '{"video":1,"image":1}' \
  --video-pruning-rate 0.5 \
  --media-io-kwargs '{"video":{"fps":2,"num_frames":256}}' \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser-plugin ultra_v3_reasoning_parser.py \
  --reasoning-parser ultra_v3

Source: huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16

Not sure which runtime to use? Compare local LLM runtimes by platform, API and features.

License

Commercial use
Commercial use: YesCommercial use is allowed.
Gated download
No. The files download directly.

Worth knowing

NVIDIA also ships a native 2-bit 24GB variant (vLLM and llama.cpp GGUF) built to fit in 24GB of memory, which is what the Ollama tag pulls.

Released (per the model card).

FAQ

How big is the NVIDIA Nemotron 3.5 Bolt 74B-A9B download?

The official BF16 weights in nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 are 149.8 GB. The smallest listed download is 22.0 GB, the 24GB (native 2-bit) GGUF build from nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF. Plan for more memory than the file size: the context (KV cache) and the operating system need room too.

Can I use NVIDIA Nemotron 3.5 Bolt 74B-A9B commercially?

NVIDIA Nemotron 3.5 Bolt 74B-A9B is released under OpenMDW License 1.1, and commercial use is allowed. The license text is at https://openmdw.ai/license/1-1/.

How much memory do I need for NVIDIA Nemotron 3.5 Bolt 74B-A9B?

By this directory's rule of thumb it fits in 32 GB and up: the smallest listed download, 22.0 GB, is at most 85% of 32 GB, which leaves room for the context and the operating system.

How do I run NVIDIA Nemotron 3.5 Bolt 74B-A9B with Ollama?

Run "ollama run nemotron-3.5-bolt", as shown at https://ollama.com/library/nemotron-3.5-bolt. The Ollama library lists the nemotron-3.5-bolt:74b-a9b tag at 25GB.

What is the context length of NVIDIA Nemotron 3.5 Bolt 74B-A9B?

262,144 tokens. A longer context needs more memory for the KV cache, so you can run with a shorter one when memory is tight.

Sources

Last verified from the pages below.

Newsletter

New open weights, every week

One email per week. New model releases, open-weight drops and what they mean for what you build.

Keep exploring

Best local AI models

Our curated picks by hardware tier, with benchmarks and the reasoning.

Model API pricing and specs

First-party prices per 1M tokens when you would rather call an API.

AI benchmark leaderboards

Sourced scores, for when you need to compare quality.

Setup guides

Point your coding tools at the model you picked.