IBM - Dense - verified
Run IBM Granite 4.2 8B locally
IBM Granite 4.2 8B is a 8.8B dense model from IBM with a 131K-token context window, released under Apache 2.0 (commercial use is allowed); the smallest listed download is 4.9 GB, the 4bit MLX build from ibm-granite/granite-4.2-8b-q4-mlx.
Official weights: ibm-granite/granite-4.2-8b
- Total params
- 8.8B
- Architecture
- Dense
- Context
- 131K
- Smallest download
- 4.9 GB
Fits in
Smallest download at most 85% of memoryFits on: RTX 5060 Ti 8GB (8 GB), RTX 5070 (12 GB), RTX 5080 (16 GB), RTX 5070 Ti (16 GB), MacBook Air (M5) (32 GB) and MacBook Pro (M5) (32 GB) and 27 more of 33 devices at their largest memory (on NVIDIA and AMD, counting only non-MLX downloads).
Downloads
The official weights and every quantization listed for this model, with file sizes from Hugging Face.
| Download | Format | Size | Repo |
|---|---|---|---|
Official weights | BF16 | 17.6 GB | ibm-granite/granite-4.2-8b |
Q4_K_M | GGUF | 5.3 GB | ibm-granite/granite-4.2-8b-GGUF |
Q8_0 | GGUF | 9.3 GB | ibm-granite/granite-4.2-8b-GGUF |
4bitSmallest listed | MLX | 4.9 GB | ibm-granite/granite-4.2-8b-q4-mlx |
GB is decimal: 1 GB = 10^9 bytes. Add memory for the context and the operating system on top of the file size.
Run it
Copied verbatim from the page linked under each command. Server commands from model cards often assume a multi-GPU node; check the flags before you run them.
vLLM
vllm serve ibm-granite/granite-4.2-8b \
--served-model-name granite-4.2-8b \
--dtype bfloat16 \
--max-model-len 131072 \
--reasoning-parser granite_thinking_parser \
--reasoning-parser-plugin ./granite_thinking_parser.py \
--tool-call-parser qwen3_coder \
--enable-auto-tool-choiceSGLang
python3 -m sglang.launch_server \
--model-path ibm-granite/granite-4.2-8b \
--dtype bfloat16 \
--context-length 131072 \
--reasoning-parser auto \
--tool-call-parser autoNot sure which runtime to use? Compare local LLM runtimes by platform, API and features.
License
- License
- Apache 2.0
apache-2.0 - Commercial use
- Commercial use: YesCommercial use is allowed.
- Gated download
- No. The files download directly.
Worth knowing
Thinking is on by default (enable_thinking=False disables it), and the vLLM command loads granite_thinking_parser.py, which ships in the repo; the card lists 128K native context with extension to 512K.
Released (per the model card).
FAQ
How big is the IBM Granite 4.2 8B download?
The official BF16 weights in ibm-granite/granite-4.2-8b are 17.6 GB. The smallest listed download is 4.9 GB, the 4bit MLX build from ibm-granite/granite-4.2-8b-q4-mlx. Plan for more memory than the file size: the context (KV cache) and the operating system need room too.
Can I use IBM Granite 4.2 8B commercially?
IBM Granite 4.2 8B is released under Apache 2.0, and commercial use is allowed. The license text is at https://www.apache.org/licenses/LICENSE-2.0.
How much memory do I need for IBM Granite 4.2 8B?
By this directory's rule of thumb it fits in 8 GB and up: the smallest listed download, 4.9 GB, is at most 85% of 8 GB, which leaves room for the context and the operating system.
How do I run IBM Granite 4.2 8B with Ollama?
Run "ollama run granite4.2:8b", as shown at https://ollama.com/library/granite4.2:30b. The Ollama library lists the granite4.2:8b tag at 5.3GB.
What is the context length of IBM Granite 4.2 8B?
131,072 tokens. A longer context needs more memory for the KV cache, so you can run with a shorter one when memory is tight.
Sources
Last verified from the pages below.
- HF API (params, license, createdAt) - huggingface.co
- HF file tree - huggingface.co
- Model card - huggingface.co
- Ollama tags - ollama.com
- IBM GGUF tree - huggingface.co
- IBM MLX tree - huggingface.co
Newsletter
New open weights, every week
One email per week. New model releases, open-weight drops and what they mean for what you build.
Keep exploring
Best local AI models
Our curated picks by hardware tier, with benchmarks and the reasoning.
Model API pricing and specs
First-party prices per 1M tokens when you would rather call an API.
AI benchmark leaderboards
Sourced scores, for when you need to compare quality.
Setup guides
Point your coding tools at the model you picked.