Inference
Active Parameters
The part of a model's parameters that is actually used to process each token.
In depth
The part of a model's parameters that is actually used to process each token. In a Mixture of Experts model, a router sends each token to only a few experts, so compute per token follows the active count while memory still has to hold every parameter. Mixtral 8x7B, for example, has 47B total parameters but uses only 13B active parameters per token. Names like Qwen3.6-35B-A3B carry both numbers: 35B total and 3B active.
Primary source: arxiv.org
Example
Mixtral 8x7B, for example, has 47B total parameters but uses only 13B active parameters per token.
Go deeper at Developers Digest
Hands-on guides, comparisons, and tutorials that cover Inference.
FAQ
Common questions
What is Active Parameters?
The part of a model's parameters that is actually used to process each token.
Why does Active Parameters matter for AI developers?
Active Parameters sits in the Inference part of the AI stack. Understanding it helps you make better decisions when building, debugging, and shipping AI features.
Where can I learn more about Active Parameters?
Developers Digest publishes tutorials and videos that cover Inference topics including Active Parameters. Check the blog and YouTube channel for hands-on walkthroughs.
Related
Related terms
Inference
Mixture of Experts (MoE)
A model architecture that routes each input to a small subset of specialized sub-networks ("experts") rather than activating the entire model.
Inference
Attention Mechanism
The core technique inside transformers that lets a model weigh the relevance of every token relative to every other token in a sequence.
Inference
State Space Models (SSM)
State space models are sequence models that carry a fixed-size hidden state forward through the input, updating it with learned linear equations, rather than comparing every token to every other token like attention.
Put this concept to work
Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
One email per weekReal code, not theoryFree forever