MOE
6 items
4 posts, 2 tools
Tencent Hy4 preview is a 770B open MoE with 49B active under Apache 2.0. Architecture, posted agentic benchmarks, OpenRouter pricing and self-hosting needs.
Tencent's Hy3 ships 295B parameters but activates only 21B per token, matching flagship performance at flash-tier pricing under Apache 2.0.
DeepSeek's open-weights frontier family, previewed April 24, 2026. V4-Pro is 1.6T total / 49B active params; V4-Flash is 284B / 13B. 1M context standard. Weights on Hugging Face.
Nemotron 3 Super developer guide: the latent MoE hybrid Mamba architecture, hardware needs, reasoning modes, and code to run NVIDIA's 120B model.
Alibaba's open-weight coding model, released in 2025. 480B total parameters, 35B active (MoE). Native 256K context, extends to 1M. Apache 2.0 license. Built for agentic coding.
NVIDIA Nemotron 3 Super combines latent mixture of experts with hybrid Mamba layers - 120B total parameters, 12B active per token, 1M context and open weights.

Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.