ATTENTION

Kimi Linear: An Attention Architecture That Outperforms Full Attention

Moonshot AI's Kimi Linear paper introduces KDA, a hybrid linear attention that beats full attention at all scales - 75% less KV cache, 6x decoding at 1M context, and open-source checkpoints.