Metis: Memory Foundation Model

Authors: Zeyu Zhang, Ziliang Guo, Yihang Sun, Xichong Zhang, Xixuan Hao, Zehao Lin, Yang Zhang, Xiaoyan Zhao, Tong Shen, Bo Tang, Zhi-Qin John Xu, Junchi Yan, Haofen Wang, Xu Chen, Feiyu Xiong, Zhiyu Li, Tat-Seng Chua

arXiv ID: 2607.26760

Problem: Agent memory today is implemented almost entirely through external modules - vector stores, retrieval pipelines, scratchpads - while the native memory capability of the foundation model itself is left unexplored. Every tool-based memory design re-solves storage and retrieval outside the model, adding latency and architectural friction.

Key Methodology:

Formalizes native memory from two perspectives: a persistent and dynamically evolving memory state within the backbone, and native memory procedures that autonomously store and utilize information through model computation
New architecture that equips a foundation model with a native memory state, where historical information is compressed into the model and accessed through memory attention
Large-scale memory-specific training data and multiple optimization objectives acquire these native memory procedures through mid-training
Online memory maintenance is gradient-free: memory updates require only a forward pass, and at inference all learned weights stay frozen while the native memory state transforms through standard forward computation

Key Results:

Metis exhibits native memory capabilities across extensive experiments, with the paper providing a detailed analysis of its strengths, limitations, and behaviors
The architecture claims advantages in end-to-end optimization and efficiency compared with external-memory designs
Project and model checkpoints are released to facilitate research on memory foundation models

Applied Context: If in-model memory holds up in practice, the default agent-memory recipe changes: instead of bolting on retrieval over past transcripts, a model can compress history into its own state at forward-pass cost, with frozen weights. That would cut both the latency and the integration complexity of memory for long-running agents. Treat the specific results cautiously until independent evaluations land, but the capability class is worth watching.

Paper: arXiv:2607.26760