DuoMem: Towards Capable On-Device Memory Agents via Dual-Space Distillation

Authors: Peyman Hosseini, Ondrej Bohdal, Ahmed Alajrami, Andrea Maracani, Ignacio Castro, Matthew Purver, Mete Ozay, Savas Ozkan, Taha Ceritli

arXiv ID: 2606.29961

Problem: Large LLM-based agents with procedural memory are too big and slow to run on resource-constrained edge devices.

Key Methodology

Context-space distillation: Replaces student-generated memories with higher-quality teacher (72B) procedural memories prepended to the student's input at inference time.
Parameter-space distillation: Fine-tunes lightweight LoRA adapters (<10M params) on successful teacher trajectories, adding only a few MB of pre-computed data.
Both distillation axes are combined and evaluated on ALFWorld, an embodied decision-making benchmark.

Key Results

4B student model boosted from 4.3% → 77.9% task success rate, closing the gap to the 72B teacher (87.1%).
DuoMem-enhanced 4B model runs >3x faster than the 72B teacher in wall-clock time.
Adds <10M trainable parameters and only a few MB of pre-computed teacher memories.

Applied Context: You can deploy capable memory-augmented agents on phones or IoT devices by distilling a large teacher into a compact student with LoRA adapters and cached procedural memories, sacrificing minimal accuracy for a massive latency and memory win.