LycheeMemory V2: Semantic Segment-Level Consolidation Slashes Long-Term Memory Construction Cost

Authors: Dongfang Li, Zixuan Liu, Junmai Wang, Jiahe Huang, Fuhao Li, Bonian Jia, Baotian Hu, Min Zhang

arXiv ID: 2608.12990

Problem: Long-horizon LLM agents must preserve information from past interactions, but existing memory systems consolidate eagerly - invoking an LLM after every interaction to extract, summarize, or update memories. Construction cost therefore grows with conversation length. Coarse summarization cuts cost but risks discarding fine-grained contextual evidence, while larger retrieval contexts or multi-hop reasoning push the overhead to query time.

Key Methodology:

Semantic segment-level consolidation: batches multiple exchanges into segments and encodes each finalized segment once into context-independent typed memory records, replacing per-turn eager consolidation
Semantic boundary detection preserves coherent event-level and temporal evidence compared with fixed-window batching
Lightweight structured indexes support query-planned evidence retrieval at query time
Evaluated with GPT-4.1-Mini on LoCoMo and LongMemEval-S, compared against A-Mem and other memory baselines

Key Results:

State of the art: 89.22% on LoCoMo and 92.20% on LongMemEval-S
Construction token reduction versus A-Mem: 86.0% on LoCoMo and 75.9% on LongMemEval-S
No increase in query-time token usage, so the savings do not migrate to the read path
The accuracy-cost trade-off depends on consolidation granularity, not only on what information is retained

What it means for developers: Memory construction is billed per consolidation call, so consolidation frequency is a real cost lever for agent platforms: per-turn eager consolidation is the expensive default, and batching to segment boundaries changes the cost curve without losing event-level evidence. The typed, context-independent record format also separates consolidation from query planning, which makes memory construction a pipeline stage that can be cached, versioned, and audited rather than a per-turn LLM side effect.

Paper: arXiv:2608.12990