UMEM: Unified Memory Extraction and Management Framework for Generalizable Memory

UMEM: Unified Memory Extraction and Management Framework for Generalizable Memory

This paper presents a systematic solution to a core bottleneck in self-evolving agents, offering the following notable contributions:

Core Problem Insight

The authors accurately identify a fundamental limitation in existing approaches: the decoupled treatment of memory extraction and memory management. Current state-of-the-art methods (e.g., ReMem, Memp) focus solely on optimizing management strategies while treating extraction as a static prompting process. This design leads to two critical pitfalls:

1. Instance-level noise accumulation: Blindly memorizing concrete details rather than generalizable principles results in a "rote memorization" trap.
2. Strategy misalignment: Low-quality extracted memories render even optimal management strategies ineffective, creating a "garbage in, garbage out" vicious cycle.

Methodological Innovations

UMEM's key innovation lies in the end-to-end joint optimization of extraction and management, with three pivotal designs ensuring generalizability:

• Semantic Neighborhood Modeling: Queries are clustered based on cosine similarity, transforming cross-task generalization into an intra-neighborhood consistency optimization problem. Theoretical guarantees for retrieval stability are provided via formal lemmas.
• Marginal Utility Reward: Memory value is evaluated at the neighborhood level through a novel reward combining success gain and efficiency regularization, compelling the agent to discard instance-specific noise.
• Online Memory Evolution: The memory bank is dynamically updated during training, enabling co-evolution of policy learning and memory states, thereby overcoming limitations imposed by static memory assumptions.

Comprehensive Experimental Validation

• UMEM achieves significant performance gains over baselines across five heterogeneous benchmarks (AIME, GPQA, ALFWorld, etc.), with improvements up to 10.67% on multi-turn interactive tasks.
• Ablation studies compellingly demonstrate that optimizing extraction alone is more critical than optimizing management alone (performance drop of 4.70 points vs. 0.73 points), challenging the prevailing "management-first" paradigm.
• Continual evolution experiments reveal that UMEM exhibits a monotonically improving trajectory, whereas baselines such as ReMem rapidly degrade due to noise accumulation—validating the long-term value of the generalization-oriented design.
• Cross-model transfer experiments (Qwen3 → GPT-5.1 → Gemini) confirm that extracted memories possess architecture-agnostic utility.

Broader Impact and Implications

This work redefines the optimization paradigm for self-evolving agents: memory quality hinges on the synergy between extraction and management, rather than maximal optimization of either component in isolation. The "neighborhood generalization" principle offers valuable insights for continual learning and experience transfer. Notably, the paper draws an analogy between agent evolution and neural network training (forward inference + backward optimization), providing a fresh perspective on understanding learning mechanisms in LLM-based agents.

Directions for Further Discussion

• The reliance on pretrained encoders for semantic neighborhood construction warrants deeper investigation into generalization boundaries under out-of-distribution tasks.
• Trade-offs between computational overhead (evaluating N neighborhood queries) and real-time deployment requirements merit further exploration.
• Potential complementarity with parameter-based continual learning approaches remains an open avenue for integration.

In summary, through rigorous problem deconstruction and an innovative joint optimization framework, UMEM establishes a vital pathway toward building truly sustainable self-evolving agents. It represents a substantive and well-grounded contribution to the field of memory-augmented agent research.