WhaleYing Memory Structured long-term memory for AI agents, with explicit conflict resolution and multi-hop retrieval. WhaleYing Memory converts conversations and long documents into structured records, tracks which values supersede older versions, retrieves current evidence with hybrid search, and supplies that evidence to an LLM for answering. GitHub: https://github.com/jing-ying-dev/whale-ying-memory DOI: https://doi.org/10.5281/zenodo.24402955 License: Apache-2.0 Python: 3.10+ System Details WhaleYing Memory has four stages: Extractor — An LLM populates four tables: facts for assertions, cards for events, metrics for measurements, and identities for entities. Deterministic validation requires extracted quotations to be substrings of the source. Supersession and fold — Versions of the same subject form a supersession chain. Older values receive invalidated_at; folding retains the latest version by sequence. Hybrid retriever — Multi-field BM25 and all-MiniLM-L6-v2 vector retrieval are fused with reciprocal rank fusion (RRF). Database vectors are cached; the measured cache test produced an approximately 1,340× retrieval speedup. Iterative answerer — Optional Self-Ask-style iteration decomposes a multi-hop question, retrieves evidence for each subquestion, and assembles the final answer from the evidence chain. Multi-hop iteration is enabled with:

export WHALE_YING_ITERATIVE=1

The answer stage uses an OpenAI-compatible API. The current evaluations used DeepSeek, gpt-4o-mini, and a local Qwen2.5-Coder 7B as answer models. Intended Uses Long-term memory for AI agents. Question answering over conversations or long documents containing updated or conflicting facts. Retrieving the current value when several versions of a fact appear in the source history. Multi-hop question answering through iterative evidence retrieval. Local-first and privacy-oriented memory architectures in which extraction and retrieval are separated. MemoryAgentBench Results The following results use MemoryAgentBench · Conflict_Resolution and its substring exact match metric. They are for the current four-table extraction, supersession/folding, hybrid retrieval, and iterative-answering configuration. Single-hop / FactConsolidation-SH Context Score Answer model Runs Reported comparison 262k 93.2% gpt-4o-mini Mean of 5 GPT-4o 60 · BM25 56 · gpt-4o-mini 45 262k 93.0% DeepSeek, thinking disabled 1 — 262k 92.0% Local Qwen2.5-Coder 7B 1 — 6k 98% DeepSeek, thinking disabled 1 — Multi-hop / FactConsolidation-MH Context Score Answer model Runs CAR paper comparison at 262k 262k 48.3% DeepSeek, thinking disabled Mean of 3 BM25 3 · HippoRAG-v2 5 · Mem0 2 · MemGPT 3 · CAR with gpt-4o 41 262k 25.3% gpt-4o-mini Mean of 3 CAR with gpt-4o-mini 27 6k 66% substring / 65% exact DeepSeek, thinking disabled 1 — Answer-model temperature was 0.7 for these MemoryAgentBench runs. Direct full-context reading vs. WhaleYing Task at 262k Direct reading With WhaleYing Gain Single-hop 64% (DeepSeek) 93.2% (gpt-4o-mini) +29pp Multi-hop 5.3% (DeepSeek) 48.3% (DeepSeek) +43pp Single-hop scores remain in the same 92–93% band across gpt-4o-mini, DeepSeek, and a local Qwen2.5-Coder 7B answerer. Multi-hop scores are more dependent on the answer model: 48.3% with DeepSeek versus 25.3% with gpt-4o-mini. Single-hop baselines: MemoryAgentBench, arXiv:2507.05257. Multi-hop baselines and CAR: arXiv:2606.01435. LoCoMo Results LoCoMo was evaluated with the official F1 string-match metric and official QA prompt over 1,986 questions, using DeepSeek at temperature 0 with thinking disabled. Category Questions No folding (BM25 only) Full config (folding + BM25 + vector + RRF) Multi-hop 282 19.9% 20.4% Single-hop 321 28.3% 30.1% Temporal 96 13.9% 15.3% Open-domain 841 43.9% 38.9% Adversarial / abstention 446 53.4% 50.4% Overall F1 1,986 38.6% 36.3% The result identifies a task boundary in the current design. Conflict-resolution folding answers “what is true now?” by retaining the latest value. LoCoMo often asks about past states, where superseded values remain relevant; consequently, no folding scores higher overall. Temporal questions are the weakest category at 13.9–15.3% F1, making historical-state retrieval and relative-time normalization direct development targets. The LoCoMo figures are single runs. Its questions mostly depend on the most recent one or two sessions, so it probes temporal conversational QA more strongly than long-horizon retention. Measured Runtime For one complete 262k evaluation: Extraction: 1.1 million characters → 67 chunks → 870 pieces; approximately 11.5 minutes with DeepSeek-flash and eight concurrent requests. Answering: approximately 4 minutes for 100 single-hop questions and 15–25 minutes for 100 multi-hop questions. Input usage: approximately 3 million tokens for extraction and 700,000 tokens for answering 100 questions. Development and evaluation ran on Windows 11 with an Intel i5-13420H, 16 GB RAM, and no discrete GPU. Current Development Targets Querying historical states without discarding the current-state conflict-resolution path. Stronger normalization of relative and cross-event time. High-level memory for local answer models. A privacy-oriented deployment with one local extraction server and multiple lightweight retrieval clients. Easier standalone integration and reproducible evaluation. Quick Start

git clone https://github.com/jing-ying-dev/whale-ying-memory.git
cd whale-ying-memory
python -m pip install -r requirements.txt
export DEEPSEEK_API_KEY="your-api-key"
python extractor/extract.py \
  --chunk examples/sample_chunk.json \
  --out output/entities.json \
  --log-dir output

See the GitHub repository for architecture details, raw multi-hop result files, and the full extraction schema.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Papers for jing-ying-dev/WhaleYing-Memory