Initial upload of MnemoCore

dbb04e4 verified about 1 month ago

17.8 kB

	# MnemoCore Roadmap

	## Scope and Intent

	This roadmap describes current known gaps and likely direction.
	It is not a promise, delivery guarantee, or commitment to specific timelines.

	---

	## Version History

	\| Version \| Phase \| Status \| Key Features \|
	\|---------\|-------\|--------\|--------------\|
	\| 3.x \| Core Architecture \| ✅ Complete \| Binary HDV, 3-Tier Storage, LTP/Decay \|
	\| 4.0 \| Cognitive Enhancements \| ✅ Complete \| XOR Attention, Bayesian LTP, Gap Detection, Immunology \|
	\| 4.1 \| Observability \| ✅ Complete \| Prometheus metrics, distributed tracing, project isolation \|
	\| 4.2 \| Stability \| ✅ Complete \| Async lock fixes, test suite hardening \|
	\| 4.3 \| Temporal Recall \| ✅ Complete \| Episodic chaining, chrono-weighting, sequential context \|
	\| 5.x \| The Perfect Brain \| 🔮 Planned \| Multi-Modal, Emotional, Working Memory \|

	---

	## Phase 5.x: The Perfect Brain

	Vision: Transform MnemoCore from a sophisticated memory storage system into a truly cognitive architecture that functions as an artificial brain - but better.

	### 5.0 Multi-Modal Memory

	Goal: Enable storage and retrieval of images, audio, code structures, and cross-modal associations.

	```
	┌─────────────────────────────────────────────────────────────────┐
	│ CURRENT: Text-only encoding │
	│ ────────────────────────────────────────────────────────────── │
	│ store("User reported bug") → BinaryHDV │
	│ │
	│ FUTURE: Multi-modal encoding │
	│ ────────────────────────────────────────────────────────────── │
	│ store("Screenshot of error", image=bytes) → CrossModalHDV │
	│ store("Voice note", audio=bytes) → AudioHDV │
	│ bind(text_id, image_id, relation="illustrates") │
	│ │
	│ query("API error", modality="image") → screenshot.png │
	│ query(image=bytes, modality="text") → "Related conversation" │
	└─────────────────────────────────────────────────────────────────┘
	```

	Implementation Plan:

	\| Component \| Description \| Dependencies \|
	\|-----------\|-------------\|--------------\|
	\| `MultiModalEncoder` \| Abstract encoder protocol \| - \|
	\| `CLIPEncoder` \| Vision encoding via CLIP \| `transformers`, `torch` \|
	\| `WhisperEncoder` \| Audio encoding via Whisper \| `openai-whisper` \|
	\| `CodeEncoder` \| AST-aware code encoding \| `tree-sitter` \|
	\| `CrossModalBinding` \| VSA operations across modalities \| BinaryHDV \|

	New API Endpoints:
	```
	POST /store/multi - Store with multiple modalities
	POST /query/cross-modal - Cross-modal semantic search
	POST /bind - Bind modalities together
	GET /memory/{id}/related - Get cross-modal related memories
	```

	---

	### 5.1 Emotional/Affective Layer

	Goal: Enable emotion-weighted memory storage, retrieval, and decay - mimicking how biological memory prioritizes emotionally significant events.

	```
	┌─────────────────────────────────────────────────────────────────┐
	│ EMOTIONAL DIMENSIONS │
	│ ────────────────────────────────────────────────────────────── │
	│ │
	│ Valence: [-1.0 ──────────────── +1.0] │
	│ (negative/unpleasant) (positive/pleasant) │
	│ │
	│ Arousal: [0.0 ────────────────── 1.0] │
	│ (calm/neutral) (intense/urgent) │
	│ │
	│ EFFECT ON MEMORY: │
	│ ────────────────────────────────────────────────────────────── │
	│ High Arousal + Negative = "Flashbulb memory" (never forget) │
	│ High Arousal + Positive = Strong consolidation │
	│ Low Arousal = Faster decay (forgettable) │
	│ │
	└─────────────────────────────────────────────────────────────────┘
	```

	MemoryNode Extensions:
	```python
	@dataclass
	class MemoryNode:
	# ... existing fields ...

	# Phase 5.1: Emotional tagging
	emotional_valence: float = 0.0 # -1.0 (negative) to +1.0 (positive)
	emotional_arousal: float = 0.0 # 0.0 (calm) to 1.0 (intense)
	emotional_tags: List[str] = field(default_factory=list) # ["frustration", "joy", "urgency"]

	def emotional_weight(self) -> float:
	"""Calculate memory importance based on emotional factors."""
	# Arousal amplifies retention regardless of valence
	# High arousal creates "flashbulb memories"
	return abs(self.emotional_valence) * self.emotional_arousal
	```

	Modified LTP Formula:
	```
	S = I × log(1+A) × e^(-λT) × (1 + E)

	Where E = emotional_weight() ∈ [0, 1]
	```

	Use Cases:
	- B2B outreach: "Customer was almost in tears when we fixed their issue" → HIGH priority
	- Support tickets: "User furious about data loss" → Never forget, prioritize retrieval
	- Positive feedback: "User loved the new feature" → Moderate retention

	---

	### 5.2 Working Memory Layer

	Goal: Active cognitive workspace for goal-directed reasoning, not just passive storage.

	```
	┌─────────────────────────────────────────────────────────────────┐
	│ COGNITIVE ARCHITECTURE │
	│ │
	│ ┌─────────────────────────────────────────────────────────┐ │
	│ │ WORKING MEMORY (Active) │ │
	│ │ Capacity: 7 ± 2 items │ │
	│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
	│ │ │ Goal │ │ Context │ │ Focus │ │ Hold │ │ │
	│ │ │ │ │ │ │ │ │ │ │ │
	│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │
	│ └─────────────────────────────────────────────────────────┘ │
	│ ↕ │
	│ ┌─────────────────────────────────────────────────────────┐ │
	│ │ HOT TIER (Fast Access) │ │
	│ │ ~2,000 memories, <1ms access │ │
	│ └─────────────────────────────────────────────────────────┘ │
	│ ↕ │
	│ ┌─────────────────────────────────────────────────────────┐ │
	│ │ WARM TIER (Qdrant/Redis) │ │
	│ │ ~100,000 memories, <10ms access │ │
	│ └─────────────────────────────────────────────────────────┘ │
	│ ↕ │
	│ ┌─────────────────────────────────────────────────────────┐ │
	│ │ COLD TIER (Archive) │ │
	│ │ Unlimited, <100ms access │ │
	│ └─────────────────────────────────────────────────────────┘ │
	│ │
	└─────────────────────────────────────────────────────────────────┘
	```

	Working Memory API:
	```python
	# Create working memory instance
	wm = engine.working_memory(capacity=7)

	# Set active goal
	wm.set_goal("Troubleshoot authentication error")

	# Load relevant context
	wm.focus_on(await engine.query("auth error", top_k=5))

	# Hold important constraints
	wm.hold("User is on deadline - prioritize speed over elegance")

	# Query with working memory context
	results = wm.query("related issues")
	# Results are RE-RANKED based on current goal + focus + held items

	# Get context summary for LLM
	context = wm.context_summary()
	# → "Working on: auth troubleshooting
	# Focus: Recent OAuth errors
	# Constraint: Time pressure"
	```

	Implementation Components:
	\| Component \| Description \|
	\|-----------\|-------------\|
	\| `WorkingMemory` \| Active workspace class \|
	\| `GoalContext` \| Goal tracking and binding \|
	\| `FocusBuffer` \| Currently attended items \|
	\| `HoldBuffer` \| Constraints and important facts \|
	\| `ContextualQuery` \| Goal-directed retrieval \|

	---

	### 5.3 Multi-Agent / Collaborative Memory

	Goal: Enable memory sharing between agents while maintaining provenance and privacy.

	```
	┌─────────────────────────────────────────────────────────────────┐
	│ COLLABORATIVE MEMORY │
	│ │
	│ Agent A Shared Memory Agent B │
	│ ┌────────┐ ┌──────────────┐ ┌────────┐ │
	│ │ Private│ │ │ │ Private│ │
	│ │ Memory │◄────►│ Consensus │◄──────►│ Memory │ │
	│ └────────┘ │ Layer │ └────────┘ │
	│ │ │ │
	│ Agent C │ Provenance │ Agent D │
	│ ┌────────┐ │ Tracking │ ┌────────┐ │
	│ │ Private│◄────►│ │◄──────►│ Private│ │
	│ │ Memory │ │ Privacy │ │ Memory │ │
	│ └────────┘ │ Filtering │ └────────┘ │
	│ └──────────────┘ │
	│ │
	└─────────────────────────────────────────────────────────────────┘
	```

	Features:
	- Memory provenance: Track which agent created/modified each memory
	- Privacy levels: Private, shared-with-group, public
	- Conflict resolution: When agents disagree on facts
	- Collective intelligence: Aggregate insights across agents

	---

	### 5.4 Continual Learning

	Goal: Enable online adaptation without catastrophic forgetting.

	```
	┌─────────────────────────────────────────────────────────────────┐
	│ CONTINUAL LEARNING │
	│ │
	│ Traditional ML: Train → Deploy → (forget) → Retrain │
	│ │
	│ MnemoCore 5.4: Learn → Consolidate → Adapt → Learn → ... │
	│ ↑______________\| │
	│ │
	│ KEY MECHANISMS: │
	│ ───────────────────────────────────────────────────────────── │
	│ • Elastic Weight Consolidation (EWC) for encoder │
	│ • Replay-based consolidation during "sleep" cycles │
	│ • Progressive neural networks for new domains │
	│ • Meta-learning for rapid adaptation │
	│ │
	└─────────────────────────────────────────────────────────────────┘
	```

	---

	## Integration Priorities

	### Agent Frameworks
	\| Framework \| Priority \| Use Case \|
	\|-----------\|----------\|----------\|
	\| Open Claw \| ⭐⭐⭐⭐⭐ \| Primary use case, deep integration \|
	\| LangChain \| ⭐⭐⭐⭐ \| Memory provider plugin \|
	\| CrewAI \| ⭐⭐⭐⭐ \| Shared memory between agents \|
	\| AutoGen \| ⭐⭐⭐ \| Conversation memory backend \|
	\| LlamaIndex \| ⭐⭐⭐ \| Vector store adapter \|

	### AI Platforms
	\| Platform \| Priority \| Integration Type \|
	\|----------\|----------\|------------------\|
	\| Claude (Anthropic) \| ⭐⭐⭐⭐⭐ \| MCP server (existing) \|
	\| OpenAI Codex \| ⭐⭐⭐⭐⭐ \| API + function calling \|
	\| Ollama \| ⭐⭐⭐⭐ \| Native memory backend \|
	\| LM Studio \| ⭐⭐⭐ \| Plugin architecture \|
	\| Gemini \| ⭐⭐⭐ \| API adapter \|

	---

	## Research Opportunities

	### Academic Collaborations
	\| Area \| Institutions \| Relevance \|
	\|------\|-------------\|-----------\|
	\| Hyperdimensional Computing \| Stanford, IBM Research, Redwood Center \| Core HDC/VSA theory \|
	\| Computational Neuroscience \| MIT, UCL, KTH \| Biological validation \|
	\| Cognitive Architecture \| Carnegie Mellon, University of Michigan \| SOAR/ACT-R comparison \|
	\| Neuromorphic Computing \| Intel Labs, ETH Zürich \| Hardware acceleration \|

	### Publication Opportunities
	1. "Binary HDC for Long-term AI Memory" - Novel approach to persistent memory
	2. "Episodic Chaining in Vector Memory Systems" - Phase 4.3 temporal features
	3. "XOR Attention Masking for Memory Isolation" - Project isolation innovation
	4. "Bayesian LTP in Artificial Memory Systems" - Biological plausibility

	---

	## Known Gaps (Current Beta)

	- Query path is still primarily HOT-tier-centric in current engine behavior.
	- Some consolidation pathways are partial or under active refinement.
	- Certain integrations (LLM/Nightlab) are intentionally marked as TODO.
	- Distributed-scale behavior from long-form blueprints is not fully productized.

	---

	## Near-Term Priorities (Pre-5.0)

	1. Improve cross-tier retrieval consistency.
	2. Harden consolidation and archival flow.
	3. Improve deletion semantics and API consistency.
	4. Expand tests around degraded dependency modes (Redis/Qdrant outages).
	5. Stabilize API contracts and publish versioned compatibility notes.
	6. MCP server integration for agent tool access.

	---

	## Not a Commitment

	Items above are directional only.
	Order, scope, and implementation details can change during development.

	---

	Last Updated: 2025-02-18
	Current Version: 4.3.0