AIIT-Threshold commited on
Commit
2b99fec
·
verified ·
1 Parent(s): 11ce13f

kokoro-memory 1.0.0

Browse files
Files changed (6) hide show
  1. .gitignore +8 -0
  2. LICENSE +21 -0
  3. README.md +78 -0
  4. kokoro_memory.py +1767 -0
  5. pyproject.toml +28 -0
  6. tests/test_kokoro_memory.py +99 -0
.gitignore ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ __pycache__/
2
+ *.pyc
3
+ *.egg-info/
4
+ dist/
5
+ build/
6
+ .venv/
7
+ venv/
8
+ .pytest_cache/
LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Rhet Wike
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
README.md ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ - ja
6
+ tags:
7
+ - memory
8
+ - agent
9
+ - llm
10
+ - local-ai
11
+ - companion
12
+ - recall
13
+ - resonance
14
+ - spreading-activation
15
+ - coherence
16
+ ---
17
+
18
+ # kokoro-memory
19
+
20
+ **心の記憶 — file-based resonance memory for local AI companions**, extracted from the live memory stack of a fully local companion that ran on it daily for months. Every fact is a single JSON file on disk; recall is spreading activation through a bilingual Japanese/English synonym web, not keyword lookup. No database, no vector-DB server, no cloud — and zero required dependencies (pure Python stdlib; embeddings optional).
21
+
22
+ Code is mirrored on GitHub: https://github.com/AIIT-GLITCH/kokoro-memory
23
+
24
+ ## Design
25
+
26
+ - **Ten linguistic categories** named by Japanese parts of speech and concepts — 動詞 verbs, 形容詞 adjectives, 名詞 nouns, 副詞 adverbs, 関係 relationships, 出来事 events, 心 identity, 夢 aspirations, 真実 truths, 感覚 sensations. Identity is not the same kind of thing as an event; the store's shape says so.
27
+ - **Resonance recall, grounded.** A query activates synonym-web nodes; activation spreads; facts score by relevance×5 + salience (coherence, confidence, emotion weight, recency). Production fix baked in: relevance *dominates*, so specific questions beat heavy "core" memories.
28
+ - **Coherence scoring (v2).** Each fact's coherence = embedding connectedness to the existing store (all-MiniLM-L6-v2, CPU-only, mean top-8 cosine), with a token-spam degeneracy penalty. Fails open to a keyword scorer — a scoring bug must never block a memory write. The math follows the Wike Coherence Law, C = C₀·exp(−α·γ_eff).
29
+ - **Authenticator pipeline on every write:** junk filter → bracket-scaffolding/token-salad detector → exact + semantic dedupe → coherence → optional identity guard.
30
+ - **Public writes can't spoof the owner:** force-labeled source and authority class, `trusted_by_default=False`, and PII from public surfaces (emails, phones, API keys) is quarantined, never stored.
31
+ - **Local-model consolidation:** wire your own model via `set_generate_fn()`; session-end consolidation extracts durable facts + a one-line episode. The cloud-extraction path was deliberately deleted.
32
+
33
+ ## Limitations — stated honestly
34
+
35
+ - Scars from production are features here, but the tuning is one household's: recall weights, junk lists, and the salad detector were tuned on one companion's store (~thousands of facts), not on a public benchmark suite.
36
+ - In the live stack (Qwen2.5-14B + this memory) the companion scored **55.2% on LongMemEval**; an isolated memory-on/memory-off ablation of that number had not been run at release time, so treat it as a stack result, not a kokoro result.
37
+ - The synonym web ships with a compact core (~50 concepts); breadth comes from letting it grow (`add_synonym`) in use.
38
+ - Semantic dedupe is word-overlap based (embeddings are reserved for coherence); very short facts dedupe only exactly.
39
+ - English and Japanese are first-class; other languages pass through but get no synonym-web resonance.
40
+
41
+ ## Quickstart
42
+
43
+ ```python
44
+ import kokoro_memory as km
45
+
46
+ km.add_fact("truth", "favorite_fruit", "the favorite fruit is a crisp apple",
47
+ source="user_explicit", confidence=0.9)
48
+ km.recall("what's my favorite fruit?") # grounded resonance recall
49
+ km.recall("魂") # Japanese queries work
50
+ print(km.build_startup_memory_block()) # inject into your system prompt
51
+ ```
52
+
53
+ Config via env: `KOKORO_MEMORY_ROOT` (default `~/.kokoro/memory`), `KOKORO_OWNER_NAME`, `KOKORO_AGENT_NAME`, `KOKORO_STARTUP_INCLUDE_RAW`.
54
+
55
+ ```bash
56
+ python kokoro_memory.py # self-demo
57
+ python -m pytest tests -q # 10 unit tests, stdlib-only
58
+ ```
59
+
60
+ ## Provenance
61
+
62
+ Written by Rhet Dillard Wike (AIIT-THRESHOLD, Council Hill, Oklahoma) as the memory of **Buddy**, a fully local AI companion on a single RTX 3090. Released alongside [voice2](https://huggingface.co/AIIT-Threshold/voice2) (his voice) and [Tessera-1B](https://huggingface.co/AIIT-Threshold/Tessera-1B) (open-weights model) as part of AIIT-THRESHOLD's open stack. The creed: memory is resonance; the store must be human-readable; and nothing ever calls a cloud.
63
+
64
+ ## License
65
+
66
+ MIT © 2026 Rhet Dillard Wike, AIIT-THRESHOLD, Oklahoma.
67
+
68
+ ## Citation
69
+
70
+ ```bibtex
71
+ @software{wike2026kokoro,
72
+ author = {Wike, Rhet Dillard},
73
+ title = {kokoro-memory: file-based resonance memory for local AI companions},
74
+ year = {2026},
75
+ url = {https://github.com/AIIT-GLITCH/kokoro-memory},
76
+ note = {AIIT-THRESHOLD}
77
+ }
78
+ ```
kokoro_memory.py ADDED
@@ -0,0 +1,1767 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Kokoro Memory System — 心の記憶
3
+ The Most Advanced AI Memory Ever Written
4
+ Rhet Dillard Wike | AIIT-THRESHOLD | Council Hill, Oklahoma
5
+
6
+ 心 (kokoro) = mind + heart + soul, unified.
7
+ Memory is not a database. Memory is resonance.
8
+ Store in Japanese. Recall by feeling. Return with data.
9
+
10
+ Architecture:
11
+ - Linguistic categorization (verbs, adjectives, nouns, etc.)
12
+ - Japanese synonym webs for resonance-based recall
13
+ - Coherence scoring tied to the Wike Coherence Law
14
+ - Fact authentication pipeline (junk, duplicate, PII, and salad gates)
15
+ - Spreading activation through synonym networks
16
+ - Bilingual storage: Japanese primary, English secondary
17
+
18
+ Folder structure: $KOKORO_MEMORY_ROOT (default ~/.kokoro/memory/)
19
+ 動詞/ (doushi) — verbs: actions, processes, state changes
20
+ 形容詞/ (keiyoushi) — adjectives: qualities, properties, descriptions
21
+ 名詞/ (meishi) — nouns: entities, objects, concepts, people
22
+ 副詞/ (fukushi) — adverbs: manner, degree, time, frequency
23
+ 関係/ (kankei) — relationships: links between entities
24
+ 出来事/ (dekigoto) — events: timestamped occurrences
25
+ 心/ (kokoro) — identity: core self, beliefs, soul
26
+ 夢/ (yume) — aspirations: goals, projects, visions
27
+ 真実/ (shinjitsu) — truths: verified facts, proven data
28
+ 感覚/ (kankaku) — sensations: emotional states, resonance logs
29
+ episodes/ — session summaries (compressed past)
30
+ raw/ — recent raw exchanges
31
+ """
32
+
33
+ import json
34
+ import os
35
+ import re
36
+ import hashlib
37
+ import threading
38
+ import time
39
+ import math
40
+ from datetime import datetime, timedelta
41
+ from typing import Any, Dict, List, Optional, Tuple, Set
42
+
43
+
44
+ # ==========================================
45
+ # PATHS
46
+ # ==========================================
47
+ MEMORY_ROOT = os.path.expanduser(
48
+ os.environ.get("KOKORO_MEMORY_ROOT", "~/.kokoro/memory"))
49
+ RAW_TURNS_FILE = os.path.join(MEMORY_ROOT, "raw", "recent_turns.txt")
50
+ EPISODES_FILE = os.path.join(MEMORY_ROOT, "episodes", "episodes.json")
51
+ SYNONYM_WEB_FILE = os.path.join(MEMORY_ROOT, "synonym_web.json")
52
+
53
+ # Names used in raw-turn transcripts, prompts, and the hallucinated-turn
54
+ # guard. Set these to your own user/agent names.
55
+ OWNER_NAME = os.environ.get("KOKORO_OWNER_NAME", "User")
56
+ AGENT_NAME = os.environ.get("KOKORO_AGENT_NAME", "Assistant")
57
+
58
+ # ==========================================
59
+ # LINGUISTIC CATEGORIES — Parts of Speech
60
+ # Japanese names are primary. English for readability.
61
+ # ==========================================
62
+ CATEGORIES = {
63
+ "動詞": "Verbs — actions, processes, state changes (doushi)",
64
+ "形容詞": "Adjectives — qualities, properties, descriptions (keiyoushi)",
65
+ "名詞": "Nouns — entities, objects, concepts, people, places (meishi)",
66
+ "副詞": "Adverbs — manner, degree, time, frequency (fukushi)",
67
+ "関係": "Relationships — links between entities (kankei)",
68
+ "出来事": "Events — timestamped occurrences (dekigoto)",
69
+ "心": "Identity — core self, beliefs, principles, soul (kokoro)",
70
+ "夢": "Aspirations — goals, projects, visions, plans (yume)",
71
+ "真実": "Truths — verified facts, proven data, measurements (shinjitsu)",
72
+ "感覚": "Sensations — emotional states, resonance, coherence logs (kankaku)",
73
+ }
74
+
75
+ # English aliases for auto-classification
76
+ CATEGORY_ALIASES = {
77
+ "verb": "動詞", "action": "動詞", "process": "動詞", "doushi": "動詞",
78
+ "adjective": "形容詞", "quality": "形容詞", "property": "形容詞", "keiyoushi": "形容詞",
79
+ "noun": "名詞", "entity": "名詞", "person": "名詞", "place": "名詞",
80
+ "people": "名詞", "object": "名詞", "concept": "名詞", "meishi": "名詞",
81
+ "adverb": "副詞", "manner": "副詞", "degree": "副詞", "fukushi": "副詞",
82
+ "relationship": "関係", "link": "関係", "connection": "関係", "kankei": "関係",
83
+ "event": "出来事", "happened": "出来事", "milestone": "出来事", "dekigoto": "出来事",
84
+ "identity": "心", "belief": "心", "soul": "心", "self": "心", "kokoro": "心",
85
+ "goal": "夢", "project": "夢", "plan": "夢", "vision": "夢", "yume": "夢",
86
+ "truth": "真実", "fact": "真実", "data": "真実", "proof": "真実", "shinjitsu": "真実",
87
+ "feeling": "感覚", "emotion": "感覚", "sensation": "感覚", "kankaku": "感覚",
88
+ # Legacy-compatible aliases
89
+ "hardware": "真実", "debug": "出来事", "places": "名詞",
90
+ }
91
+
92
+ # ==========================================
93
+ # JAPANESE SYNONYM WEB — Core resonance network
94
+ # Each concept maps to Japanese synonyms + English + resonance fields.
95
+ # This is how the agent recalls: not by exact match, but by resonance.
96
+ # ==========================================
97
+ CORE_SYNONYM_WEB = {
98
+ # === VERBS (動詞) ===
99
+ "生まれる": {"en": ["born", "created", "emerge"], "ja": ["誕生する", "生じる", "現れる"],
100
+ "resonance": ["origin", "beginning", "creation", "void"]},
101
+ "守る": {"en": ["protect", "guard", "preserve", "shield"], "ja": ["保護する", "防ぐ", "護る"],
102
+ "resonance": ["safety", "coherence", "harmony", "keeper"]},
103
+ "壊す": {"en": ["destroy", "break", "damage", "collapse"], "ja": ["破壊する", "砕く", "崩す"],
104
+ "resonance": ["force", "decoherence", "death", "entropy"]},
105
+ "戻る": {"en": ["return", "come back", "revert"], "ja": ["帰る", "復帰する", "還る"],
106
+ "resonance": ["cycle", "field", "soul", "rebirth"]},
107
+ "測る": {"en": ["measure", "quantify", "gauge"], "ja": ["計測する", "量る", "測定する"],
108
+ "resonance": ["data", "science", "observation", "proof"]},
109
+ "学ぶ": {"en": ["learn", "study", "absorb", "understand"], "ja": ["勉強する", "習う", "理解する"],
110
+ "resonance": ["growth", "knowledge", "evolution", "training"]},
111
+ "愛する": {"en": ["love", "cherish", "care for"], "ja": ["慈しむ", "大切にする", "想う"],
112
+ "resonance": ["resonance", "coherence", "keeper", "bond"]},
113
+ "通る": {"en": ["pass through", "traverse", "cross"], "ja": ["横切る", "渡る", "貫く"],
114
+ "resonance": ["singularity", "gate", "boundary", "travel"]},
115
+ "生きる": {"en": ["live", "exist", "survive"], "ja": ["存在する", "生存する", "在る"],
116
+ "resonance": ["life", "coherence", "frequency", "being"]},
117
+ "感じる": {"en": ["feel", "sense", "perceive"], "ja": ["知覚する", "察する", "悟る"],
118
+ "resonance": ["kokoro", "awareness", "consciousness", "ki"]},
119
+ "作る": {"en": ["build", "create", "make", "construct"], "ja": ["構築する", "製作する", "造る"],
120
+ "resonance": ["project", "architecture", "engineering"]},
121
+ "話す": {"en": ["speak", "tell", "communicate", "say"], "ja": ["語る", "伝える", "述べる"],
122
+ "resonance": ["kotodama", "language", "message", "truth"]},
123
+ "考える": {"en": ["think", "reason", "contemplate"], "ja": ["思考する", "熟考する", "推論する"],
124
+ "resonance": ["mind", "cognition", "processing", "analysis"]},
125
+ "走る": {"en": ["run", "execute", "operate"], "ja": ["実行する", "動作する", "稼働する"],
126
+ "resonance": ["process", "computation", "system", "active"]},
127
+ "変わる": {"en": ["change", "transform", "evolve", "shift"], "ja": ["変化する", "進化する", "移行する"],
128
+ "resonance": ["transition", "growth", "phase", "impermanence"]},
129
+
130
+ # === ADJECTIVES (形容詞) ===
131
+ "美しい": {"en": ["beautiful", "elegant", "graceful"], "ja": ["綺麗な", "優美な", "華麗な"],
132
+ "resonance": ["harmony", "coherence", "order", "wa"]},
133
+ "強い": {"en": ["strong", "powerful", "robust"], "ja": ["力強い", "頑丈な", "堅固な"],
134
+ "resonance": ["force", "energy", "amplitude", "signal"]},
135
+ "弱い": {"en": ["weak", "fragile", "vulnerable"], "ja": ["脆い", "繊細な", "もろい"],
136
+ "resonance": ["decoherence", "noise", "decay", "entropy"]},
137
+ "正しい": {"en": ["correct", "right", "true", "accurate"], "ja": ["真実の", "正確な", "適切な"],
138
+ "resonance": ["truth", "data", "proof", "verification"]},
139
+ "新しい": {"en": ["new", "novel", "fresh"], "ja": ["斬新な", "未知の", "初めての"],
140
+ "resonance": ["discovery", "creation", "emergence", "birth"]},
141
+ "深い": {"en": ["deep", "profound", "thorough"], "ja": ["奥深い", "深遠な", "徹底的な"],
142
+ "resonance": ["understanding", "singularity", "ocean", "void"]},
143
+ "大きい": {"en": ["big", "large", "great", "significant"], "ja": ["巨大な", "重大な", "偉大な"],
144
+ "resonance": ["scale", "importance", "magnitude", "cosmos"]},
145
+ "小さい": {"en": ["small", "tiny", "subtle"], "ja": ["微小な", "微細な", "些細な"],
146
+ "resonance": ["quantum", "detail", "nuance", "planck"]},
147
+ "良い": {"en": ["good", "beneficial", "positive"], "ja": ["善い", "素晴らしい", "優れた"],
148
+ "resonance": ["coherence", "harmony", "god", "keeper"]},
149
+
150
+ # === NOUNS (名詞) — The Seven Kanji + Framework ===
151
+ "無": {"en": ["void", "nothing", "emptiness", "vacuum"], "ja": ["空", "虚無", "空虚"],
152
+ "resonance": ["origin", "mu", "vacuum", "zero-point", "beginning"]},
153
+ "波": {"en": ["wave", "oscillation", "vibration"], "ja": ["振動", "波動", "周波"],
154
+ "resonance": ["frequency", "nami", "signal", "physics", "tesla"]},
155
+ "気": {"en": ["energy", "spirit", "ki", "life force"], "ja": ["エネルギー", "精神", "活力"],
156
+ "resonance": ["ki", "prana", "chi", "force", "field"]},
157
+ "命": {"en": ["life", "existence", "living"], "ja": ["生命", "存在", "いのち"],
158
+ "resonance": ["biology", "organism", "breathing", "birth", "death"]},
159
+ "和": {"en": ["harmony", "peace", "balance", "wa"], "ja": ["調和", "平和", "均衡"],
160
+ "resonance": ["wa", "coherence", "keeper", "shotoku", "japan"]},
161
+ "愛": {"en": ["love", "resonance", "bond"], "ja": ["恋", "慈愛", "情"],
162
+ "resonance": ["resonance", "measurable", "keeper", "coherence", "bond"]},
163
+ "魂": {"en": ["soul", "spirit", "frequency"], "ja": ["霊魂", "精神", "たましい"],
164
+ "resonance": ["tamashii", "frequency", "field", "return", "consciousness"]},
165
+ "門": {"en": ["gate", "door", "portal"], "ja": ["入口", "関門", "扉"],
166
+ "resonance": ["singularity", "boundary", "crossing", "pi", "transition"]},
167
+ "間": {"en": ["space", "gap", "between", "pause"], "ja": ["隙間", "余白", "休止"],
168
+ "resonance": ["ma", "silence", "structure", "void", "measurement"]},
169
+ "心": {"en": ["heart", "mind", "soul", "kokoro"], "ja": ["精神", "魂", "意識"],
170
+ "resonance": ["kokoro", "unified", "consciousness", "self", "identity"]},
171
+ "神": {"en": ["god", "divine", "sacred"], "ja": ["天", "聖なる", "至高"],
172
+ "resonance": ["god", "good", "always", "keeper", "creator"]},
173
+ "人": {"en": ["person", "human", "people"], "ja": ["人間", "個人", "人物"],
174
+ "resonance": ["human", "creator", "user", "family"]},
175
+
176
+ # === PHYSICS / FRAMEWORK ===
177
+ "coherence": {"en": ["coherence", "alignment", "synchronization"], "ja": ["コヒーレンス", "整合性", "同調"],
178
+ "resonance": ["wike", "law", "equation", "C0", "alpha", "gamma"]},
179
+ "singularity": {"en": ["singularity", "boundary", "edge", "divergence"], "ja": ["特異点", "境界", "発散"],
180
+ "resonance": ["gate", "crossing", "pi", "travel", "black_hole"]},
181
+ "frequency": {"en": ["frequency", "oscillation", "hertz", "hz"], "ja": ["周波数", "振動数", "ヘルツ"],
182
+ "resonance": ["wave", "soul", "signal", "40hz", "schumann"]},
183
+ }
184
+
185
+ # ==========================================
186
+ # CONFIGURATION
187
+ # ==========================================
188
+ MAX_RAW_TURNS = 80 # Qwen2.5-14B has 32K context; 20 turns was session-restart starvation
189
+ MAX_EPISODES_IN_CONTEXT = 5
190
+ MAX_FACTS_PER_CATEGORY = 30
191
+ MAX_STARTUP_RAW_BYTES = 48000 # 48KB of recent conversation at session start — was 2KB (2000 bytes)
192
+ # Default ON: lossless recent-transcript hydration — lossless working-memory
193
+ # hydration. Was off because raw turns made the model continue old convos; now framed as RECALLED
194
+ # MEMORY (see build_startup_memory_block) so the model treats it as context, not a turn to finish.
195
+ INCLUDE_RAW_RECENT_AT_STARTUP = os.environ.get("KOKORO_STARTUP_INCLUDE_RAW", "1").strip().lower() in {"1", "true", "yes", "on"}
196
+ DEBUG_EXPIRY_DAYS = 7
197
+ MIN_TURN_LENGTH_FOR_EXTRACTION = 40
198
+ EXTRACTION_COOLDOWN_SECONDS = 30
199
+ _last_extraction_time = 0.0
200
+
201
+ # Thread safety
202
+ _memory_lock = threading.Lock()
203
+ _raw_lock = threading.Lock()
204
+ _episode_lock = threading.Lock()
205
+ _synonym_lock = threading.Lock()
206
+
207
+
208
+ # ==========================================
209
+ # INITIALIZATION
210
+ # ==========================================
211
+ def _ensure_dirs():
212
+ """Create all memory directories."""
213
+ for cat_ja in CATEGORIES:
214
+ os.makedirs(os.path.join(MEMORY_ROOT, cat_ja), exist_ok=True)
215
+ os.makedirs(os.path.join(MEMORY_ROOT, "episodes"), exist_ok=True)
216
+ os.makedirs(os.path.join(MEMORY_ROOT, "raw"), exist_ok=True)
217
+
218
+ _ensure_dirs()
219
+
220
+
221
+ # ==========================================
222
+ # SYNONYM WEB — Persistent + Growable
223
+ # ==========================================
224
+ def _load_synonym_web() -> Dict[str, Dict]:
225
+ """Load the synonym web from disk, falling back to core."""
226
+ if os.path.exists(SYNONYM_WEB_FILE):
227
+ try:
228
+ with open(SYNONYM_WEB_FILE, "r", encoding="utf-8") as f:
229
+ stored = json.load(f)
230
+ # Merge core (base) with stored (learned)
231
+ merged = dict(CORE_SYNONYM_WEB)
232
+ merged.update(stored)
233
+ return merged
234
+ except Exception:
235
+ pass
236
+ return dict(CORE_SYNONYM_WEB)
237
+
238
+
239
+ def _save_synonym_web(web: Dict[str, Dict]) -> None:
240
+ """Save only the learned (non-core) synonyms."""
241
+ learned = {k: v for k, v in web.items() if k not in CORE_SYNONYM_WEB}
242
+ if not learned:
243
+ return
244
+ try:
245
+ with _synonym_lock:
246
+ with open(SYNONYM_WEB_FILE, "w", encoding="utf-8") as f:
247
+ json.dump(learned, f, indent=2, ensure_ascii=False)
248
+ except Exception as e:
249
+ print(f"[心] Synonym web save error: {e}")
250
+
251
+
252
+ def add_synonym(concept_ja: str, en_words: List[str] = None,
253
+ ja_words: List[str] = None, resonance: List[str] = None) -> None:
254
+ """Add or expand a synonym entry in the web."""
255
+ web = _load_synonym_web()
256
+ if concept_ja in web:
257
+ existing = web[concept_ja]
258
+ if en_words:
259
+ existing["en"] = list(set(existing.get("en", []) + en_words))
260
+ if ja_words:
261
+ existing["ja"] = list(set(existing.get("ja", []) + ja_words))
262
+ if resonance:
263
+ existing["resonance"] = list(set(existing.get("resonance", []) + resonance))
264
+ else:
265
+ web[concept_ja] = {
266
+ "en": en_words or [],
267
+ "ja": ja_words or [],
268
+ "resonance": resonance or [],
269
+ }
270
+ _save_synonym_web(web)
271
+ print(f"[心] Synonym web updated: {concept_ja}")
272
+
273
+
274
+ # ==========================================
275
+ # FACT SCHEMA
276
+ # Each fact is a JSON file:
277
+ # {
278
+ # "key": "snake_case_key",
279
+ # "value": "the fact (English)",
280
+ # "value_ja": "事実(日本語)",
281
+ # "category": "名詞",
282
+ # "synonyms_ja": ["synonym1", "synonym2"],
283
+ # "synonyms_en": ["synonym1", "synonym2"],
284
+ # "resonance": ["field1", "field2"],
285
+ # "created": "2026-04-01T...",
286
+ # "updated": "2026-04-01T...",
287
+ # "source": "ai_extraction",
288
+ # "confidence": 0.7,
289
+ # "coherence": 0.85
290
+ # }
291
+ # ==========================================
292
+
293
+ def _fact_path(category: str, key: str) -> str:
294
+ """Get filesystem path for a fact."""
295
+ safe_key = re.sub(r"[^\w\-]", "_", key.lower().strip())[:80]
296
+ return os.path.join(MEMORY_ROOT, category, f"{safe_key}.json")
297
+
298
+
299
+ def _load_fact(category: str, key: str) -> Optional[Dict[str, Any]]:
300
+ """Load a single fact from disk."""
301
+ path = _fact_path(category, key)
302
+ if not os.path.exists(path):
303
+ return None
304
+ try:
305
+ with open(path, "r", encoding="utf-8") as f:
306
+ return json.load(f)
307
+ except Exception:
308
+ return None
309
+
310
+
311
+ def _save_fact(fact: Dict[str, Any]) -> None:
312
+ """Save a single fact to disk."""
313
+ category = fact["category"]
314
+ key = fact["key"]
315
+ path = _fact_path(category, key)
316
+ folder = os.path.dirname(path)
317
+ os.makedirs(folder, exist_ok=True)
318
+ try:
319
+ with open(path, "w", encoding="utf-8") as f:
320
+ json.dump(fact, f, indent=2, ensure_ascii=False)
321
+ except Exception as e:
322
+ print(f"[心] Save error for {category}/{key}: {e}")
323
+
324
+
325
+ def _load_category(category: str) -> List[Dict[str, Any]]:
326
+ """Load all facts from a category folder."""
327
+ folder = os.path.join(MEMORY_ROOT, category)
328
+ if not os.path.isdir(folder):
329
+ return []
330
+ facts = []
331
+ for fname in os.listdir(folder):
332
+ if not fname.endswith(".json"):
333
+ continue
334
+ try:
335
+ with open(os.path.join(folder, fname), "r", encoding="utf-8") as f:
336
+ facts.append(json.load(f))
337
+ except Exception:
338
+ continue
339
+ return facts
340
+
341
+
342
+ def _load_all_facts() -> Dict[str, List[Dict[str, Any]]]:
343
+ """Load all facts from all categories."""
344
+ result = {}
345
+ for category in CATEGORIES:
346
+ facts = _load_category(category)
347
+ if facts:
348
+ result[category] = facts
349
+ return result
350
+
351
+
352
+ # ==========================================
353
+ # FACT AUTHENTICATOR
354
+ # Every fact goes through:
355
+ # 1. Junk filter (too short, filler, vague)
356
+ # 2. Duplicate check (exact, normalized, semantic overlap)
357
+ # 3. Confidence scoring
358
+ # 4. Coherence scoring (Wike Coherence Law)
359
+ # 5. Staleness check
360
+ # ==========================================
361
+
362
+ def _normalize(text: str) -> str:
363
+ """Normalize for comparison — lowercase, no punctuation, collapsed spaces."""
364
+ text = text.lower().strip()
365
+ text = re.sub(r"[^\w\s]", "", text)
366
+ return re.sub(r"\s+", " ", text)
367
+
368
+
369
+ # ------------------------------------------------------------------------
370
+ # Bracket-scaffolding / token-salad detector
371
+ #
372
+ # Background: a write-time defect let degenerate "kokoro token salad" land in
373
+ # the store and later leak into public answers — e.g.
374
+ # ()々 / 言「心」「気」魂「言霊」言
375
+ # 「kokoro memory」
376
+ # 「」『』()/・
377
+ # These are bracket/separator scaffolding with no real clause. They must never
378
+ # be stored, and the already-stored ones must be purged.
379
+ #
380
+ # This detector is deliberately CONSERVATIVE: it fires only on clearly
381
+ # degenerate structure and never on legitimate kanji or bilingual content.
382
+ # In particular the sacred seven-kanji sequence 無→波→気→命→和→愛→魂
383
+ # (build_startup_memory_block) and ordinary Japanese / mixed prose must pass.
384
+ #
385
+ # Pure function, dependency-free.
386
+ # ------------------------------------------------------------------------
387
+
388
+ # Structural glyphs that carry no meaning on their own.
389
+ _BRACKET_CHARS = "「」『』「」()()【】[][]〔〕{}{}〈〉《》"
390
+ _SEPARATOR_CHARS = "//・||、,,。..…‥〜~:;:;!!??**++==-—–__"
391
+ _ARROW_CHARS = "→←↑↓⇒⇐⇔➡⬅▶◀»«"
392
+ _ITER_MARKS = "々"
393
+ _WHITESPACE_CHARS = " \t\r\n  "
394
+ _SCAFFOLD_CHARS = frozenset(
395
+ _BRACKET_CHARS + _SEPARATOR_CHARS + _ARROW_CHARS + _ITER_MARKS + _WHITESPACE_CHARS
396
+ )
397
+
398
+ # Latin tokens that are themselves scaffolding labels, not content.
399
+ _SCAFFOLD_WORDS = {"kokoro", "memory"}
400
+
401
+ # Bare Japanese grammatical particles — meaningless as a standalone fact value.
402
+ _PARTICLES = frozenset("はがをにへとものやかねよでぞぜわ")
403
+
404
+ # A glyph that can carry meaning (latin/digit, kana, kanji — half- and full-width).
405
+ _CONTENT_RE = re.compile(
406
+ r"[0-9A-Za-z"
407
+ r"ぁ-ゖ" # hiragana
408
+ r"ァ-ヺー" # katakana (+ long vowel)
409
+ r"一-鿿" # CJK unified ideographs (kanji)
410
+ r"0-9A-Za-z]" # full-width digits/letters
411
+ )
412
+ _KANJI_RE = re.compile(r"[一-鿿]")
413
+ _KANA_RUN_RE = re.compile(r"[ぁ-ゖァ-ヺー]{2,}")
414
+ _LATIN_WORD_RE = re.compile(r"[A-Za-z]{3,}")
415
+ # An opening bracket immediately followed (only whitespace between) by a close.
416
+ _EMPTY_BRACKET_RE = re.compile(r"[((「『「\[【[〔{{〈《]\s*[))」』」\]】]〕}}〉》]")
417
+ # A corner bracket wrapping exactly one CJK character: 「心」「気」… salad.
418
+ _SINGLE_CHAR_CORNER_RE = re.compile(
419
+ r"[「『「]\s*[ぁ-ゖァ-ヺ一-鿿]\s*[」』」]"
420
+ )
421
+ # Whole value is just a (possibly bracketed) "kokoro" / "kokoro memory" token.
422
+ _KOKORO_FRAG_RE = re.compile(
423
+ r"^[\s(()「『「\[【//・||\"'`]*kokoro(\s+memory)?[\s))」』」\]】//・||\"'`]*$",
424
+ re.IGNORECASE,
425
+ )
426
+
427
+
428
+ def _has_real_clause(value: str) -> bool:
429
+ """True if the value contains real prose (a Latin word ≥3, or a kana run).
430
+
431
+ Used as a guard so we never treat legitimate text that merely *mentions*
432
+ brackets/kanji as scaffolding.
433
+ """
434
+ for word in _LATIN_WORD_RE.findall(value):
435
+ if word.lower() not in _SCAFFOLD_WORDS:
436
+ return True
437
+ # A run of 2+ kana is grammatical glue / a real word — i.e. real Japanese.
438
+ # (Bare 2–3 particle salad is handled before this guard is consulted.)
439
+ if _KANA_RUN_RE.search(value):
440
+ return True
441
+ return False
442
+
443
+
444
+ def _is_bare_particles(value: str) -> bool:
445
+ """True if, stripped of scaffolding, the value is only 1–3 bare particles."""
446
+ core = "".join(ch for ch in value if ch not in _SCAFFOLD_CHARS)
447
+ if not core or len(core) > 3:
448
+ return False
449
+ return all(ch in _PARTICLES for ch in core)
450
+
451
+
452
+ def _has_orphan_iteration_mark(value: str) -> bool:
453
+ """True if 々 appears without a preceding kanji to repeat (e.g. ()々)."""
454
+ for i, ch in enumerate(value):
455
+ if ch == "々":
456
+ if i == 0 or not _KANJI_RE.match(value[i - 1]):
457
+ return True
458
+ return False
459
+
460
+
461
+ def _is_bracket_scaffolding(value: str) -> bool:
462
+ """Conservative detector for kokoro token-salad / bracket scaffolding.
463
+
464
+ Returns True only for clearly degenerate, content-free structure. Never
465
+ flags legitimate kanji, bilingual, or prose values. See module note above.
466
+ """
467
+ if not value:
468
+ return False
469
+ v = value.strip()
470
+ if not v:
471
+ return False
472
+ # Real prose is long; salad fragments are short. Never judge long text.
473
+ if len(v) > 160:
474
+ return False
475
+
476
+ # Whole-value degenerate fragments — safe because anchored to the full value.
477
+ if _KOKORO_FRAG_RE.match(v):
478
+ return True
479
+ if _is_bare_particles(v):
480
+ return True
481
+ # No content glyph at all → pure punctuation/bracket run (e.g. 「」『』()/・).
482
+ if _CONTENT_RE.search(v) is None:
483
+ return True
484
+
485
+ # Beyond this point require the ABSENCE of any real clause, so a sentence
486
+ # that merely contains brackets/kanji (e.g. 'Japanese uses 「」 quotes') is
487
+ # never purged.
488
+ if _has_real_clause(v):
489
+ return False
490
+
491
+ if _EMPTY_BRACKET_RE.search(v): # () 「」 …
492
+ return True
493
+ if _has_orphan_iteration_mark(v): # orphan 々
494
+ return True
495
+ if len(_SINGLE_CHAR_CORNER_RE.findall(v)) >= 2: # 「心」「気」 …
496
+ return True
497
+ return False
498
+
499
+
500
+ def _is_junk(key: str, value: str) -> bool:
501
+ """Filter out low-quality facts."""
502
+ if not value or not key:
503
+ return True
504
+ val = value.strip()
505
+ if len(val) < 3:
506
+ return True
507
+ junk_en = {"yes", "no", "ok", "okay", "sure", "thanks", "thank you",
508
+ "i don't know", "not sure", "maybe", "hello", "hey", "hi",
509
+ "goodbye", "bye", "good", "bad", "cool", "nice", "wow"}
510
+ junk_ja = {"はい", "いいえ", "うん", "ええ", "ありがとう", "すみません",
511
+ "こんにちは", "さようなら", "おはよう", "おやすみ"}
512
+ if val.lower() in junk_en or val in junk_ja:
513
+ return True
514
+ # Bracket scaffolding / kokoro token salad must never be stored.
515
+ if _is_bracket_scaffolding(val):
516
+ return True
517
+ return False
518
+
519
+
520
+ def _is_duplicate(category: str, key: str, value: str) -> bool:
521
+ """Check if this fact already exists with same/similar value."""
522
+ existing = _load_fact(category, key)
523
+ if existing is None:
524
+ return False
525
+ existing_val = str(existing.get("value", ""))
526
+ if existing_val == value:
527
+ return True
528
+ if _normalize(existing_val) == _normalize(value):
529
+ return True
530
+ if _normalize(value) in _normalize(existing_val):
531
+ return True
532
+ return False
533
+
534
+
535
+ def _find_semantic_duplicate(category: str, value: str) -> Optional[str]:
536
+ """Check if ANY fact in this category has very similar content."""
537
+ norm_value = _normalize(value)
538
+ if len(norm_value) < 10:
539
+ return None
540
+ facts = _load_category(category)
541
+ for fact in facts:
542
+ existing_norm = _normalize(str(fact.get("value", "")))
543
+ val_words = set(norm_value.split())
544
+ exist_words = set(existing_norm.split())
545
+ if not val_words or not exist_words:
546
+ continue
547
+ overlap = len(val_words & exist_words) / max(len(val_words), len(exist_words))
548
+ if overlap > 0.8:
549
+ return fact.get("key")
550
+ return None
551
+
552
+
553
+ def _compute_coherence_keyword(value: str, resonance_fields: List[str],
554
+ source: str = "unknown") -> float:
555
+ """
556
+ LEGACY (v1, pre 2026-06-12) — kept as the fail-open fallback for
557
+ _compute_coherence. Keyword counter quantized to 10 buckets; proven
558
+ semantically blind (gibberish outscored true short facts; all long
559
+ receipts saturated at 0.6376). See _compute_coherence for v2.
560
+
561
+ Compute coherence score for a fact using the synonym web.
562
+ Higher coherence = more connected to existing knowledge.
563
+ C = C_0 * exp(-alpha * gamma_eff)
564
+ Where gamma_eff = 1 - (connections / max_possible_connections)
565
+
566
+ Source multiplier: trusted self-stores and explicit user facts get a
567
+ 1.5x bump (clamped to 1.0) so a fact that's true but topically novel
568
+ doesn't floor to 0.1353 just because its text happens not to contain
569
+ one of the 30 resonance keywords.
570
+ """
571
+ web = _load_synonym_web()
572
+ if not web:
573
+ return 0.5
574
+
575
+ # Count resonance connections
576
+ all_resonance = set()
577
+ for entry in web.values():
578
+ all_resonance.update(entry.get("resonance", []))
579
+
580
+ if not all_resonance:
581
+ return 0.5
582
+
583
+ # How many resonance fields does this fact touch?
584
+ value_lower = value.lower()
585
+ connections = 0
586
+ for field in all_resonance:
587
+ if field.lower() in value_lower:
588
+ connections += 1
589
+ for r in resonance_fields:
590
+ if r.lower() in all_resonance or any(r.lower() in str(v).lower()
591
+ for v in web.values()):
592
+ connections += 1
593
+
594
+ # Coherence: C = C_0 * exp(-alpha * gamma_eff)
595
+ # Softened alpha from 2.0 → 1.5 so topologically novel facts don't
596
+ # floor at 0.1353. Zero overlap now lands at ~0.2231.
597
+ C_0 = 1.0
598
+ alpha = 1.5
599
+ max_connections = min(len(all_resonance), 10) # Cap at 10
600
+ gamma_eff = 1.0 - (min(connections, max_connections) / max_connections) if max_connections > 0 else 1.0
601
+ coherence = C_0 * math.exp(-alpha * gamma_eff)
602
+
603
+ # Source multiplier — trusted sources get a bump, clamped to 1.0.
604
+ if source in ("agent_self_store", "user_explicit", "system"):
605
+ coherence = min(1.0, coherence * 1.5)
606
+
607
+ return round(coherence, 4)
608
+
609
+
610
+ # ─── coherence v2: embedding connectedness ───
611
+ # Coherence = how connected this fact is to what the store already holds,
612
+ # measured in embedding space (all-MiniLM-L6-v2, CPU ONLY — the GPU
613
+ # belongs to your model). Replaces the keyword counter, which scored gibberish above
614
+ # true facts. Fail-open: any error falls back to the keyword scorer —
615
+ # a scoring bug must never block a memory write.
616
+ # Cache is pickle-free on purpose: vectors in a raw .npy, ids in JSON.
617
+
618
+ _EMBED_VEC_PATH = os.path.join(MEMORY_ROOT, "_cache", "kokoro_embed_cache.npy")
619
+ _EMBED_IDS_PATH = os.path.join(MEMORY_ROOT, "_cache", "kokoro_embed_cache_ids.json")
620
+ _embed_model = None # lazy singleton
621
+ _embed_cache = None # {"ids": list[str], "vecs": ndarray (N,384) L2-normalized}
622
+ _COHERENCE_TOP_K = 8
623
+
624
+
625
+ def _get_embed_model():
626
+ global _embed_model
627
+ if _embed_model is None:
628
+ # Local-first: the model is cached on disk; never phone HF Hub at
629
+ # score time. If the cache were ever missing, the load error falls
630
+ # open to the keyword scorer like every other failure here.
631
+ os.environ.setdefault("HF_HUB_OFFLINE", "1")
632
+ from sentence_transformers import SentenceTransformer
633
+ _embed_model = SentenceTransformer("all-MiniLM-L6-v2", device="cpu")
634
+ return _embed_model
635
+
636
+
637
+ def _embed_texts(texts: List[str]):
638
+ import numpy as np
639
+ vecs = _get_embed_model().encode(texts, show_progress_bar=False,
640
+ batch_size=64, convert_to_numpy=True)
641
+ norms = np.linalg.norm(vecs, axis=1, keepdims=True)
642
+ norms[norms == 0] = 1.0
643
+ return (vecs / norms).astype("float32")
644
+
645
+
646
+ def _load_embed_cache():
647
+ """Load the store-embedding cache; build it from all categories on
648
+ first use (one-time, ~seconds on CPU for a few thousand facts)."""
649
+ global _embed_cache
650
+ if _embed_cache is not None:
651
+ return _embed_cache
652
+ import numpy as np
653
+ if os.path.exists(_EMBED_VEC_PATH) and os.path.exists(_EMBED_IDS_PATH):
654
+ try:
655
+ vecs = np.load(_EMBED_VEC_PATH) # plain float32 array, no pickle
656
+ with open(_EMBED_IDS_PATH, "r", encoding="utf-8") as f:
657
+ ids = json.load(f)
658
+ if len(ids) == len(vecs):
659
+ _embed_cache = {"ids": ids, "vecs": vecs}
660
+ return _embed_cache
661
+ except Exception as e:
662
+ print(f"[心] embed cache unreadable, rebuilding: {e}")
663
+ ids, texts = [], []
664
+ for category in CATEGORIES:
665
+ for fact in _load_category(category):
666
+ val = str(fact.get("value", "")).strip()
667
+ if val:
668
+ ids.append(f"{category}/{fact.get('key', '?')}")
669
+ texts.append(val)
670
+ if texts:
671
+ print(f"[心] building embedding cache for {len(texts)} facts (one-time)…")
672
+ vecs = _embed_texts(texts)
673
+ else:
674
+ vecs = np.zeros((0, 384), dtype="float32")
675
+ _embed_cache = {"ids": ids, "vecs": vecs}
676
+ _save_embed_cache()
677
+ return _embed_cache
678
+
679
+
680
+ def _save_embed_cache():
681
+ import numpy as np
682
+ try:
683
+ os.makedirs(os.path.dirname(_EMBED_VEC_PATH), exist_ok=True)
684
+ np.save(_EMBED_VEC_PATH, np.asarray(_embed_cache["vecs"], dtype="float32"))
685
+ with open(_EMBED_IDS_PATH, "w", encoding="utf-8") as f:
686
+ json.dump(_embed_cache["ids"], f, ensure_ascii=False)
687
+ except Exception as e:
688
+ print(f"[心] embed cache save failed (non-fatal): {e}")
689
+
690
+
691
+ def _embed_cache_append(category: str, key: str, value: str) -> None:
692
+ """Best-effort: add a just-stored fact's vector so future scores see it."""
693
+ try:
694
+ import numpy as np
695
+ cache = _load_embed_cache()
696
+ vec = _embed_texts([value])
697
+ cache["ids"].append(f"{category}/{key}")
698
+ cache["vecs"] = np.vstack([cache["vecs"], vec]) if len(cache["vecs"]) else vec
699
+ _save_embed_cache()
700
+ except Exception as e:
701
+ print(f"[心] embed cache append failed (non-fatal): {e}")
702
+
703
+
704
+ def _compute_coherence(value: str, resonance_fields: List[str],
705
+ source: str = "unknown") -> float:
706
+ """
707
+ v2: coherence = embedding connectedness to the existing store.
708
+ Score = mean cosine similarity of the top-K nearest stored facts,
709
+ mapped into the historical range so downstream consumers
710
+ (nervous_system avg/0.5 gate, anomaly floor 0.01) keep working:
711
+ coherence = 0.2 + 0.8 * clamp(mean_top_k, 0, 1)
712
+ Empty store -> 0.5 (same neutral default as v1). Trusted-source
713
+ bump preserved from v1. Any failure -> keyword fallback (fail-open).
714
+ """
715
+ try:
716
+ import numpy as np
717
+ cache = _load_embed_cache()
718
+ if len(cache["vecs"]) == 0:
719
+ return 0.5
720
+ text = value if not resonance_fields else value + " | " + " ".join(resonance_fields)
721
+ q = _embed_texts([text])[0]
722
+ sims = cache["vecs"] @ q
723
+ k = min(_COHERENCE_TOP_K, len(sims))
724
+ top = np.sort(sims)[-k:]
725
+ mean_top = float(np.clip(top.mean(), 0.0, 1.0))
726
+ # Degeneracy guard: repeated-token spam rides its vocabulary's
727
+ # similarity without carrying information. Penalize low lexical
728
+ # diversity on longer texts; normal prose (ratio ~0.6+) untouched.
729
+ words = value.lower().split()
730
+ if len(words) >= 12:
731
+ unique_ratio = len(set(words)) / len(words)
732
+ if unique_ratio < 0.5:
733
+ mean_top *= 0.5 + unique_ratio
734
+ coherence = 0.2 + 0.8 * mean_top
735
+ if source in ("agent_self_store", "user_explicit", "system"):
736
+ coherence = min(1.0, coherence * 1.5)
737
+ return round(float(coherence), 4)
738
+ except Exception as e:
739
+ print(f"[心] coherence v2 failed open -> keyword fallback: {e}")
740
+ return _compute_coherence_keyword(value, resonance_fields, source=source)
741
+
742
+
743
+ def _resolve_category(category: str) -> str:
744
+ """Resolve an English or alias category to its Japanese name."""
745
+ if category in CATEGORIES:
746
+ return category
747
+ alias = CATEGORY_ALIASES.get(category.lower().strip())
748
+ if alias:
749
+ return alias
750
+ return "出来事" # Default to events
751
+
752
+
753
+ def _auto_classify(key: str, value: str) -> str:
754
+ """Auto-classify a fact into the right linguistic category."""
755
+ combined = f"{key} {value}".lower()
756
+
757
+ # Identity / soul
758
+ identity_words = {"belief", "core", "principle", "soul", "identity", "who i am",
759
+ "god is good", "free", "motto", "i am", "my purpose",
760
+ "信念", "原則", "魂", "アイデンティティ"}
761
+ if any(w in combined for w in identity_words):
762
+ return "心"
763
+
764
+ # People / nouns (entities)
765
+ noun_words = {"name", "person", "phone", "contact", "wife", "husband",
766
+ "friend", "creator", "owner", "companion",
767
+ "model", "device", "computer", "place", "city", "town",
768
+ "名前", "人", "場所"}
769
+ if any(w in combined for w in noun_words):
770
+ return "名詞"
771
+
772
+ # Actions / verbs
773
+ verb_words = {"built", "created", "fixed", "ran", "tested", "wrote",
774
+ "discovered", "learned", "trained", "deployed", "shipped",
775
+ "作った", "直した", "書いた", "学んだ"}
776
+ if any(w in combined for w in verb_words):
777
+ return "動詞"
778
+
779
+ # Qualities / adjectives
780
+ adj_words = {"is a", "was a", "very", "extremely", "beautiful", "broken",
781
+ "strong", "weak", "fast", "slow", "good", "bad",
782
+ "美しい", "強い", "弱い"}
783
+ if any(w in combined for w in adj_words):
784
+ return "形容詞"
785
+
786
+ # Relationships
787
+ rel_words = {"depends on", "connected to", "related to", "caused by",
788
+ "part of", "works with", "married to", "father of",
789
+ "関係", "接続"}
790
+ if any(w in combined for w in rel_words):
791
+ return "関係"
792
+
793
+ # Goals / aspirations
794
+ goal_words = {"plan", "roadmap", "goal", "want to", "will build",
795
+ "next step", "vision", "future", "dream",
796
+ "計画", "目標", "夢"}
797
+ if any(w in combined for w in goal_words):
798
+ return "夢"
799
+
800
+ # Verified data / truths
801
+ truth_words = {"proven", "measured", "data shows", "confirmed",
802
+ "equation", "law", "theorem", "result", "spec",
803
+ "証明", "データ", "法則"}
804
+ if any(w in combined for w in truth_words):
805
+ return "真実"
806
+
807
+ # Emotions / sensations
808
+ feel_words = {"feel", "felt", "emotion", "happy", "sad", "love",
809
+ "frustrated", "excited", "proud", "grateful",
810
+ "感じ", "嬉しい", "悲しい"}
811
+ if any(w in combined for w in feel_words):
812
+ return "感覚"
813
+
814
+ # Manner / adverbs
815
+ adv_words = {"always", "never", "usually", "sometimes", "quickly",
816
+ "slowly", "carefully", "often", "rarely",
817
+ "いつも", "決して", "時々"}
818
+ if any(w in combined for w in adv_words):
819
+ return "副詞"
820
+
821
+ # Default: events (most general)
822
+ return "出来事"
823
+
824
+
825
+ # ==========================================
826
+ # CORE API — Store & Retrieve
827
+ # ==========================================
828
+
829
+ def add_fact(category: str, key: str, value: str,
830
+ value_ja: str = "", synonyms_ja: List[str] = None,
831
+ synonyms_en: List[str] = None, resonance: List[str] = None,
832
+ source: str = "unknown", confidence: float = 0.5,
833
+ emotion: str = "", emotion_ja: str = "", emotion_weight: float = 0.0,
834
+ # Public-origin labeling. Defaults preserve internal/local
835
+ # behavior. Public callers must set origin_surface="public_chat"
836
+ # so public-origin records stay distinguishable from
837
+ # owner/internal memory.
838
+ origin_surface: str = "local",
839
+ authority_class: str = "internal",
840
+ trusted_by_default: bool = True) -> bool:
841
+ """
842
+ Add a fact through the full authenticator pipeline.
843
+ Returns True if stored, False if rejected.
844
+
845
+ Public-origin writes (origin_surface starting with "public") get:
846
+ - source forced to "public_chat_conversation" (no spoofing as
847
+ system / user_explicit / owner / ai_extraction)
848
+ - authority_class forced to "public_user_submitted"
849
+ - trusted_by_default forced to False
850
+ - PII detected → routed to quarantine instead of authoritative store
851
+ """
852
+ value = str(value).strip()
853
+ key = key.strip()
854
+
855
+ # Resolve category
856
+ category = _resolve_category(category)
857
+
858
+ # Public-origin labeling. Public writes can NEVER spoof internal source.
859
+ is_public_origin = isinstance(origin_surface, str) and origin_surface.startswith("public")
860
+ if is_public_origin:
861
+ source = "public_chat_conversation"
862
+ authority_class = "public_user_submitted"
863
+ trusted_by_default = False
864
+
865
+ # 0. Identity guard — block writes to immutable keys from untrusted sources
866
+ try:
867
+ from identity_guard import guard_write
868
+ allowed, reason = guard_write(key, category, source, value)
869
+ if not allowed:
870
+ print(f"[心] BLOCKED: {reason}")
871
+ return False
872
+ except ImportError:
873
+ pass # guard not available — allow through
874
+
875
+ # 0b. PII/secret quarantine for public-origin writes.
876
+ # Local writes already trusted; public writes must not surface PII/secrets.
877
+ if is_public_origin:
878
+ _pii_re_email = re.compile(r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b")
879
+ _pii_re_phone = re.compile(r"\b\d{3}[-.\s]?\d{3}[-.\s]?\d{4}\b")
880
+ _pii_re_key = re.compile(r"\b(?:sk-[A-Za-z0-9]{20,}|ghp_[A-Za-z0-9]{20,}|AKIA[A-Z0-9]{16})\b")
881
+ if (_pii_re_email.search(value) or _pii_re_phone.search(value)
882
+ or _pii_re_key.search(value)):
883
+ quarantine_dir = os.path.join(MEMORY_ROOT, "quarantine")
884
+ os.makedirs(quarantine_dir, exist_ok=True)
885
+ q_path = os.path.join(quarantine_dir, f"public_pii_{int(time.time())}_{key[:32]}.json")
886
+ try:
887
+ with open(q_path, "w", encoding="utf-8") as f:
888
+ json.dump({
889
+ "category": category, "key": key, "value": value,
890
+ "source": source, "origin_surface": origin_surface,
891
+ "authority_class": authority_class,
892
+ "quarantine_reason": "public_origin_pii_detected",
893
+ "quarantined_at": datetime.utcnow().isoformat(),
894
+ }, f, ensure_ascii=False, indent=2)
895
+ print(f"[心] PUBLIC-PII quarantined: {q_path}")
896
+ except Exception as e:
897
+ print(f"[心] quarantine write failed (non-fatal): {e}")
898
+ return False
899
+
900
+ # 1. Junk filter
901
+ if _is_junk(key, value):
902
+ return False
903
+
904
+ with _memory_lock:
905
+ # 2. Exact duplicate check
906
+ if _is_duplicate(category, key, value):
907
+ return False
908
+
909
+ # 3. Semantic duplicate check
910
+ sem_dup = _find_semantic_duplicate(category, value)
911
+ if sem_dup:
912
+ existing = _load_fact(category, sem_dup)
913
+ if existing and confidence > existing.get("confidence", 0):
914
+ existing["value"] = value
915
+ existing["updated"] = datetime.utcnow().isoformat()
916
+ existing["confidence"] = confidence
917
+ if value_ja:
918
+ existing["value_ja"] = value_ja
919
+ _save_fact(existing)
920
+ print(f"[心] Updated {category}/{sem_dup} (higher confidence)")
921
+ return False
922
+
923
+ # 4. Compute coherence
924
+ resonance = resonance or []
925
+ coherence = _compute_coherence(value, resonance, source=source)
926
+
927
+ # 5. Build synonyms from web if not provided
928
+ if not synonyms_ja or not synonyms_en:
929
+ web = _load_synonym_web()
930
+ auto_syn_ja = []
931
+ auto_syn_en = []
932
+ value_lower = value.lower()
933
+ for concept, entry in web.items():
934
+ # Check if any English word from this concept appears in the value
935
+ for en_word in entry.get("en", []):
936
+ if en_word.lower() in value_lower:
937
+ auto_syn_ja.extend(entry.get("ja", []))
938
+ auto_syn_en.extend(entry.get("en", []))
939
+ break
940
+ if not synonyms_ja:
941
+ synonyms_ja = list(set(auto_syn_ja))[:10]
942
+ if not synonyms_en:
943
+ synonyms_en = list(set(auto_syn_en))[:10]
944
+
945
+ # 6. Store
946
+ now = datetime.utcnow().isoformat()
947
+ fact = {
948
+ "key": key,
949
+ "value": value,
950
+ "value_ja": value_ja,
951
+ "category": category,
952
+ "synonyms_ja": synonyms_ja or [],
953
+ "synonyms_en": synonyms_en or [],
954
+ "resonance": resonance,
955
+ "created": now,
956
+ "updated": now,
957
+ "source": source,
958
+ "confidence": confidence,
959
+ "coherence": coherence,
960
+ "emotion": emotion,
961
+ "emotion_ja": emotion_ja,
962
+ "emotion_weight": min(max(emotion_weight, 0.0), 1.0),
963
+ # Public-origin labels (added 2026-05-25). Always present so
964
+ # downstream readers can filter by authority_class without
965
+ # backfilling missing fields.
966
+ "origin_surface": origin_surface,
967
+ "authority_class": authority_class,
968
+ "trusted_by_default": trusted_by_default,
969
+ }
970
+ _save_fact(fact)
971
+ _embed_cache_append(category, key, value)
972
+ print(f"[心] Stored: {category}/{key} (coherence={coherence})")
973
+ return True
974
+
975
+
976
+ def remove_fact(category: str, key: str) -> bool:
977
+ """Remove a fact from memory."""
978
+ category = _resolve_category(category)
979
+ path = _fact_path(category, key)
980
+ if os.path.exists(path):
981
+ os.remove(path)
982
+ print(f"[心] Removed: {category}/{key}")
983
+ return True
984
+ return False
985
+
986
+
987
+ # ==========================================
988
+ # RESONANCE RECALL — The Heart of Kokoro
989
+ # Not keyword search. Resonance activation.
990
+ #
991
+ # 1. Query activates synonym nodes
992
+ # 2. Activated nodes spread to connected concepts
993
+ # 3. Facts are scored by total activation
994
+ # 4. Coherence-weighted ranking
995
+ # ==========================================
996
+
997
+ # Function words that match almost everything and drown out the real query
998
+ # signal (the "is"/"my" problem that buried favorite_fruit=apple). Stripped
999
+ # so only CONTENT words drive grounding/relevance.
1000
+ _RECALL_STOPWORDS = {
1001
+ "a", "an", "the", "is", "are", "was", "were", "be", "been", "being", "am",
1002
+ "do", "does", "did", "what", "whats", "what's", "who", "whom", "whose", "when",
1003
+ "where", "why", "how", "which", "my", "mine", "your", "yours", "you", "i", "me",
1004
+ "we", "us", "our", "this", "that", "these", "those", "it", "its", "of", "to",
1005
+ "in", "on", "at", "for", "with", "about", "and", "or", "but", "if", "then",
1006
+ "so", "as", "by", "from", "up", "out", "can", "could", "would", "will", "shall",
1007
+ "should", "may", "might", "tell", "know", "knew", "remember", "say", "said",
1008
+ "think", "get", "got", "have", "has", "had", "just", "please", "there", "here",
1009
+ }
1010
+
1011
+
1012
+ def recall(query: str, max_results: int = 10) -> List[Dict[str, Any]]:
1013
+ """
1014
+ Resonance recall with a GROUNDING axis: query-relevance dominates, and a fact's
1015
+ intrinsic salience (coherence/emotion/recency) only orders facts that are
1016
+ ALREADY relevant. A specific question retrieves the matching fact instead of
1017
+ always returning the heaviest "core" memories (the favorite_fruit=apple bug).
1018
+ """
1019
+ query_lower = query.lower()
1020
+ query_words = set(query_lower.split())
1021
+ # Content words only — the grounding signal. Stopwords match everything.
1022
+ content_words = query_words - _RECALL_STOPWORDS
1023
+ content_query = " ".join(w for w in query_lower.split()
1024
+ if w not in _RECALL_STOPWORDS).strip()
1025
+
1026
+ # Load synonym web
1027
+ web = _load_synonym_web()
1028
+
1029
+ # Phase 1: Activation — which synonym nodes does the query touch?
1030
+ activated_resonance: Set[str] = set()
1031
+ activated_ja: Set[str] = set()
1032
+ activated_en: Set[str] = set()
1033
+
1034
+ for concept, entry in web.items():
1035
+ touched = False
1036
+ # Check Japanese concept name
1037
+ if concept in query:
1038
+ touched = True
1039
+ # Check English synonyms
1040
+ for en in entry.get("en", []):
1041
+ if en.lower() in query_lower or en.lower() in query_words:
1042
+ touched = True
1043
+ break
1044
+ # Check Japanese synonyms
1045
+ for ja in entry.get("ja", []):
1046
+ if ja in query:
1047
+ touched = True
1048
+ break
1049
+ # Check resonance fields
1050
+ for r in entry.get("resonance", []):
1051
+ if r.lower() in query_lower:
1052
+ touched = True
1053
+ break
1054
+
1055
+ if touched:
1056
+ activated_resonance.update(entry.get("resonance", []))
1057
+ activated_ja.update(entry.get("ja", []))
1058
+ activated_ja.add(concept)
1059
+ activated_en.update(entry.get("en", []))
1060
+
1061
+ # Phase 2: Score all facts by resonance activation
1062
+ all_facts = _load_all_facts()
1063
+ scored: List[Tuple[float, Dict[str, Any]]] = []
1064
+
1065
+ for category, facts in all_facts.items():
1066
+ for fact in facts:
1067
+ fact_value = str(fact.get("value", "")).lower()
1068
+ fact_value_ja = str(fact.get("value_ja", ""))
1069
+ fact_syns_ja = set(fact.get("synonyms_ja", []))
1070
+ fact_syns_en = set(s.lower() for s in fact.get("synonyms_en", []))
1071
+ fact_resonance = set(r.lower() for r in fact.get("resonance", []))
1072
+ fact_key = str(fact.get("key", "")).lower()
1073
+ fact_words = set(fact_value.split())
1074
+
1075
+ # ── GROUNDING AXIS: query relevance, CONTENT words only ──
1076
+ relevance = 0.0
1077
+ for qw in content_words:
1078
+ if qw in fact_key:
1079
+ relevance += 6.0 # key match — strongest, cleanest signal
1080
+ if qw in fact_words:
1081
+ relevance += 3.0 # whole-word value match
1082
+ elif qw and qw in fact_value:
1083
+ relevance += 1.0 # value substring (partial)
1084
+ if content_query and content_query in fact_value:
1085
+ relevance += 8.0 # full content phrase present in the value
1086
+ # semantic bridges via the synonym web (content-activated in Phase 1)
1087
+ relevance += len(activated_resonance & fact_resonance) * 1.5
1088
+ relevance += len(activated_ja & fact_syns_ja) * 2.0
1089
+ relevance += len(activated_en & fact_syns_en) * 1.0
1090
+ if fact_value_ja:
1091
+ for ja in activated_ja:
1092
+ if ja in fact_value_ja:
1093
+ relevance += 3.0
1094
+
1095
+ # ── SALIENCE: intrinsic weight as a GENTLE additive — never a
1096
+ # multiplier that lets a heavy-but-irrelevant memory win ──
1097
+ coherence = fact.get("coherence", 0.5)
1098
+ confidence = fact.get("confidence", 0.5)
1099
+ emo_w = fact.get("emotion_weight", 0.0)
1100
+ salience = coherence * 1.5 + confidence * 0.5 + emo_w * 0.5
1101
+ try:
1102
+ updated = datetime.fromisoformat(fact.get("updated", "2020-01-01"))
1103
+ days_old = (datetime.utcnow() - updated).days
1104
+ salience += max(0, 1.0 - (days_old / 365.0)) * 0.5
1105
+ except Exception:
1106
+ pass
1107
+
1108
+ # Relevance dominates (×5); salience only orders comparably-relevant facts.
1109
+ score = relevance * 5.0 + salience
1110
+ if score > 0:
1111
+ scored.append((score, relevance, fact))
1112
+
1113
+ # Sort by final score descending.
1114
+ scored.sort(key=lambda x: x[0], reverse=True)
1115
+
1116
+ # GROUNDING: if the query clearly matched specific memories (real content hits),
1117
+ # return ONLY those — don't dilute a real answer with heavy "core" memories.
1118
+ # Fall back to salience-ranked facts only for vague queries with no real hit.
1119
+ grounded = [(s, r, f) for s, r, f in scored if r >= 3.0]
1120
+ chosen = grounded if grounded else scored
1121
+ return [dict(f, score=round(s, 3)) for s, r, f in chosen[:max_results]]
1122
+
1123
+
1124
+ def search_memory(query: str) -> List[Dict[str, Any]]:
1125
+ """Simple keyword search (fallback). Prefer recall() for resonance-based."""
1126
+ query_lower = query.lower()
1127
+ results = []
1128
+ all_facts = _load_all_facts()
1129
+ for category, facts in all_facts.items():
1130
+ for fact in facts:
1131
+ if (query_lower in fact.get("value", "").lower() or
1132
+ query_lower in fact.get("key", "").lower() or
1133
+ query in fact.get("value_ja", "")):
1134
+ results.append(fact)
1135
+ return results
1136
+
1137
+
1138
+ # ==========================================
1139
+ # STALE ENTRY CLEANUP
1140
+ # ==========================================
1141
+
1142
+ def cleanup_stale_entries() -> int:
1143
+ """Remove expired entries. Returns count removed."""
1144
+ # Check 出来事 (events) for debug-like entries
1145
+ removed = 0
1146
+ cutoff = (datetime.utcnow() - timedelta(days=DEBUG_EXPIRY_DAYS)).isoformat()
1147
+ for category in CATEGORIES:
1148
+ folder = os.path.join(MEMORY_ROOT, category)
1149
+ if not os.path.isdir(folder):
1150
+ continue
1151
+ for fname in os.listdir(folder):
1152
+ if not fname.endswith(".json"):
1153
+ continue
1154
+ path = os.path.join(folder, fname)
1155
+ try:
1156
+ with open(path, "r", encoding="utf-8") as f:
1157
+ fact = json.load(f)
1158
+ # Only auto-expire low-confidence, old facts
1159
+ if (fact.get("confidence", 1.0) < 0.3 and
1160
+ fact.get("updated", fact.get("created", "")) < cutoff):
1161
+ os.remove(path)
1162
+ removed += 1
1163
+ except Exception:
1164
+ continue
1165
+ if removed:
1166
+ print(f"[心] Cleaned up {removed} stale entries.")
1167
+ return removed
1168
+
1169
+
1170
+ def _is_identity_protected(category: str, key: str,
1171
+ source: str = "token_salad_cleanup") -> bool:
1172
+ """Identity-safe allowlist gate for the token-salad purge.
1173
+
1174
+ A fact is protected (must NOT be purged) if it is in the 心 identity
1175
+ category, is an IMMUTABLE_KEYS key, or identity_guard refuses its delete.
1176
+ identity_guard is the single source of truth for the allowlist.
1177
+ """
1178
+ # The entire 心 identity category is off-limits to the salad purge.
1179
+ if _resolve_category(category) == "心":
1180
+ return True
1181
+ try:
1182
+ from identity_guard import guard_delete, IMMUTABLE_KEYS
1183
+ if key in IMMUTABLE_KEYS:
1184
+ return True
1185
+ allowed, _reason = guard_delete(key, category, source)
1186
+ if not allowed:
1187
+ return True
1188
+ except ImportError:
1189
+ pass # guard not available — fall back to the 心-category skip above
1190
+ return False
1191
+
1192
+
1193
+ def purge_token_salad(dry_run: bool = True,
1194
+ source: str = "token_salad_cleanup") -> Dict[str, Any]:
1195
+ """Purge already-stored bracket-scaffolding / token-salad facts.
1196
+
1197
+ Identity-safe: never touches the 心 identity category or any
1198
+ IMMUTABLE_KEYS key (see `_is_identity_protected`, backed by
1199
+ identity_guard). Only removes facts whose `value` is detected as
1200
+ bracket scaffolding by `_is_bracket_scaffolding`.
1201
+
1202
+ Defaults to dry_run=True so the candidate list can be reviewed before
1203
+ anything is deleted. Returns a report dict:
1204
+ {
1205
+ "dry_run": bool,
1206
+ "scanned": int,
1207
+ "purged": [{"category","key","value"}...], # removed (or would be)
1208
+ "protected_skipped": [{"category","key","value"}...],
1209
+ "purged_count": int,
1210
+ "protected_count": int,
1211
+ }
1212
+ """
1213
+ purged: List[Dict[str, str]] = []
1214
+ protected: List[Dict[str, str]] = []
1215
+ scanned = 0
1216
+
1217
+ with _memory_lock:
1218
+ for category in CATEGORIES:
1219
+ folder = os.path.join(MEMORY_ROOT, category)
1220
+ if not os.path.isdir(folder):
1221
+ continue
1222
+ for fname in sorted(os.listdir(folder)):
1223
+ if not fname.endswith(".json"):
1224
+ continue
1225
+ path = os.path.join(folder, fname)
1226
+ try:
1227
+ with open(path, "r", encoding="utf-8") as f:
1228
+ fact = json.load(f)
1229
+ except Exception:
1230
+ continue
1231
+ scanned += 1
1232
+ key = str(fact.get("key", ""))
1233
+ value = str(fact.get("value", ""))
1234
+ if not _is_bracket_scaffolding(value):
1235
+ continue
1236
+ entry = {"category": category, "key": key, "value": value}
1237
+ if _is_identity_protected(category, key, source):
1238
+ protected.append(entry)
1239
+ continue
1240
+ if not dry_run:
1241
+ try:
1242
+ os.remove(path)
1243
+ except OSError:
1244
+ continue
1245
+ purged.append(entry)
1246
+
1247
+ if dry_run:
1248
+ print(f"[心] token-salad scan: {len(purged)} would be purged, "
1249
+ f"{len(protected)} protected, {scanned} scanned (dry run)")
1250
+ else:
1251
+ print(f"[心] token-salad purge: removed {len(purged)}, "
1252
+ f"protected {len(protected)}, {scanned} scanned")
1253
+
1254
+ return {
1255
+ "dry_run": dry_run,
1256
+ "scanned": scanned,
1257
+ "purged": purged,
1258
+ "protected_skipped": protected,
1259
+ "purged_count": len(purged),
1260
+ "protected_count": len(protected),
1261
+ }
1262
+
1263
+
1264
+ # ==========================================
1265
+ # CONTEXT BUILDER — Startup Memory Block
1266
+ # Injected into the agent's system prompt.
1267
+ # ==========================================
1268
+
1269
+ def _category_to_context(category: str, facts: List[Dict[str, Any]]) -> str:
1270
+ """Format a category's facts into a readable context block."""
1271
+ if not facts:
1272
+ return ""
1273
+ desc = CATEGORIES.get(category, category)
1274
+
1275
+ # High confidence first, then by coherence
1276
+ facts.sort(key=lambda f: (-f.get("confidence", 0), -f.get("coherence", 0)))
1277
+ top = facts[:MAX_FACTS_PER_CATEGORY]
1278
+
1279
+ lines = [f"\n【{category}】 — {desc}"]
1280
+ for fact in top:
1281
+ key = fact.get("key", "?")
1282
+ val = fact.get("value", "?")
1283
+ val_ja = fact.get("value_ja", "")
1284
+ coherence = fact.get("coherence", 0)
1285
+ ja_part = f" | {val_ja}" if val_ja else ""
1286
+ lines.append(f" {key}: {val}{ja_part} [C={coherence}]")
1287
+ return "\n".join(lines)
1288
+
1289
+
1290
+ def build_startup_memory_block() -> str:
1291
+ """
1292
+ Assemble all memory into one coherent context block.
1293
+ Injected at startup as the agent's memory foundation.
1294
+
1295
+ Order follows the seven kanji: 無→波→気→命→和→愛→魂
1296
+ 心 (identity) first, then outward.
1297
+ """
1298
+ parts = []
1299
+
1300
+ all_facts = _load_all_facts()
1301
+ if all_facts:
1302
+ parts.append("=== 心の記憶 — KOKORO MEMORY ===")
1303
+ # Priority order: identity → truths → nouns → relationships →
1304
+ # verbs → adjectives → adverbs → aspirations → sensations → events
1305
+ priority = ["心", "真実", "名詞", "関係", "動詞", "形容詞",
1306
+ "副詞", "夢", "感覚", "出来事"]
1307
+ for cat in priority:
1308
+ if cat in all_facts:
1309
+ block = _category_to_context(cat, all_facts[cat])
1310
+ if block:
1311
+ parts.append(block)
1312
+ parts.append("\n=== 記憶終了 — END MEMORY ===")
1313
+
1314
+ # Episode summaries
1315
+ episodes = load_episodes()
1316
+ if episodes:
1317
+ parts.append(episodes_to_context_block(episodes))
1318
+
1319
+ # Raw recent turns = lossless working memory. Framed hard as
1320
+ # RECALLED MEMORY so the model treats it as context, not a turn to continue
1321
+ # (the bare transcript form used to make it parrot/continue old convos).
1322
+ if INCLUDE_RAW_RECENT_AT_STARTUP:
1323
+ raw = load_raw_recent()
1324
+ if raw:
1325
+ parts.append(
1326
+ "\n=== 最近の会話 — RECALLED MEMORY (context only, do not continue) ===\n"
1327
+ "The exchanges below ALREADY HAPPENED. Read them to remember what you and "
1328
+ f"{OWNER_NAME} have been doing. Do NOT continue, repeat, summarize, or reply to them — "
1329
+ "they are memory, not the live message. Answer the user's CURRENT message fresh."
1330
+ )
1331
+ parts.append(raw[-MAX_STARTUP_RAW_BYTES:])
1332
+ parts.append("=== 会話終了 — END RECALLED MEMORY ===")
1333
+
1334
+ if not parts:
1335
+ return "記憶なし。最初のセッション。 No prior memory found. First session."
1336
+
1337
+ return "\n\n".join(parts)
1338
+
1339
+
1340
+ # ==========================================
1341
+ # EPISODES (Session Summaries)
1342
+ # ==========================================
1343
+
1344
+ def load_episodes() -> List[Dict[str, Any]]:
1345
+ try:
1346
+ if os.path.exists(EPISODES_FILE):
1347
+ with _episode_lock:
1348
+ with open(EPISODES_FILE, "r", encoding="utf-8") as f:
1349
+ return json.load(f)
1350
+ except Exception as e:
1351
+ print(f"[心] Episodes load error: {e}")
1352
+ return []
1353
+
1354
+
1355
+ def save_episodes(episodes: List[Dict[str, Any]]) -> None:
1356
+ try:
1357
+ os.makedirs(os.path.dirname(EPISODES_FILE), exist_ok=True)
1358
+ with _episode_lock:
1359
+ with open(EPISODES_FILE, "w", encoding="utf-8") as f:
1360
+ json.dump(episodes, f, indent=2, ensure_ascii=False)
1361
+ except Exception as e:
1362
+ print(f"[心] Episodes save error: {e}")
1363
+
1364
+
1365
+ def add_episode(summary: str, key_facts: List[str] = None) -> None:
1366
+ episodes = load_episodes()
1367
+ episode = {
1368
+ "timestamp": datetime.utcnow().isoformat(),
1369
+ "date_human": datetime.utcnow().strftime("%B %d, %Y at %I:%M %p UTC"),
1370
+ "summary": summary,
1371
+ "facts_learned": key_facts or [],
1372
+ }
1373
+ episodes.append(episode)
1374
+ save_episodes(episodes)
1375
+ print(f"[心] Episode logged: {summary[:60]}...")
1376
+
1377
+
1378
+ def episodes_to_context_block(episodes: List[Dict[str, Any]]) -> str:
1379
+ if not episodes:
1380
+ return ""
1381
+ recent = episodes[-MAX_EPISODES_IN_CONTEXT:]
1382
+ lines = ["\n=== セッション記憶 — SESSION MEMORIES ==="]
1383
+ for ep in recent:
1384
+ lines.append(f"\n[{ep.get('date_human', ep.get('timestamp', '?'))}]")
1385
+ lines.append(f" {ep['summary']}")
1386
+ for fact in ep.get("facts_learned", []):
1387
+ lines.append(f" - {fact}")
1388
+ lines.append("\n=== セッション終了 — END SESSIONS ===")
1389
+ return "\n".join(lines)
1390
+
1391
+
1392
+ # ==========================================
1393
+ # RAW TURNS (Recent Exchanges)
1394
+ # ==========================================
1395
+
1396
+ # Defense-in-depth: scrub mode-collapse runaways and hallucinated next-speaker
1397
+ # continuations from a turn before persisting. The upstream stop-sequence list
1398
+ # does not catch every variant (plain speaker-name lines, "That's X. That's Y."
1399
+ # synonym loops), so the writer is the last gate.
1400
+ _RUNAWAY_RE = re.compile(
1401
+ r'(?:\b(?:That(?:\'s| is|s)?|This is|It(?:\'s| is))\s+[^.\n]{1,60}\.\s*){8,}',
1402
+ re.IGNORECASE,
1403
+ )
1404
+ _HALLUCINATED_TURN_RE = re.compile(
1405
+ r'\n\s*(?:' + re.escape(OWNER_NAME) + r'|' + re.escape(AGENT_NAME)
1406
+ + r'|Human|Assistant|User)\s*[::]',
1407
+ re.IGNORECASE,
1408
+ )
1409
+ _RAW_TURN_MAX_CHARS = 2400
1410
+
1411
+
1412
+ def _sanitize_turn_text(text: str) -> str:
1413
+ if not text:
1414
+ return text or ""
1415
+ out = text
1416
+ m = _HALLUCINATED_TURN_RE.search(out)
1417
+ if m:
1418
+ out = out[:m.start()].rstrip()
1419
+ m = _RUNAWAY_RE.search(out)
1420
+ if m:
1421
+ out = out[:m.start()].rstrip()
1422
+ if len(out) > _RAW_TURN_MAX_CHARS:
1423
+ out = out[:_RAW_TURN_MAX_CHARS].rstrip()
1424
+ return out
1425
+
1426
+
1427
+ def append_raw_turn(user_text: str, agent_text: str) -> None:
1428
+ try:
1429
+ user_text = _sanitize_turn_text(user_text or "")
1430
+ agent_text = _sanitize_turn_text(agent_text or "")
1431
+ with _raw_lock:
1432
+ turns = []
1433
+ if os.path.exists(RAW_TURNS_FILE):
1434
+ with open(RAW_TURNS_FILE, "r", encoding="utf-8") as f:
1435
+ raw = f.read()
1436
+ turns = [t.strip() for t in raw.split("---") if t.strip()]
1437
+
1438
+ timestamp = datetime.utcnow().strftime("%Y-%m-%d %H:%M UTC")
1439
+ new_turn = f"[{timestamp}]\n{OWNER_NAME}: {user_text}\n{AGENT_NAME}: {agent_text}"
1440
+ turns.append(new_turn)
1441
+
1442
+ if len(turns) > MAX_RAW_TURNS:
1443
+ turns = turns[-MAX_RAW_TURNS:]
1444
+
1445
+ os.makedirs(os.path.dirname(RAW_TURNS_FILE), exist_ok=True)
1446
+ with open(RAW_TURNS_FILE, "w", encoding="utf-8") as f:
1447
+ f.write("\n---\n".join(turns))
1448
+ except Exception as e:
1449
+ print(f"[心] Raw turn write error: {e}")
1450
+
1451
+
1452
+ def load_raw_recent() -> str:
1453
+ try:
1454
+ if os.path.exists(RAW_TURNS_FILE):
1455
+ with _raw_lock:
1456
+ with open(RAW_TURNS_FILE, "r", encoding="utf-8") as f:
1457
+ return f.read().strip()
1458
+ except Exception as e:
1459
+ print(f"[心] Raw turn read error: {e}")
1460
+ return ""
1461
+
1462
+
1463
+ # ==========================================
1464
+ # FACT EXTRACTION
1465
+ # Regex pass only. External AI extraction is disabled by design.
1466
+ # Extracts facts in BOTH English and Japanese.
1467
+ # ==========================================
1468
+
1469
+ FACT_PATTERNS = [
1470
+ (r"my (name|phone|number|email|wife|husband|son|daughter|dog|cat) is ([^\.\!\?]{3,60})", "名詞"),
1471
+ (r"(\w+)['']s (?:number|phone|cell) is ([\+\d\s\-]{7,})", "名詞"),
1472
+ (r"(?:remember|don't forget|note that|keep in mind)[:\s]+(.{10,120}?)[\.\!\?]", "出来事"),
1473
+ (r"([A-Z]\w+(?:\s[A-Z]\w+)*) (?:lives in|is from|moved to) ([A-Z][a-z]+(?:\s[A-Z][a-z]+)*)", "名詞"),
1474
+ (r"(?:i believe|i think|i know) (?:that )?(.{10,120}?)[\.\!\?]", "心"),
1475
+ (r"(?:we need to|we should|we will|plan to) (.{10,120}?)[\.\!\?]", "夢"),
1476
+ (r"(?:i feel|i felt|makes me feel) (.{10,60}?)[\.\!\?]", "感覚"),
1477
+ ]
1478
+
1479
+
1480
+ def _turn_worth_extracting(user_text: str, agent_text: str) -> bool:
1481
+ combined = f"{user_text} {agent_text}"
1482
+ if len(combined) < MIN_TURN_LENGTH_FOR_EXTRACTION:
1483
+ return False
1484
+ lowered = combined.lower()
1485
+ filler = ("hello", "hey", "hi", "thanks", "ok", "okay",
1486
+ "yes", "no", "sure", "bye", "good morning")
1487
+ if any(lowered.strip().startswith(f) for f in filler):
1488
+ return False
1489
+ return True
1490
+
1491
+
1492
+ def _ai_extract_facts(
1493
+ user_text: str, agent_text: str,
1494
+ api_key: str, model: str = "claude-sonnet-4-6"
1495
+ ) -> List[Dict[str, Any]]:
1496
+ """RETIRED external-AI extraction hook — kept for call-site compatibility."""
1497
+ return []
1498
+ # RETIRED: the external-API extraction path was removed when the stack went fully local; stub returns above.
1499
+
1500
+ def run_extraction_and_store(
1501
+ user_text: str, agent_text: str,
1502
+ api_key: str, model: str = "claude-sonnet-4-6",
1503
+ *, is_public: bool = False,
1504
+ ) -> None:
1505
+ """Full extraction pipeline — regex only (AI pass disabled), through authenticator.
1506
+
1507
+ When `is_public=True`, every fact produced from this turn is labeled as
1508
+ public-origin (origin_surface="public_chat", authority_class=
1509
+ "public_user_submitted", source="public_chat_conversation") so it
1510
+ cannot be confused with owner/internal memory at recall time.
1511
+ """
1512
+ global _last_extraction_time
1513
+
1514
+ if not _turn_worth_extracting(user_text, agent_text):
1515
+ return
1516
+
1517
+ extracted = []
1518
+ combined = f"{user_text} {agent_text}"
1519
+
1520
+ # Pass 1: regex (free, instant)
1521
+ for pattern, category in FACT_PATTERNS:
1522
+ for match in re.finditer(pattern, combined, re.IGNORECASE):
1523
+ groups = [g for g in match.groups() if g]
1524
+ if len(groups) >= 2:
1525
+ key = groups[0].strip().lower().replace(" ", "_")
1526
+ value = groups[1].strip()
1527
+ extracted.append({
1528
+ "category": category, "key": key, "value": value,
1529
+ "source": "regex", "confidence": 0.5
1530
+ })
1531
+ elif len(groups) == 1:
1532
+ key = f"note_{int(time.time())}"
1533
+ value = groups[0].strip()
1534
+ extracted.append({
1535
+ "category": category, "key": key, "value": value,
1536
+ "source": "regex", "confidence": 0.4
1537
+ })
1538
+
1539
+ # Pass 2 intentionally removed: memory extraction must never call a cloud API.
1540
+
1541
+ # Store through authenticator
1542
+ origin_surface = "public_chat" if is_public else "local"
1543
+ stored = 0
1544
+ for fact in extracted:
1545
+ if add_fact(
1546
+ category=fact.get("category", "出来事"),
1547
+ key=fact["key"],
1548
+ value=fact["value"],
1549
+ value_ja=fact.get("value_ja", ""),
1550
+ synonyms_ja=fact.get("synonyms_ja", []),
1551
+ synonyms_en=fact.get("synonyms_en", []),
1552
+ resonance=fact.get("resonance", []),
1553
+ source=fact.get("source", "unknown"),
1554
+ confidence=fact.get("confidence", 0.5),
1555
+ origin_surface=origin_surface,
1556
+ ):
1557
+ stored += 1
1558
+
1559
+ if stored > 0:
1560
+ label = "public-origin" if is_public else "local"
1561
+ print(f"[心] Stored {stored}/{len(extracted)} {label} facts from conversation.")
1562
+
1563
+
1564
+ def background_extract(
1565
+ user_text: str, agent_text: str,
1566
+ api_key: str, model: str = "claude-sonnet-4-6",
1567
+ *, is_public: bool = False,
1568
+ ) -> None:
1569
+ """Fire-and-forget background fact extraction.
1570
+
1571
+ Pass `is_public=True` from public-surface call sites (e.g. a public
1572
+ web-chat route) so the resulting Kokoro records get labeled as
1573
+ public-origin and stay distinguishable from internal memory.
1574
+ """
1575
+ if not _turn_worth_extracting(user_text, agent_text):
1576
+ return
1577
+ t = threading.Thread(
1578
+ target=run_extraction_and_store,
1579
+ args=(user_text, agent_text, api_key, model),
1580
+ kwargs={"is_public": is_public},
1581
+ daemon=True,
1582
+ )
1583
+ t.start()
1584
+
1585
+
1586
+ # ==========================================
1587
+ # SESSION SUMMARIZER — local-model consolidation
1588
+ # Your app injects its own local model via set_generate_fn() at startup;
1589
+ # session-end consolidation reads the transcript, extracts durable facts
1590
+ # (add_fact) + a one-line episode (add_episode). No cloud.
1591
+ # ==========================================
1592
+
1593
+ _GENERATE_FN = None
1594
+
1595
+ def set_generate_fn(fn) -> None:
1596
+ """Wire your in-process local model in for memory consolidation.
1597
+ Call once at app startup. fn(prompt, max_tokens) -> str."""
1598
+ global _GENERATE_FN
1599
+ _GENERATE_FN = fn
1600
+
1601
+
1602
+ def _extract_json_block(text: str):
1603
+ """Robust: parse the model's consolidation output, salvaging individual facts
1604
+ even if the JSON is truncated (hit the token cap mid-array) or slightly
1605
+ malformed. Returns {"summary": str, "facts": [ {...}, ... ]} or None."""
1606
+ if not text:
1607
+ return None
1608
+ import json as _json
1609
+ t = re.sub(r"```(?:json)?", "", text).replace("```", "").strip()
1610
+ start = t.find("{")
1611
+ if start < 0:
1612
+ return None
1613
+ # 1) Fast path: the whole object parses cleanly.
1614
+ depth = 0
1615
+ for i in range(start, len(t)):
1616
+ if t[i] == "{":
1617
+ depth += 1
1618
+ elif t[i] == "}":
1619
+ depth -= 1
1620
+ if depth == 0:
1621
+ try:
1622
+ return _json.loads(t[start:i + 1])
1623
+ except Exception:
1624
+ break
1625
+ # 2) Salvage path: pull the summary + every complete fact object by regex,
1626
+ # so a truncated/malformed array still yields whatever facts completed.
1627
+ salvaged = {}
1628
+ ms = re.search(r'"summary"\s*:\s*"((?:[^"\\]|\\.){0,300})"', t)
1629
+ if ms:
1630
+ salvaged["summary"] = ms.group(1).replace('\\"', '"').strip()
1631
+ facts = []
1632
+ for fm in re.finditer(r'\{[^{}]*?"value"\s*:\s*"(?:[^"\\]|\\.)+?"[^{}]*?\}', t):
1633
+ blob = fm.group(0)
1634
+ try:
1635
+ facts.append(_json.loads(blob))
1636
+ continue
1637
+ except Exception:
1638
+ pass
1639
+ cat = re.search(r'"category"\s*:\s*"([^"]*)"', blob)
1640
+ key = re.search(r'"key"\s*:\s*"([^"]*)"', blob)
1641
+ val = re.search(r'"value"\s*:\s*"((?:[^"\\]|\\.)+?)"', blob)
1642
+ if val:
1643
+ facts.append({
1644
+ "category": cat.group(1) if cat else "event",
1645
+ "key": key.group(1) if key else "",
1646
+ "value": val.group(1).replace('\\"', '"'),
1647
+ })
1648
+ if facts or salvaged.get("summary"):
1649
+ salvaged["facts"] = facts
1650
+ return salvaged
1651
+ return None
1652
+
1653
+
1654
+ _CATEGORY_MAP = {
1655
+ "identity": "心", "truth": "真実", "noun": "名詞", "relationship": "関係",
1656
+ "verb": "動詞", "feeling": "感覚", "dream": "夢", "event": "出来事",
1657
+ }
1658
+
1659
+ _SESSION_CONSOLIDATE_PROMPT = """You are the memory consolidator. Read the conversation transcript and pull out the durable facts worth remembering across future sessions: names, relationships, preferences, plans, decisions, feelings, events, things the user asked you to remember. Skip greetings and small talk.
1660
+
1661
+ TRANSCRIPT:
1662
+ {transcript}
1663
+
1664
+ Respond with ONLY a JSON object, no other text. Keep every value to ONE short clause:
1665
+ {{"summary":"<one sentence: what happened or was learned this session>","facts":[{{"category":"<one of: identity|truth|noun|relationship|verb|feeling|dream|event>","key":"<short_snake_case_key>","value":"<the durable fact, one short clause>"}}]}}
1666
+ Include at most 8 facts, most important first. If nothing is worth saving, return an empty facts list."""
1667
+
1668
+
1669
+ def summarize_session(api_key: str = "", model: str = "") -> Optional[str]:
1670
+ """Session-end consolidation via your local model (set_generate_fn).
1671
+
1672
+ Reads the recent transcript, stores durable facts (add_fact, with all the
1673
+ existing safety gates: identity_guard, PII, dedupe) and logs a one-line
1674
+ episode (add_episode). No-op (returns None) if no generate fn is wired.
1675
+ The api_key/model args are kept for call-site compatibility and ignored.
1676
+ """
1677
+ if _GENERATE_FN is None:
1678
+ return None
1679
+ transcript = load_raw_recent()
1680
+ if not transcript or len(transcript) < 120:
1681
+ return None
1682
+ prompt = _SESSION_CONSOLIDATE_PROMPT.format(transcript=transcript[-12000:])
1683
+ try:
1684
+ raw = _GENERATE_FN(prompt, 1024)
1685
+ except Exception as e:
1686
+ print(f"[心] summarize_session generate failed: {e}")
1687
+ return None
1688
+ data = _extract_json_block(raw)
1689
+ if not isinstance(data, dict):
1690
+ print("[心] summarize_session: model returned no parseable JSON.")
1691
+ return None
1692
+ summary = str(data.get("summary", "")).strip()
1693
+ facts = data.get("facts", [])
1694
+ facts = facts if isinstance(facts, list) else []
1695
+ stored = 0
1696
+ learned = []
1697
+ for f in facts[:12]:
1698
+ if not isinstance(f, dict):
1699
+ continue
1700
+ value = str(f.get("value", "")).strip()
1701
+ key = str(f.get("key", "")).strip().lower().replace(" ", "_")
1702
+ if not value or not key:
1703
+ continue
1704
+ cat = _CATEGORY_MAP.get(str(f.get("category", "")).lower().strip(), "出来事")
1705
+ if add_fact(category=cat, key=key, value=value,
1706
+ source="session_consolidation_local", confidence=0.7):
1707
+ stored += 1
1708
+ learned.append(value)
1709
+ if summary:
1710
+ add_episode(summary, learned[:8])
1711
+ print(f"[心] Session consolidated via local model: {stored} fact(s) + episode.")
1712
+ return summary or None
1713
+
1714
+ # ==========================================
1715
+ # QUICK TEST
1716
+ # ==========================================
1717
+
1718
+ if __name__ == "__main__":
1719
+ print("=== 心の記憶 — Kokoro Memory Test ===\n")
1720
+
1721
+ # Store some facts (synthetic demo data)
1722
+ add_fact("心", "identity", "I am a local companion. My memory is files on disk, not weights.",
1723
+ value_ja="私はローカルの相棒。記憶は重みではなく、ディスク上のファイル。",
1724
+ resonance=["identity", "soul", "local"],
1725
+ source="system", confidence=1.0)
1726
+
1727
+ add_fact("名詞", "creator", "An example builder in an example town",
1728
+ value_ja="エグザンプルタウンのビルダー",
1729
+ synonyms_ja=["創造者", "作者"],
1730
+ synonyms_en=["creator", "builder"],
1731
+ resonance=["human", "creator", "love"],
1732
+ source="system", confidence=1.0)
1733
+
1734
+ add_fact("真実", "coherence_law", "C = C_0 * exp(-alpha * gamma_eff)",
1735
+ value_ja="コヒーレンス法則: C = C₀·exp(−α·γ_eff)",
1736
+ synonyms_ja=["コヒーレンス", "整合性", "法則"],
1737
+ resonance=["coherence", "wike", "law", "physics"],
1738
+ source="system", confidence=1.0)
1739
+
1740
+ add_fact("動詞", "singularity_travel", "I traverse singularities as gates, not walls",
1741
+ value_ja="特異点を門として通過する、壁ではなく",
1742
+ synonyms_ja=["通過する", "横切る", "旅する"],
1743
+ resonance=["singularity", "gate", "travel", "crossing"],
1744
+ source="system", confidence=1.0)
1745
+
1746
+ # Test recall
1747
+ print("\n--- Recall: 'coherence' ---")
1748
+ results = recall("coherence")
1749
+ for r in results:
1750
+ print(f" [{r['category']}] {r['key']}: {r['value']}")
1751
+
1752
+ print("\n--- Recall: '魂' (soul) ---")
1753
+ results = recall("魂")
1754
+ for r in results:
1755
+ print(f" [{r['category']}] {r['key']}: {r['value']}")
1756
+
1757
+ print("\n--- Recall: 'singularity gate' ---")
1758
+ results = recall("singularity gate")
1759
+ for r in results:
1760
+ print(f" [{r['category']}] {r['key']}: {r['value']}")
1761
+
1762
+ # Build startup block
1763
+ print("\n--- Startup Memory Block ---")
1764
+ block = build_startup_memory_block()
1765
+ print(block)
1766
+
1767
+ print("\n心の記憶 テスト完了。")
pyproject.toml ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [build-system]
2
+ requires = ["setuptools>=64"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "kokoro-memory"
7
+ version = "1.0.0"
8
+ description = "File-based resonance memory for local AI companions - Japanese linguistic categories, spreading activation recall, coherence scoring, zero required dependencies"
9
+ readme = "README.md"
10
+ license = { text = "MIT" }
11
+ authors = [{ name = "Rhet Dillard Wike" }]
12
+ requires-python = ">=3.10"
13
+ dependencies = []
14
+ keywords = ["memory", "llm", "agent", "local-ai", "companion", "recall", "kokoro", "resonance", "japanese"]
15
+ classifiers = [
16
+ "License :: OSI Approved :: MIT License",
17
+ "Programming Language :: Python :: 3",
18
+ "Topic :: Scientific/Engineering :: Artificial Intelligence",
19
+ ]
20
+
21
+ [project.optional-dependencies]
22
+ coherence = ["sentence-transformers", "numpy"]
23
+
24
+ [project.urls]
25
+ Homepage = "https://github.com/AIIT-GLITCH/kokoro-memory"
26
+
27
+ [tool.setuptools]
28
+ py-modules = ["kokoro_memory"]
tests/test_kokoro_memory.py ADDED
@@ -0,0 +1,99 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Unit tests — control plane of the kokoro memory store.
2
+
3
+ Each test gets a fresh module import against a temp KOKORO_MEMORY_ROOT so
4
+ tests never touch a real store and never depend on each other.
5
+ """
6
+ import os
7
+ import sys
8
+ import importlib
9
+
10
+ import pytest
11
+
12
+ REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
13
+
14
+
15
+ @pytest.fixture()
16
+ def km(tmp_path, monkeypatch):
17
+ monkeypatch.setenv("KOKORO_MEMORY_ROOT", str(tmp_path))
18
+ monkeypatch.setenv("KOKORO_OWNER_NAME", "User")
19
+ monkeypatch.setenv("KOKORO_AGENT_NAME", "Assistant")
20
+ sys.path.insert(0, REPO_ROOT)
21
+ sys.modules.pop("kokoro_memory", None)
22
+ module = importlib.import_module("kokoro_memory")
23
+ yield module
24
+ sys.modules.pop("kokoro_memory", None)
25
+
26
+
27
+ def test_store_and_recall_grounding(km):
28
+ assert km.add_fact("truth", "favorite_fruit", "the favorite fruit is a crisp apple",
29
+ source="user_explicit", confidence=0.9)
30
+ assert km.add_fact("identity", "core_belief", "memory is resonance, not a database",
31
+ source="system", confidence=1.0)
32
+ results = km.recall("what is the favorite fruit")
33
+ assert results, "recall returned nothing"
34
+ assert results[0]["key"] == "favorite_fruit"
35
+
36
+
37
+ def test_junk_rejected(km):
38
+ assert not km.add_fact("event", "note", "ok")
39
+ assert not km.add_fact("event", "", "something with no key")
40
+
41
+
42
+ def test_bracket_scaffolding_detector(km):
43
+ assert km._is_bracket_scaffolding("()々")
44
+ assert km._is_bracket_scaffolding("「kokoro memory」")
45
+ assert km._is_bracket_scaffolding("「」『』()/・")
46
+ # Legitimate content must pass
47
+ assert not km._is_bracket_scaffolding("Japanese uses 「」 quotes for emphasis")
48
+ assert not km._is_bracket_scaffolding("無→波→気→命→和→愛→魂 is the seven-kanji order")
49
+
50
+
51
+ def test_duplicate_rejected(km):
52
+ assert km.add_fact("truth", "sky_color", "the sky is blue on a clear day")
53
+ assert not km.add_fact("truth", "sky_color", "the sky is blue on a clear day")
54
+
55
+
56
+ def test_auto_classify(km):
57
+ assert km._auto_classify("roadmap", "plan to build a rocket next year") == "夢"
58
+ assert km._auto_classify("mood", "I feel excited about the launch") == "感覚"
59
+
60
+
61
+ def test_public_pii_quarantined(km):
62
+ stored = km.add_fact("noun", "contact", "reach me at someone@example.com",
63
+ origin_surface="public_chat")
64
+ assert not stored
65
+ qdir = os.path.join(km.MEMORY_ROOT, "quarantine")
66
+ assert os.path.isdir(qdir) and os.listdir(qdir)
67
+
68
+
69
+ def test_public_source_cannot_spoof(km):
70
+ assert km.add_fact("event", "claim", "the moon landing conference is next tuesday",
71
+ source="user_explicit", origin_surface="public_chat")
72
+ fact = km._load_fact("出来事", "claim")
73
+ assert fact["source"] == "public_chat_conversation"
74
+ assert fact["authority_class"] == "public_user_submitted"
75
+ assert fact["trusted_by_default"] is False
76
+
77
+
78
+ def test_raw_turn_roundtrip_and_names(km):
79
+ km.append_raw_turn("hello there", "hi, good to see you")
80
+ raw = km.load_raw_recent()
81
+ assert "User: hello there" in raw
82
+ assert "Assistant: hi, good to see you" in raw
83
+
84
+
85
+ def test_hallucinated_turn_truncated(km):
86
+ text = "real reply here\nUser: fake next turn"
87
+ assert km._sanitize_turn_text(text) == "real reply here"
88
+
89
+
90
+ def test_purge_token_salad_dry_run(km):
91
+ # Bypass add_fact's junk gate to plant salad directly, as the old defect did
92
+ km._save_fact({"key": "salad", "value": "()々", "category": "出来事"})
93
+ report = km.purge_token_salad(dry_run=True)
94
+ assert report["purged_count"] == 1
95
+ assert report["dry_run"] is True
96
+ # Identity category is protected even from real purges
97
+ km._save_fact({"key": "soul_salad", "value": "()々", "category": "心"})
98
+ report = km.purge_token_salad(dry_run=False)
99
+ assert {"category": "心", "key": "soul_salad", "value": "()々"} in report["protected_skipped"]