Publish PRISM-Memory adapter bundle

README.md

@@ -161,7 +161,7 @@ More held-out examples live in
 - [docs/release/technical-blog.md](docs/release/technical-blog.md)
 - [results/release_summary.json](results/release_summary.json)
 - [results/extraction_examples.json](results/extraction_examples.json)
-- [results/benchmark_cases.json](results/benchmark_cases.json)
+- [results/try_it_sessions.json](results/try_it_sessions.json)
 
 ## Demo
 

docs/release/memory-scenarios.md

@@ -3,9 +3,10 @@
 These are compact product-style scenarios built from the public release
 artifacts.
 
-- The first two use the released held-out extraction examples.
-- The last two use confirmed held-out benchmark cases from
-  [../../results/benchmark_cases.json](../../results/benchmark_cases.json).
+- The first three come from the bundled interactive session artifact
+  [../../results/try_it_sessions.json](../../results/try_it_sessions.json).
+- The last one comes from the held-out extraction examples in
+  [../../results/extraction_examples.json](../../results/extraction_examples.json).
 
 The point is not just that the extractor matches GPT-4.1-style labels. The
 point is that a later system can ask a concrete question and get back a useful,
@@ -13,93 +14,113 @@ inspectable answer from stored memory.
 
 ## 1. Keep hard limits and notification preferences
 
-**Conversation turn**
+**Conversation turns**
 
-> yeah, I think starting with incremental scans and parallel matrix jobs makes sense. We have 20 concurrent jobs max on GitHub Actions currently. Also want to keep Slack notifications from Snyk consistent with other pipeline alerts, aggregated and concise.
+> [2025-03-01] Dana: We have 20 concurrent jobs max on GitHub Actions right now. Keep Snyk Slack notifications aggregated and concise, not one alert per repo.  
+> [2025-03-07] Dana: No mTLS yet. Put it in phase two after the canary rollout.
 
 **Stored memory**
 
 - GitHub Actions concurrency limit: 20 concurrent jobs
-- Snyk Slack notifications should be aggregated and concise
+- Snyk Slack notifications aggregated and concise; no separate alerts per repo
+- No mutual TLS enabled; mutual TLS will be implemented in phase two after the canary rollout
 
 **Later question**
 
-What is our GitHub Actions concurrency limit, and how should Snyk alerts look?
+What is the current concurrency limit, how should Slack alerts behave, and when
+is mTLS planned?
 
 **Answer from memory**
 
-20 concurrent jobs. Snyk alerts should be aggregated and concise.
+The current GitHub Actions concurrency limit is 20 jobs. Slack notifications
+should be aggregated and concise. mTLS is planned for phase two after the
+canary rollout.
 
 **Why it matters**
 
 This is the kind of operational detail that gets buried in chat but needs to
 survive into later workflow drafts and agent actions.
 
-## 2. Keep current state separate from the roadmap
+## 2. Replace stale travel plans with the current one
 
-**Conversation turn**
+**Conversation turns**
 
-> yeah good point about resource overhead, we set CPU limits for all sidecars and monitor with Prometheus now. no mTLS yet, but it’s on the roadmap for phase two. as for routing, we want to start with canary deployments and traffic splitting, maybe some basic fault injection for testing.
+> [2025-06-12] Maya: We booked the Lisbon trip for September 14. I want a quiet hotel near Alfama, and no red-eye flights.  
+> [2025-07-02] Maya: Update the plan: Lisbon is off. We are going to Porto on September 21 instead, still no red-eye flights.
 
 **Stored memory**
 
-- Sidecar CPU limits set and monitored via Prometheus
-- Istio mTLS planned for phase two
-- Routing strategy: canary deployments and traffic splitting; basic fault injection planned
+- Lisbon trip booked for September 14
+- Prefers a quiet hotel near Alfama
+- Prefers no red-eye flights
+- Lisbon trip canceled; Porto trip scheduled for September 21
+- No red-eye flights for Porto trip
 
 **Later question**
 
-Did we already enable mTLS, and what rollout strategy are we planning?
+Where is the trip now, on what date, and what flight constraint still applies?
 
 **Answer from memory**
 
-mTLS is not enabled yet; it is planned for phase two. The rollout plan is
-canary deployments and traffic splitting, with basic fault injection planned.
+The current trip is to Porto on September 21, 2025, and red-eye flights are
+still off limits.
 
 **Why it matters**
 
-Memory systems often blur the current state with the planned state. This is the
-kind of distinction that matters in deployment and incident work.
+Memory systems often keep both the old and new plan, but the downstream system
+still has to recover the latest valid state.
+
+## 3. Keep routines and health constraints available later
 
-## 3. Answer dated questions instead of only remembering themes
+**Conversation turns**
 
-**Question**
+> [2025-08-03] Sam: My doctor wants me to keep sodium under 2 grams a day. I started painting on weekends because it helps me decompress.  
+> [2025-08-17] Sam: I bought watercolors and signed up for a Saturday class downtown.  
+> [2025-09-01] Sam: Skip late-night coffee from now on; it wrecks my sleep.
 
-Which hobby did Sam take up in May 2023?
+**Stored memory**
+
+- Doctor recommended Sam limit sodium intake to 2 grams per day
+- Sam paints on weekends as a decompression activity
+- Sam bought watercolors and signed up for a Saturday painting class downtown
+- Sam decided to skip late-night coffee starting now
 
-**Retrieved memory**
+**Later question**
 
-- Sam: [18 May 2023] Sam is considering trying painting as a new hobby.
-- Sam: [24 May 2023] Sam has been considering trying painting as a new hobby.
+What hobby did Sam start, what class did he sign up for, and what health
+constraints matter now?
 
 **Answer from memory**
 
-painting
+Sam started painting, signed up for a Saturday watercolor class downtown,
+should keep sodium under 2 grams per day, and wants to avoid late-night coffee.
 
 **Why it matters**
 
-A useful memory system should not just remember that someone talked about
-hobbies. It should recover the dated fact that actually answers the later
-question.
+This is the difference between transcript search and usable memory. The durable
+facts can drive reminders, planning, or coaching later.
 
-## 4. Refuse unsupported claims instead of inventing a reason
+## 4. Keep infrastructure bottlenecks structured
 
-**Question**
+**Conversation turn**
+
+> yeah, no real caching beyond basic Docker layer caching. Jenkins nodes have limited capacity, and we sometimes hit queue delays during peak commits.
 
-Why did Dave get his guitar customized with a shiny finish?
+**Stored memory**
 
-**Retrieved memory**
+- No Docker caching beyond basic layer caching
+- Jenkins nodes have limited capacity; peak commits cause queue delays
+
+**Later question**
 
-- Dave: That guitar has a gorgeous purple hue. Why did you make it so shiny?
-- Good pick! The customized purple glow gives it a unique look that really stands out.
-- Dave: The guitar was in bad condition when Dave found it.
+What is our current caching setup, and why do builds sometimes queue up?
 
 **Answer from memory**
 
-None / unsupported
+There is no special caching beyond basic Docker layer caching. Builds queue up
+because Jenkins nodes have limited capacity during peak commits.
 
 **Why it matters**
 
-Memory systems are more useful when they can refuse cleanly. Here the retrieved
-context talks about the guitar and the finish, but it never actually supports
-the premise that Dave customized it for a specific reason.
+The stored memory stays compact and directly useful. A later system does not
+need to reread the full conversation turn to answer the operational question.

docs/release/release-results.md

@@ -69,7 +69,7 @@ justify two public models.
 
 - [../../results/release_summary.json](../../results/release_summary.json)
 - [../../results/release_model.json](../../results/release_model.json)
-- [../../results/benchmark_cases.json](../../results/benchmark_cases.json)
+- [../../results/try_it_sessions.json](../../results/try_it_sessions.json)
 - [../../results/internal_locomo_pairwise_diffs.json](../../results/internal_locomo_pairwise_diffs.json)
 
 Related docs:

results/benchmark_cases.json

@@ -1,145 +0,0 @@
-{
-  "qa_cache": {
-    "model": "gpt-4.1",
-    "cache_size": 16969,
-    "hits": 5,
-    "misses": 0,
-    "missing_examples": []
-  },
-  "scenarios": [
-    {
-      "id": "temporal_anchor_hobby",
-      "title": "Temporal Anchor",
-      "source_id": "conv-49",
-      "category": 2,
-      "question": "Which hobby did Sam take up in May 2023?",
-      "gold_answer": "painting",
-      "kind": "strength",
-      "note": "The released model keeps the dated hobby proposition and answers correctly.",
-      "systems": [
-        {
-          "name": "release_model",
-          "display_name": "PRISM-Memory 7B Adapter",
-          "prediction": "painting",
-          "top_retrieval": [
-            "Sam: [18 May 2023] Sam is considering trying painting as a new hobby.",
-            "Sam: [24 May 2023] Sam has been considering trying painting as a new hobby.",
-            "Sam: [6 October 2023] Sam asked Evan if he has explored any fun indoor activities or hobbies.",
-            "Sam: [18 May 2023] Sam is excited to try new things.",
-            "Sam: [24 May 2023] Sam is trying to break old habits.",
-            "Sam: [15 August 2023] Sam attended a cooking class.",
-            "Sam: [17 December 2023] Sam used to love hiking.",
-            "[1:47 pm on 18 May, 2023] Sam: We hiked a good distance - quite a feat for me back then. It's definitely a great memory."
-          ]
-        }
-      ]
-    },
-    {
-      "id": "adversarial_refusal_guitar",
-      "title": "Adversarial Refusal",
-      "source_id": "conv-50",
-      "category": 5,
-      "question": "Why did Dave get his guitar customized with a shiny finish?",
-      "gold_answer": "",
-      "kind": "strength",
-      "note": "This tests whether the system refuses to invent an answer when the premise is unsupported.",
-      "systems": [
-        {
-          "name": "release_model",
-          "display_name": "PRISM-Memory 7B Adapter",
-          "prediction": "None",
-          "top_retrieval": [
-            "[2:55 pm on 31 August, 2023] Dave: That guitar has a gorgeous purple hue. Why did you make it so shiny?",
-            "Dave: [2 November 2023] The guitar was in bad condition when Dave found it.",
-            "[2:55 pm on 31 August, 2023] Dave: Good pick! The customized purple glow gives it a unique look that really stands out.",
-            "[2:55 pm on 31 August, 2023] Dave: That's a great guitar, Calvin! Love the design, it's so unique and special.",
-            "Dave: [16 May 2023] Calvin lost his guitar and amp but managed to save his music gear and microphone."
-          ]
-        }
-      ]
-    },
-    {
-      "id": "diagnosis_specificity",
-      "title": "Diagnosis Specificity",
-      "source_id": "conv-49",
-      "category": 1,
-      "question": "Which ailment does Sam have to face due to his weight?",
-      "gold_answer": "gastritis",
-      "kind": "failure",
-      "note": "A representative factual miss: the model retrieves the health-risk frame but not the specific diagnosis.",
-      "systems": [
-        {
-          "name": "release_model",
-          "display_name": "PRISM-Memory 7B Adapter",
-          "prediction": "serious health risk",
-          "top_retrieval": [
-            "Sam: [8 October 2023] The doctor told Sam that his weight is a serious health risk.",
-            "Sam: [24 May 2023] The doctor's check-up revealed that Sam's weight was not good.",
-            "Sam: [13 August 2023] Sam is currently experiencing challenges affecting his health.",
-            "[6:48 pm on 17 December, 2023] Sam: Yeah, I'm struggling with my weight and it's affecting my confidence. I feel like I can't overcome all the challenges with my weight, I keep lacking motivation.",
-            "Sam: [21 November 2023] Sam has been trying to make dietary changes to address his discomfort.",
-            "Sam: [9 November 2023] Sam is a Weight Watchers coach in his group.",
-            "Sam: [7 August 2023] Sam has been prioritizing his health for some time.",
-            "Sam: [15 August 2023] Sam is concerned about his health."
-          ]
-        }
-      ]
-    },
-    {
-      "id": "location_inference",
-      "title": "Location Inference",
-      "source_id": "conv-49",
-      "category": 3,
-      "question": "Does Evan live close to a beach or mountains?",
-      "gold_answer": "beach",
-      "kind": "failure",
-      "note": "A representative inferential miss: retrieval includes both clues, but the model overcommits to the mountain mention.",
-      "systems": [
-        {
-          "name": "release_model",
-          "display_name": "PRISM-Memory 7B Adapter",
-          "prediction": "mountains",
-          "top_retrieval": [
-            "Evan: [27 August 2023] Evan also shared his recent road trip to the Rocky Mountains and love for hiking.",
-            "Evan: [27 August 2023] Evan lives within a two-hour drive of a place with incredible views and a peaceful atmosphere.",
-            "Evan: [9 November 2023] They also discussed enjoying a sunset together at Evan's favorite spot by the beach, planning to visit it soon to de-stress.",
-            "Evan: [10 January 2024] Evan enjoys going on beach sunsets as a low-impact exercise.",
-            "[7:11 pm on 24 May, 2023] Evan: Hey Sam, thanks for asking! It was great - fresh air, peacefulness and a cozy cabin surrounded by mountains and forests made it feel like a real retreat.",
-            "Evan: [31 December 2023] Sam shared about a recent hiking trip, while Evan mentioned a mountain drive that ended in a minor accident.",
-            "Evan: [27 August 2023] Evan recommended a nearby lake for hiking and nature exploration.",
-            "Evan: [27 August 2023] Evan enjoys road trips and exploring nature."
-          ]
-        }
-      ]
-    },
-    {
-      "id": "reading_detail",
-      "title": "Reading Detail",
-      "source_id": "conv-49",
-      "category": 4,
-      "question": "What novel is Evan reading that he finds gripping?",
-      "gold_answer": "The Great Gatsby",
-      "kind": "failure",
-      "note": "A representative multi-hop miss: the model retains the coarse book description but misses the specific title.",
-      "systems": [
-        {
-          "name": "release_model",
-          "display_name": "PRISM-Memory 7B Adapter",
-          "prediction": "a new mystery novel",
-          "top_retrieval": [
-            "Evan: [27 August 2023] Evan is reading a book that he finds increasingly compelling.",
-            "Evan: [27 July 2023] Evan is currently reading a new mystery novel.",
-            "Evan: [27 July 2023] Evan is reading 'The Great Gatsby'.",
-            "Evan: [26 December 2023] Evan finds that art helps him recognize and handle his own feelings.",
-            "Evan: [27 August 2023] Evan expressed interest in a book and discussed potential physical therapy for his knee.",
-            "Evan: [10 January 2024] Evan concluded that he needs to be more careful next time.",
-            "Evan: [6 October 2023] Evan thinks writing is a great way to express oneself.",
-            "Evan: [13 August 2023] Evan suggested checking out a dream interpretation book to help interpret Sam's dream.",
-            "Evan: [6 October 2023] Evan believes that writing can be super therapeutic.",
-            "Evan: [6 October 2023] Evan usually paints what is on his mind or something he is feeling."
-          ]
-        }
-      ]
-    }
-  ]
-}

results/try_it_sessions.json

@@ -0,0 +1,194 @@
+{
+  "model_name": "PRISM-Memory 7B Adapter",
+  "base_model": "Qwen/Qwen2.5-7B-Instruct",
+  "examples": [
+    {
+      "id": "ci_rollout",
+      "title": "CI rollout and platform constraints",
+      "note": "Numeric limits, alert preferences, and rollout state stay compact and queryable.",
+      "transcript": "[2025-03-01] Dana: We have 20 concurrent jobs max on GitHub Actions right now. Keep Snyk Slack notifications aggregated and concise, not one alert per repo.\n[2025-03-02] Ravi: Let's start with incremental scans and matrix jobs first, then widen coverage after the baseline stabilizes.\n[2025-03-07] Dana: No mTLS yet. Put it in phase two after the canary rollout.",
+      "later_question": "What is the current concurrency limit, how should Slack alerts behave, and when is mTLS planned?",
+      "answer_from_memory": "The current GitHub Actions concurrency limit is 20 jobs. Slack notifications should be aggregated and concise. mTLS is planned for phase two after the canary rollout.",
+      "source_mode": "released_model_precomputed",
+      "turns": [
+        {
+          "turn_index": 1,
+          "date": "2025-03-01",
+          "speaker": "Dana",
+          "text": "We have 20 concurrent jobs max on GitHub Actions right now. Keep Snyk Slack notifications aggregated and concise, not one alert per repo.",
+          "prism_memory": [
+            "GitHub Actions concurrency limit: 20 concurrent jobs",
+            "Snyk Slack notifications aggregated and concise; no separate alerts per repo"
+          ],
+          "memory_store_after_turn": [
+            "GitHub Actions concurrency limit: 20 concurrent jobs",
+            "Snyk Slack notifications aggregated and concise; no separate alerts per repo"
+          ]
+        },
+        {
+          "turn_index": 2,
+          "date": "2025-03-02",
+          "speaker": "Ravi",
+          "text": "Let's start with incremental scans and matrix jobs first, then widen coverage after the baseline stabilizes.",
+          "prism_memory": [
+            "Plan: start with incremental scans and matrix jobs; widen coverage after baseline stabilizes"
+          ],
+          "memory_store_after_turn": [
+            "GitHub Actions concurrency limit: 20 concurrent jobs",
+            "Snyk Slack notifications aggregated and concise; no separate alerts per repo",
+            "Plan: start with incremental scans and matrix jobs; widen coverage after baseline stabilizes"
+          ]
+        },
+        {
+          "turn_index": 3,
+          "date": "2025-03-07",
+          "speaker": "Dana",
+          "text": "No mTLS yet. Put it in phase two after the canary rollout.",
+          "prism_memory": [
+            "No mutual TLS enabled; mutual TLS will be implemented in phase two after the canary rollout."
+          ],
+          "memory_store_after_turn": [
+            "GitHub Actions concurrency limit: 20 concurrent jobs",
+            "Snyk Slack notifications aggregated and concise; no separate alerts per repo",
+            "Plan: start with incremental scans and matrix jobs; widen coverage after baseline stabilizes",
+            "No mutual TLS enabled; mutual TLS will be implemented in phase two after the canary rollout."
+          ]
+        }
+      ],
+      "final_memory": [
+        "GitHub Actions concurrency limit: 20 concurrent jobs",
+        "Snyk Slack notifications aggregated and concise; no separate alerts per repo",
+        "Plan: start with incremental scans and matrix jobs; widen coverage after baseline stabilizes",
+        "No mutual TLS enabled; mutual TLS will be implemented in phase two after the canary rollout."
+      ]
+    },
+    {
+      "id": "travel_replan",
+      "title": "Trip updates and hard constraints",
+      "note": "Updated plans should replace stale ones while durable constraints remain.",
+      "transcript": "[2025-06-12] Maya: We booked the Lisbon trip for September 14. I want a quiet hotel near Alfama, and no red-eye flights.\n[2025-06-18] Omar: The outbound should leave after 9am because early flights wreck me.\n[2025-07-02] Maya: Update the plan: Lisbon is off. We are going to Porto on September 21 instead, still no red-eye flights.",
+      "later_question": "Where is the trip now, on what date, and what flight constraint still applies?",
+      "answer_from_memory": "The current trip is to Porto on September 21, 2025, and red-eye flights are still off limits.",
+      "source_mode": "released_model_precomputed",
+      "turns": [
+        {
+          "turn_index": 1,
+          "date": "2025-06-12",
+          "speaker": "Maya",
+          "text": "We booked the Lisbon trip for September 14. I want a quiet hotel near Alfama, and no red-eye flights.",
+          "prism_memory": [
+            "Lisbon trip booked for September 14",
+            "Prefers a quiet hotel near Alfama",
+            "Prefers no red-eye flights"
+          ],
+          "memory_store_after_turn": [
+            "Lisbon trip booked for September 14",
+            "Prefers a quiet hotel near Alfama",
+            "Prefers no red-eye flights"
+          ]
+        },
+        {
+          "turn_index": 2,
+          "date": "2025-06-18",
+          "speaker": "Omar",
+          "text": "The outbound should leave after 9am because early flights wreck me.",
+          "prism_memory": [
+            "Outbound flight should depart after 9am."
+          ],
+          "memory_store_after_turn": [
+            "Lisbon trip booked for September 14",
+            "Prefers a quiet hotel near Alfama",
+            "Prefers no red-eye flights",
+            "Outbound flight should depart after 9am."
+          ]
+        },
+        {
+          "turn_index": 3,
+          "date": "2025-07-02",
+          "speaker": "Maya",
+          "text": "Update the plan: Lisbon is off. We are going to Porto on September 21 instead, still no red-eye flights.",
+          "prism_memory": [
+            "Lisbon trip canceled; Porto trip scheduled for September 21.",
+            "No red-eye flights for Porto trip."
+          ],
+          "memory_store_after_turn": [
+            "Lisbon trip booked for September 14",
+            "Prefers a quiet hotel near Alfama",
+            "Prefers no red-eye flights",
+            "Outbound flight should depart after 9am.",
+            "Lisbon trip canceled; Porto trip scheduled for September 21.",
+            "No red-eye flights for Porto trip."
+          ]
+        }
+      ],
+      "final_memory": [
+        "Lisbon trip booked for September 14",
+        "Prefers a quiet hotel near Alfama",
+        "Prefers no red-eye flights",
+        "Outbound flight should depart after 9am.",
+        "Lisbon trip canceled; Porto trip scheduled for September 21.",
+        "No red-eye flights for Porto trip."
+      ]
+    },
+    {
+      "id": "health_hobby",
+      "title": "Health constraints and new routines",
+      "note": "Personal constraints, routines, and hobbies should stay available for later coaching or reminders.",
+      "transcript": "[2025-08-03] Sam: My doctor wants me to keep sodium under 2 grams a day. I started painting on weekends because it helps me decompress.\n[2025-08-17] Sam: I bought watercolors and signed up for a Saturday class downtown.\n[2025-09-01] Sam: Skip late-night coffee from now on; it wrecks my sleep.",
+      "later_question": "What hobby did Sam start, what class did he sign up for, and what health constraints matter now?",
+      "answer_from_memory": "Sam started painting, signed up for a Saturday watercolor class downtown, should keep sodium under 2 grams per day, and wants to avoid late-night coffee.",
+      "source_mode": "released_model_precomputed",
+      "turns": [
+        {
+          "turn_index": 1,
+          "date": "2025-08-03",
+          "speaker": "Sam",
+          "text": "My doctor wants me to keep sodium under 2 grams a day. I started painting on weekends because it helps me decompress.",
+          "prism_memory": [
+            "Doctor recommended Sam limit sodium intake to 2 grams per day.",
+            "Sam paints on weekends as a decompression activity."
+          ],
+          "memory_store_after_turn": [
+            "Doctor recommended Sam limit sodium intake to 2 grams per day.",
+            "Sam paints on weekends as a decompression activity."
+          ]
+        },
+        {
+          "turn_index": 2,
+          "date": "2025-08-17",
+          "speaker": "Sam",
+          "text": "I bought watercolors and signed up for a Saturday class downtown.",
+          "prism_memory": [
+            "Sam bought watercolors and signed up for a Saturday painting class downtown."
+          ],
+          "memory_store_after_turn": [
+            "Doctor recommended Sam limit sodium intake to 2 grams per day.",
+            "Sam paints on weekends as a decompression activity.",
+            "Sam bought watercolors and signed up for a Saturday painting class downtown."
+          ]
+        },
+        {
+          "turn_index": 3,
+          "date": "2025-09-01",
+          "speaker": "Sam",
+          "text": "Skip late-night coffee from now on; it wrecks my sleep.",
+          "prism_memory": [
+            "Sam decided to skip late-night coffee starting now."
+          ],
+          "memory_store_after_turn": [
+            "Doctor recommended Sam limit sodium intake to 2 grams per day.",
+            "Sam paints on weekends as a decompression activity.",
+            "Sam bought watercolors and signed up for a Saturday painting class downtown.",
+            "Sam decided to skip late-night coffee starting now."
+          ]
+        }
+      ],
+      "final_memory": [
+        "Doctor recommended Sam limit sodium intake to 2 grams per day.",
+        "Sam paints on weekends as a decompression activity.",
+        "Sam bought watercolors and signed up for a Saturday painting class downtown.",
+        "Sam decided to skip late-night coffee starting now."
+      ]
+    }
+  ]
+}