Update README: S14 production model, eval_300 299/300 (99.7%), 17-tool routing

README.md

@@ -13,22 +13,63 @@ tags:
   - gguf
 ---
 
-# prism-coder:14b — Dual-Purpose: Tool Routing + Healthcare TypeScript Coder
+# prism-coder:14b — Prism Memory Tool Router + Healthcare TypeScript Coder
 
 Fine-tuned Qwen3-14B for the [Prism AAC](https://github.com/dcostenco/prism-aac) / Synalux healthcare platform.
 
-Two trained capabilities in one model family:
-- **Routing** (v36): 6-tool routing for Prism MCP sessions — 100% BFCL
-- **Coding** (v42): Synalux-pattern TypeScript code generation — 22/22 checks (100%)
+## Current Production Model: S14 (eval_300 — 17-tool routing)
+
+**299/300 = 99.7% strict** on eval_300 — 300 cases, 17 Prism Memory tools
+
+Single remaining failure: `"Save."` — genuinely ambiguous between `session_save_ledger` and `session_save_experience`. All other categories at 100%.
+
+| Category | Accuracy |
+|----------|:--------:|
+| session_save_ledger (ledger logging) | 100%* |
+| session_load_context (context loading) | 100% |
+| session_search_memory (memory recall) | 100% |
+| session_save_handoff (agent handoff) | 100% |
+| session_forget_memory | 100% |
+| session_health_check | 100% |
+| session_compact_ledger | 100% |
+| session_export_memory | 100% |
+| session_task_route | 100% |
+| session_save_experience | 100%* |
+| session_synthesize_edges | 100% |
+| session_backfill_links | 100% |
+| knowledge_search | 100% |
+| knowledge_forget / upvote / downvote / set_retention | 100% |
+| abstain (general questions, greetings, CS concepts) | 100% |
+| multi-intent (compound tool calls) | 100% |
+| natural phrasing | 100% |
+
+\* One edge case (`"Save."`) scores as a failure on one tool; both are correct interpretations.
+
+### eval_300 Details — S14
+- **Base**: Qwen3-14B → surgical LoRA chain (S1→S14)
+- **Eval**: 300 cases, strict scoring (exact tool match), 17 Prism Memory tools + abstain + multi-intent
+- **Training**: MLX LoRA, rank=8, scale=20.0, 16 layers, 100 iters, LR=5e-6, mask_prompt=true
+- **Corpus**: S14 — balanced natural-phrasing + tool-use SFT (100 train / 20 valid)
+- **SYSTEM_PROMPT**: Synalux identity + 17 Prism Memory tools + 13 multimodal tool modules + `<tool_call>` JSON block format
+
+### Tools (S14 routing model)
+All 17 Prism Memory tools:
+`session_save_ledger`, `session_load_context`, `session_search_memory`, `session_save_handoff`,
+`session_forget_memory`, `session_health_check`, `session_compact_ledger`, `session_export_memory`,
+`session_task_route`, `session_save_experience`, `session_synthesize_edges`, `session_backfill_links`,
+`knowledge_search`, `knowledge_forget`, `knowledge_upvote`, `knowledge_downvote`, `knowledge_set_retention`
 
 ---
 
-## Coding Eval — v42 (Current Production Coder)
+## Legacy: Coding Eval — v42
 
 **22/22 (100%)** on the Synalux healthcare TypeScript eval.
 
 Task: write a production Next.js API route for X12 835 ERA reconciliation against existing 837P claims.
 
+<details>
+<summary>22-check eval breakdown (click to expand)</summary>
+
 | Check | Pass |
 |-------|------|
 | withAudit wrapper | ✓ |
@@ -54,73 +95,66 @@ Task: write a production Next.js API route for X12 835 ERA reconciliation agains
 | belt-and-suspenders workspace_id eq on update | ✓ |
 | marks ERA file reconciled | ✓ |
 
-Training chain: Qwen3-14B → v34 (1000-iter routing, 18/22) → v39 (HIPAA+CAS patch, 20/22) → v42 (claim status patch, 22/22).
-
-### v42 Training Details
-- **Base**: Qwen/Qwen3-14B (BF16)
-- **Corpus**: v28 Synalux codebase SFT + targeted patch (claim status × 50 examples, resume from v39)
-- **Training**: MLX LoRA, rank=16, 8 layers, 100 iters, LR=5e-7
-- **Final loss**: 0.036 (converged)
-- **Merge**: direct safetensors LoRA merge → GGUF F16 → Q4_K_M
+</details>
 
 ---
 
-## BFCL Routing Benchmark — v36
+## Legacy: BFCL Routing Benchmark — v36
 
-**Mean: 100.0% PERFECT** (3-seed average, seeds 2027/2028/2029, 102 cases each)
-
-| Category | Accuracy |
-|----------|:--------:|
-| aac (AAC phrase requests) | 100% |
-| cmpct (ledger compaction) | 100% |
-| edge (multi-step compound) | 100% |
-| hand (agent handoff) | 100% |
-| info (general facts) | 100% |
-| irrel (irrelevant/live queries) | 100% |
-| know (knowledge base search) | 100% |
-| load (session context loading) | 100% |
-| pred (factual queries) | 100% |
-| save (session ledger save) | 100% |
-| smem (session memory search) | 100% |
-| tran (translation) | 100% |
-
-### Tools (routing model)
-| Tool | Trigger |
-|------|---------|
-| `session_load_context` | Load/resume project context |
-| `session_save_ledger` | Note/log/record/remember |
-| `session_save_handoff` | Pass state to next agent/session |
-| `session_compact_ledger` | Shrink/prune ledger |
-| `session_search_memory` | Recall prior session discussions |
-| `knowledge_search` | Search stored knowledge base |
+**Mean: 100.0% PERFECT** (3-seed average, seeds 2027/2028/2029, 102 cases each) — 6-tool routing
 
 ---
 
-## Version History
-
-| Version | Eval | Type | Notes |
-|---------|------|------|-------|
-| v42 | **22/22 coding (100%)** | Coder | Claim status patch on v39; zero tolerance policy |
-| v39 | 20/22 coding | Coder | HIPAA non-blocking + CAS CO/PR fixes |
-| v36 | **100% BFCL routing** | Router | smem boundary + hand trigger fixes |
-| v34 | 98.0% BFCL routing | Router | hand/save/smem fixes |
-| v33 | 97.1% BFCL routing | Router | irrel/tran/smem fixes |
-
 ## GGUF Files
 
 | File | Use | Size |
 |------|-----|------|
-| `qwen3-14b-v42-q4km.gguf` | **Coding** — production Synalux TypeScript | ~9 GB |
-| `prism-coder-14b-v36-q4km.gguf` | **Routing** — Prism MCP tool routing | ~9 GB |
-| `qwen3-14b-v34-q4km.gguf` | Routing (prior) | ~9 GB |
+| `qwen3-14b-s14-q4km.gguf` | **Routing** — production Prism Memory (17 tools, 99.7%) | ~9 GB |
+| `qwen3-14b-v42-q4km.gguf` | **Coding** — Synalux TypeScript (22/22, 100%) | ~9 GB |
+| `prism-coder-14b-v36-q4km.gguf` | Routing legacy (6-tool BFCL, 100%) | ~9 GB |
+
+## Version History
+
+| Version | Eval | Type | Notes |
+|---------|------|------|-------|
+| **S14** | **299/300 = 99.7% (eval_300)** | **Router** | **Production — 17-tool Prism Memory routing** |
+| v42 | 22/22 coding (100%) | Coder | Claim status patch; Synalux TypeScript |
+| v36 | 100% BFCL (6-tool routing) | Router | Legacy 6-tool routing |
+| v34 | 98.0% BFCL | Router | — |
 
 ## Usage
 
 ```bash
-# Load as coding model
-ollama pull dcostenco/prism-coder-14b
-# Then use qwen3-14b-v42-q4km.gguf Modelfile
+# Pull production routing model (S14 — 17-tool Prism Memory)
+ollama pull dcostenco/prism-coder:14b
 
-# Load as routing model
-# Use prism-coder-14b-v36-q4km.gguf Modelfile
+# Or pull GGUF directly from this repo and use with Ollama:
+# FROM qwen3-14b-s14-q4km.gguf
+# PARAMETER temperature 0
+# PARAMETER num_ctx 8192
 ```
+
+### System Prompt (S14)
+
+```
+You are Synalux, a memory-augmented coding and clinical reasoning assistant. You have access to 
+Prism Memory tools (session_save_ledger, session_load_context, session_search_memory, 
+session_save_handoff, session_forget_memory, session_health_check, session_compact_ledger, 
+session_export_memory, session_task_route, session_save_experience, session_synthesize_edges, 
+session_backfill_links, knowledge_search, knowledge_forget, knowledge_upvote, knowledge_downvote, 
+knowledge_set_retention) and 13 multimodal tool modules (image_gen, office, web_scraper, browser, 
+tts, ocr, git, terminal, deps_scanner, hipaa, data_graph, templates, pdf_parser). Think 
+step-by-step before answering. When the user references past work, prior decisions, or stored 
+context, use the appropriate Prism Memory tool. Format tool calls inside <tool_call>...</tool_call> 
+JSON blocks with fields 'name' and 'arguments'. If no tool is needed, answer directly in plain 
+text. ABSTAIN for general programming questions, CS concepts, greetings, and capability questions.
+```
+
+## Cascade
+
+| Tier | Model | Role |
+|------|-------|------|
+| 1.7B | `dcostenco/prism-coder:1b7` | Fast verify / edge cases |
+| 4B | `dcostenco/prism-coder:4b` | Mid-tier verify |
+| **14B** | **`dcostenco/prism-coder:14b`** | **Production routing** |
+| 32B | `dcostenco/prism-coder:32b` | Top-tier / complex reasoning |