| --- |
| license: other |
| license_name: ameforge-proprietary |
| license_link: LICENSE |
| language: |
| - en |
| pipeline_tag: question-answering |
| datasets: |
| - rajpurkar/squad |
| tags: |
| - continual-learning |
| - memory |
| - retrieval |
| - research-prototype |
| --- |
| |
| # Ananke v1 |
|
|
| **A memory that learns by reading, with a learned reader.** Research prototype by **AMEFORGE** |
| (Ontario, Canada). |
|
|
| 👉 **Try it:** [Ananke demo Space](https://huggingface.co/spaces/ameforge/ananke) |
|
|
| ## What changed since v0 |
|
|
| [Ananke v0](https://huggingface.co/AMFORGE/ananke-v0) found the right passages in its memory well, |
| but picked the short answer inside them with hand-written rules — its main weakness. |
|
|
| In v1: |
|
|
| - **The memory is unchanged.** It still reads text once, keeps learning, cites its sources and |
| refuses on subjects it never read. |
| - **The short answer is chosen by a small learned reader**, a proprietary AMEFORGE architecture |
| (not a pretrained language model). It looks only at the passages the memory retrieved. |
| - The reader was trained once, on a GPU, on SQuAD v1.1 questions. It runs on a CPU. |
| Articles used for the demo memory were excluded from its training data. |
|
|
| ## Evaluation (honest numbers) |
|
|
| All question sets were written and frozen **before** each evaluation and never used for training. |
| The final set (41 questions) was evaluated a single time. |
|
|
| | | Ananke v0 (rules) | Ananke v1 (learned reader) | |
| |---|---|---| |
| | Final frozen set (41 questions) | 14 | **16** | |
| | All frozen sets so far (323 questions) | 109 (34 %) | **119 (37 %)** | |
| | Refusal on never-read subjects (3) | 3 / 3 | 3 / 3 | |
|
|
| What this means: |
|
|
| - v1 is **slightly better** than v0, but the difference is **not statistically significant yet** |
| (sign test, p ≈ 0.34). Each system answers some questions the other misses. |
| - The exact short answer is still wrong about **two times out of three**: the quoted passages |
| matter more than the headline answer. |
| - When the memory retrieves the wrong passage, the reader cannot recover. |
|
|
| ## Limitations |
|
|
| - **Not a chatbot or an LLM.** It does not reason, combine facts or write free text. |
| - It **believes what it reads**: wrong text in, wrong answer out. |
| - English works best. Lists ("which countries border…") are often incomplete. |
| - If a word of the question is known for an unrelated reason, it may answer from an unrelated |
| passage instead of refusing. **Always read the quotes.** |
|
|
| ## Intended use |
|
|
| Research and demonstration of learning by reading. A self-updating, citable memory over a document |
| collection. Not for medical, legal, financial or safety-critical decisions. |
|
|
| ## Weights and code |
|
|
| The engine, the reader and the memory are proprietary to AMEFORGE and are not distributed; |
| the Space provides public access for testing. |
|
|
| ## Data and attribution |
|
|
| - Reader training: [SQuAD v1.1](https://rajpurkar.github.io/SQuAD-explorer/) (Rajpurkar et al., 2016), |
| CC BY-SA 4.0. |
| - Demo memory: English Wikipedia articles, CC BY-SA 4.0. Quoted passages are shown with their source. |
|
|
| ## Contact |
|
|
| AMEFORGE — Mike Amega · [huggingface.co/AMFORGE](https://huggingface.co/AMFORGE) |
|
|
