Upload Concept Memory base model

README.md

@@ -18,71 +18,94 @@ criteria-conditioned Choice, ordinal Score, and Yes/No (Noul) judgments. It is
 not a language model pretrained from scratch. Its text encoder is
 `sentence-transformers/all-mpnet-base-v2`.
 
-## What the model does
-
-Concept Memory is a compact model for making typed judgments about a piece of
-text or other supplied state. You provide the state to assess, an instruction
-describing the judgment, and (for Choice and Score) the criteria to consider.
-The model encodes these inputs and returns a prediction with probabilities.
-It does not generate free-form answers.
-
-The three supported judgment types are:
-
-- **Choice:** select the best matching criterion, such as routing a support
-  request to a team.
-- **Score:** estimate an ordinal position among the supplied criteria, from
-  lowest to highest.
-- **Noul (Yes/No):** return a binary judgment for the instruction.
-
-## Concept Memory and the Concept Bank
-
-Jev- or Laya-style typed decision systems return a structured judgment, such as
-a Choice label and its probabilities. Concept Memory can make that same kind of
-prediction and optionally add a **Concept Bank**: a separate, editable set of
-human-readable concepts with links to outcomes. For example, “integration
-failure” may support “Technical,” while “payment issue” may support “Billing.”
-This externalizes some domain knowledge from the model's learned weights so it
-can be inspected, reused, and adapted to different tasks or domains.
-
-With a bank, inference has two paths: the base model predicts an outcome, and a
-second Choice question ranks which bank concepts are relevant to the input.
-The bank maps those relevant concepts to outcome votes. In the current helper,
-the bank adjusts the probabilities only when its leading outcome agrees with
-the base model's leading outcome; the adjusted distribution is a blend of 20%
-raw model probabilities and 80% normalized bank votes. The bank therefore
-provides constrained decision support rather than an independent rule engine.
-
-Concept rankings and declared concept-to-outcome links make part of the
-decision process inspectable. They are not proof of the model's internal
-causal reasoning: relevance is a relative ranking, not calibrated evidence,
-and a ranked concept is not necessarily what caused the raw prediction.
-
-## What this does and does not establish
-
-The Concept Bank offers a way to change or extend domain mappings without
-retraining the encoder, but new concepts still need evaluation on representative
-examples. This is an architectural capability, not evidence that Concept
-Memory is more accurate than Jev, Laya, or another model. Accuracy, calibration,
-robustness, and latency need head-to-head evaluation on the same data and
-conditions before making comparative performance claims.
-
-The current implementation ranks concepts with a second Choice question and
-uses an agreement gate. It does not independently detect each concept or let a
-bank vote overturn a raw prediction. Concept hierarchies, dependencies,
-contradictions, interventions, and provenance tracking are possible research
-directions, not capabilities of this checkpoint.
+## What is the problem?
 
-## Files
+Typed decision models can return structured outputs such as a Choice label and
+its probabilities. In a domain workflow, teams may also need to understand
+which reusable ideas are relevant, apply domain knowledge consistently, and
+adapt those mappings as policies change. A prediction by itself does not
+provide those capabilities, and a probability is not an explanation.
 
-- `model.pt`: PyTorch state dictionary and training metadata.
-- `config.json`: encoder, task, and training run metadata.
+Concept Memory is designed to explore this gap. It supports three judgment
+types:
+
+- **Choice:** select one criterion, such as a support team.
+- **Score:** estimate an ordinal position among supplied criteria.
+- **Noul (Yes/No):** return a binary judgment.
+
+## What is the solution?
+
+### Theory
+
+Concept Memory pairs a learned language representation with an optional,
+editable Concept Bank. A concept describes a reusable pattern, such as
+“integration failure,” and can link that pattern to outcomes such as
+“Technical.” This separates some domain knowledge from the model's learned
+weights, making the definitions and mappings easier to inspect, reuse, and
+adapt across tasks.
+
+This is a different design emphasis from a decision-only interface: the model
+still returns a typed prediction, while the bank offers an additional path for
+relating relevant concepts to outcomes. This is a potential architectural
+benefit, not evidence of higher accuracy than Jev, Laya, or another model.
+
+### Technical design
+
+The base model first predicts from the input state, question instructions, and
+criteria. When a Concept Bank is supplied, a second Choice question ranks the
+bank's concepts by relevance. The bank maps those concept signals to outcome
+votes. The current helper adjusts probabilities only when the bank's top
+outcome agrees with the base model's top outcome; when it does, it blends 20%
+raw model probabilities with 80% normalized bank votes. The bank is therefore
+constrained decision support, not an independent rule engine, and it cannot
+overturn the raw model's winner.
+
+Concept rankings and declared mappings make part of the decision process
+inspectable. They are not causal explanations: relevance is a relative ranking,
+not calibrated evidence, and it does not prove what caused the raw prediction.
+The current implementation also does not independently detect each concept.
+Concept hierarchies, dependencies, interventions, and provenance tracking are
+future research directions, not capabilities of this checkpoint.
+
+This architecture allows domain mappings to be changed without retraining the
+encoder, but new concepts must still be evaluated on representative examples.
+Accuracy, calibration, robustness, and latency should be compared on the same
+data and conditions before making performance claims.
+
+## How can you use it?
+
+### Run the standalone Choice example
 
-The Python inference implementation is not distributed in this model
-repository. The checkpoint is a PyTorch artifact and is not directly
-compatible with `transformers.AutoModel.from_pretrained`. For interactive
-inference, use the [Concept Memory playground](https://dewmal--concept-memory-playground-playground.modal.run/).
+The local [`artifacts/base_model/example.py`](./example.py) script:
 
-## Example Concept Bank entry
+1. Downloads the `Syigen/rgcm` repository snapshot from Hugging Face.
+2. Loads `model.pt` and constructs the model using its saved encoder settings.
+3. Creates a Choice question with Billing, Technical, and Sales criteria.
+4. Predicts a category for a sample Stripe integration problem, then prints
+   the chosen criterion and probabilities.
+
+From the project root, install the dependencies and run it:
+
+```bash
+pip install torch transformers huggingface_hub
+python artifacts/base_model/example.py
+```
+
+The example demonstrates Choice. Score and Noul questions use the same typed
+question interface with the appropriate `type` and criteria. The checkpoint is
+a PyTorch artifact for this project's model class; it is not directly loadable
+with `transformers.AutoModel.from_pretrained`.
+
+### Try the linked examples
+
+- [Hugging Face `example.py`](https://huggingface.co/Syigen/rgcm/blob/main/example.py) is the hosted Python example file.
+- [Open the Colab notebook](https://colab.research.google.com/drive/1LMLbdff8rAVxPy-qwcVI6iFROkjUk8EK?usp=sharing) to try the linked notebook.
+- [Open the Concept Memory playground](https://dewmal--concept-memory-playground-playground.modal.run/) for interactive inference.
+
+### Add concepts
+
+Concept Bank entries define a readable concept and the outcomes it supports or
+contradicts. For example:
 
 ```json
 {
@@ -94,9 +117,14 @@ inference, use the [Concept Memory playground](https://dewmal--concept-memory-pl
 }
 ```
 
-Concept entries describe reusable patterns and link them to outcome IDs or
-labels. Add entries to the playground's Concept Bank field to inspect which
-concepts rank as relevant and how their links contribute to a prediction.
+Add concepts to the playground's Concept Bank field to see which ones rank as
+relevant and how their links contribute to the decision.
+
+## Files
+
+- `model.pt`: PyTorch state dictionary and training metadata.
+- `config.json`: encoder, task, and training run metadata.
+- `example.py`: standalone Python Choice example.
 
 ## Training metadata