Upload folder using huggingface_hub

README.md

@@ -0,0 +1,90 @@
+# 🧠 MemoryBERT
+
+A RoBERTa-based transformer model for **Cognitive Memory Recognition (CMR)** – classifying natural language into six memory categories inspired by cognitive science.
+
+---
+
+## 🧭 Overview
+
+MemoryBERT is fine-tuned to classify user-generated text into:
+- **Episodic memory**
+- **Semantic memory**
+- **Spatial memory**
+- **Emotional memory**
+- **Associative memory**
+- **Non-memory**
+
+This model supports research into memory-type classification, schema formation, and personalized AI interaction systems.
+
+---
+
+## 🧪 Model Details
+
+- **Base model**: `roberta-base`
+- **Task**: Multi-class sequence classification
+- **Classes**: 6
+- **Max sequence length**: 128 tokens
+- **Training epochs**: 1.5
+- **Label smoothing**: 0.1
+- **Loss function**: CrossEntropyLoss
+- **Optimizer**: AdamW
+- **Batch size**: 8
+
+---
+
+## 📊 Evaluation Results
+
+On a synthetic 400-example test set balanced across classes:
+
+| Class         | Precision | Recall | F1-score | Support |
+|---------------|-----------|--------|----------|---------|
+| Associative   | 1.00      | 1.00   | 1.00     | 39      |
+| Emotional     | 1.00      | 1.00   | 1.00     | 40      |
+| Episodic      | 1.00      | 1.00   | 1.00     | 39      |
+| Non-memory    | 1.00      | 1.00   | 1.00     | 200     |
+| Semantic      | 1.00      | 1.00   | 1.00     | 40      |
+| Spatial       | 1.00      | 1.00   | 1.00     | 42      |
+
+- **Macro F1**: 1.00  
+- **Eval loss**: 0.423  
+- **Epochs**: 1.5  
+- **Accuracy**: 100%
+
+> ⚠️ Note: These results are from a synthetic dataset — further real-world validation is ongoing and expansion of baseline dataset used for version 1 of memoryBERT
+
+---
+
+## 🧠 Dataset
+
+MemoryBERT was trained on a synthetic dataset of 4,000 curated examples (2,000 memory and 2,000 non-memory)
+
+Each entry is labeled with one of six memory types and tagged by domain and span group.
+
+---
+
+## 🚀 Usage
+
+```python
+from transformers import RobertaTokenizer, RobertaForSequenceClassification
+
+model = RobertaForSequenceClassification.from_pretrained("DimitriosPanagoulias/MemoryBERT")
+tokenizer = RobertaTokenizer.from_pretrained("DimitriosPanagoulias/MemoryBERT")
+
+def predict_memory_type(text):
+    inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=128)
+    outputs = model(**inputs)
+    predicted_id = outputs.logits.argmax(dim=-1).item()
+    return model.config.id2label[predicted_id]
+
+predict_memory_type("Without a map, I navigated the winding back roads to reach my childhood home.")
+```
+
+## Citation
+
+You can cite either one or both of the following previous related work: 
+
+- Panagoulias, D.P. et al. “Memory and Schema in Human–Generative Artificial Intelligence Interactions.”
+2024 IEEE ICTAI Conference (in press)
+
+- Panagoulias, D.P. et al. Mathematical representation of memory and schema for improving human-generative AI interactions.”
+2024 IEEE IISA Conference (in press)

config.json

@@ -0,0 +1,43 @@
+{
+  "_name_or_path": "roberta-base",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "associative",
+    "1": "emotional",
+    "2": "episodic",
+    "3": "non_memory",
+    "4": "semantic",
+    "5": "spatial"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "associative": 0,
+    "emotional": 1,
+    "episodic": 2,
+    "non_memory": 3,
+    "semantic": 4,
+    "spatial": 5
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.40.1",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 50265
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c30a3553436a20192f71a397eb36c69c96da5dee281cfa84076b494284821d62
+size 498625128

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,56 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50264": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "mask_token": "<mask>",
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "RobertaTokenizer",
+  "unk_token": "<unk>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f6d562bc8eecf58632b0f7f99ba17ab5c0f68ac67bcff8fdbaba3d8d20350ea
+size 5393