---
pipeline_tag: text-classification
tags:
- memory
- text-classification
- roberta
- cognitive-nlp
- noetiv
license: mit
library_name: transformers
language:
- en
metrics:
- accuracy
---

### 🧠 About NOETIV

This project is part of the **NOETIV** initiative — a modular AI platform for healthcare proffesionals.  
🔗 Visit us at [noetiv.com](https://www.noetiv.com)

# 🧠 MemoryBERT

A RoBERTa-based transformer model for **Cognitive Memory Recognition (CMR)** – classifying natural language into six memory categories inspired by cognitive science.

---

## 🧭 Overview

MemoryBERT is fine-tuned to classify user-generated text into:
- **Episodic memory**
- **Semantic memory**
- **Spatial memory**
- **Emotional memory**
- **Associative memory**
- **Non-memory**

This model supports research into memory-type classification, schema formation, and personalized AI interaction systems.

## 🧪 Model Details

- **Base model**: `roberta-base`
- **Task**: Multi-class sequence classification
- **Classes**: 6
- **Max sequence length**: 128 tokens
- **Training epochs**: 1.5
- **Label smoothing**: 0.1
- **Loss function**: CrossEntropyLoss
- **Optimizer**: AdamW
- **Batch size**: 8

---

## 📊 Evaluation Results

On a synthetic 400-example test set balanced across classes:

| Class         | Precision | Recall | F1-score | Support |
|---------------|-----------|--------|----------|---------|
| Associative   | 1.00      | 1.00   | 1.00     | 39      |
| Emotional     | 1.00      | 1.00   | 1.00     | 40      |
| Episodic      | 1.00      | 1.00   | 1.00     | 39      |
| Non-memory    | 1.00      | 1.00   | 1.00     | 200     |
| Semantic      | 1.00      | 1.00   | 1.00     | 40      |
| Spatial       | 1.00      | 1.00   | 1.00     | 42      |

- **Macro F1**: 1.00  
- **Eval loss**: 0.423  
- **Epochs**: 1.5  
- **Accuracy**: 100%

> ⚠️ Note: These results are from a synthetic dataset — further real-world validation is ongoing and expansion of baseline dataset used for version 1 of memoryBERT

---

## 🧠 Dataset

MemoryBERT was trained on a synthetic dataset of 4,000 curated examples (2,000 memory and 2,000 non-memory)

Each entry is labeled with one of six memory types and tagged by domain and span group.

---

## 🚀 Usage

```python
from transformers import RobertaTokenizer, RobertaForSequenceClassification

model = RobertaForSequenceClassification.from_pretrained("DimitriosPanagoulias/MemoryBERT")
tokenizer = RobertaTokenizer.from_pretrained("DimitriosPanagoulias/MemoryBERT")

def predict_memory_type(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=128)
    outputs = model(**inputs)
    predicted_id = outputs.logits.argmax(dim=-1).item()
    return model.config.id2label[predicted_id]

predict_memory_type("Without a map, I navigated the winding back roads to reach my childhood home.")
```
or via huggingface pipeline
```python
# Use a pipeline as a high-level helper
from transformers import pipeline
import torch
device = 0 if torch.cuda.is_available() else -1  # 0 = GPU, -1 = CPU
pipe = pipeline("text-classification", model="DimitriosPanagoulias/MemoryBERT", device=device)
pipe("I remember the long walk to my childhood school.")
```
outputs:
```bash
[{'label': 'episodic', 'score': 0.9272529482841492}]
```

## Authors

- **Dimitrios P. Panagoulias**, Department of Informatics, University of Piraeus  
- **Persephone Papatheodosiou**, Sleep Research Unit, Department of Psychiatry, National and Kapodistrian University of Athens  
- **Anastasios Bonakis**, Second Department of Neurology, National and Kapodistrian University of Athens  
- **Dimitris Dikeos**, Sleep Research Unit, Department of Psychiatry, National and Kapodistrian University of Athens  
- **Maria Virvou**, Lab of Software Engineering, Department of Informatics, University of Piraeus  
- **George A. Tsihrintzis**, Lab of Pattern Recognition and Machine Learning – Multimedia Systems, Department of Informatics, University of Piraeus

## Citation

You can cite either one or both of the following previous related work: 

- Panagoulias, D.P. et al. “Memory and Schema in Human–Generative Artificial Intelligence Interactions.”
2024 IEEE ICTAI Conference (in press)

Available at: https://ieeexplore.ieee.org/document/10849404

- Panagoulias, D.P. et al. Mathematical representation of memory and schema for improving human-generative AI interactions.”
2024 IEEE IISA Conference (in press)

Available at: https://ieeexplore.ieee.org/document/10786703