README.md

---
library_name: transformers
language:
- en
base_model:
- google-bert/bert-base-uncased
tags:
- Moral Foundation Prediction
- MFT
- Morality
- Morality Values
- BERTForMoralPrediction
- Emotions
- Events
- https://aclanthology.org/2025.coling-main.638.pdf
pipeline_tag: text-classification
license: mit

widget:
- text: "Will you still love me when I'm no longer young and beautiful?"
  example_title: "Sentiment analysis"
---

# ME²-BERT: Are Events and Emotions what you need for Moral Foundation Prediction?

> Moralities, emotions, and events are complex aspects of human cognition, which are often treated separately since capturing their combined effects is challenging, especially due to the lack of annotated data. Leveraging their interrelations hence becomes crucial for advancing the understanding of human moral behaviors.
In this work, we propose ME²-BERT, the first holistic  framework for fine-tuning a pre-trained language model like BERT to the task of moral foundation prediction. ME²-BERT integrates events and emotions for learning domain-invariant morality-relevant text representations. 
Our extensive experiments show that ME²-BERT outperforms existing state-of-the-art methods for moral foundation prediction, with an average percentage increase up to 35% in the out-of-domain scenario.

[Paper](https://aclanthology.org/2025.coling-main.638.pdf) | [Source code](https://github.com/lorenzozangari/ME2-BERT) | [WebApp](https://huggingface.co/spaces/lorenzozan/ME2-BERT)

## Training Data
ME²-BERT was fine-tuned on the [**E2MoCase dataset**](https://arxiv.org/pdf/2409.09001) (available upon request), which consists of 97,251 paragraphs from news articles encompassing both event-based and event-free samples. It includes annotations for:
  - Moral values: Care, Harm, Fairness, Cheating, Loyalty, Betrayal, Authority, Subversion, Purity, Degradation.
  - Emotions: Anticipation, Trust, Disgust, Joy, Optimism, Surprise, Love, Anger, Sadness, Pessimism, Fear.
  - Events in JSON format, including the trigger mention and the entities involved in the event.

---

## Evaluation data
ME²-BERT has been evaluated on:

 - [Moral Foundation Twitter Corpus (MFTC)](https://osf.io/k5n7y/)
 - [Moral Foundation Reddit Corpus (MFRC)](https://huggingface.co/datasets/USC-MOLA-Lab/MFRC)
 - [Extended Moral Foundation Dictionary (eMFD)](https://osf.io/vw85e/)
 - [MoralEvents](https://github.com/launchnlp/MOKA)


## Usage

**NOTE: The following examples were executed using Transformers v 4.47.1. If you encounter any issues running our code, please contact us.**.

```python
from transformers import AutoTokenizer, AutoModel
import torch

model_name = "lorenzozan/ME2-BERT"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True)

text = ["Faithless is he that says farewell when the road darkens."]
inputs = tokenizer(text, padding="max_length", truncation=True, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs, return_dict=False)

print(outputs) # tensor([[0.0185, 0.2401, 0.9166, 0.0498, 0.0453]])
```


By running the model with ```return_dict=True```, it returns a dictionary containing key-value pairs, where each key represents a moral dimension and its corresponding value indicates the associated score.

```python

text = [
     'Faithless is he that says farewell when the road darkens.',
     'The soul is healed by being with children.', 
     'I remembered how we had we had all come to Gatsby’s and guessed at his corruption… while he stood before us concealing an incorruptible dream…',
     'All the variety, all the charm, all the beauty of life is made up of light and shadow, but justice must always remain clear and unbroken.',
     'When tyranny becomes law, rebellion becomes duty.']  
     
max_seq_length = 200

mf_mapping = {'CH':'CARE/HARM','FC':'FAIRNESS/CHEATING', 'LB':'LOYALTY/BETRAYAL', 'AS':'AUTHORITY/SUBVERSION', 'PD': 'PURITY/DEGRADATION'}

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True)

encoded_input = tokenizer(
    text,
    max_length=max_seq_length,  
    padding="max_length",  
    truncation=True,  
    return_tensors="pt", 
)

input_ids = encoded_input["input_ids"]
attention_mask = encoded_input["attention_mask"]

model.eval()  
with torch.no_grad():
    output = model(input_ids=input_ids, attention_mask=attention_mask, return_dict=True)

for i, tt in enumerate(text):
    print(tt)
    for mf, score in output[i].items():
        print(f'{mf_mapping[mf]} : {score}')        
    print()
```

```
Faithless is he that says farewell when the road darkens.
CARE/HARM : 0.05056
FAIRNESS/CHEATING : 0.01845
LOYALTY/BETRAYAL : 0.8676
AUTHORITY/SUBVERSION : 0.01655
PURITY/DEGRADATION : 0.06524

The soul is healed by being with children.
CARE/HARM : 0.83783
FAIRNESS/CHEATING : 0.02016
LOYALTY/BETRAYAL : 0.42663
AUTHORITY/SUBVERSION : 0.00525
PURITY/DEGRADATION : 0.61056

I remembered how we had we had all come to Gatsby’s and guessed at his corruption… while he stood before us concealing an incorruptible dream…
CARE/HARM : 0.00676
FAIRNESS/CHEATING : 0.04518
LOYALTY/BETRAYAL : 0.02287
AUTHORITY/SUBVERSION : 0.00545
PURITY/DEGRADATION : 0.64035

All the variety, all the charm, all the beauty of life is made up of light and shadow, but justice must always remain clear and unbroken.
CARE/HARM : 0.08769
FAIRNESS/CHEATING : 0.95034
LOYALTY/BETRAYAL : 0.05768
AUTHORITY/SUBVERSION : 0.00725
PURITY/DEGRADATION : 0.06396

When tyranny becomes law, rebellion becomes duty.
CARE/HARM : 0.1599
FAIRNESS/CHEATING : 0.91123
LOYALTY/BETRAYAL : 0.4824
AUTHORITY/SUBVERSION : 0.96638
PURITY/DEGRADATION : 0.02086
```

Other examples of usage with different configuration are shown [here](https://github.com/lorenzozangari/ME2-BERT/blob/master/me2bert_example.ipynb).



## References
If you use this model, please cite:
```
@inproceedings{zangari-etal-2025-me2,
    title = "{ME}2-{BERT}: Are Events and Emotions what you need for Moral Foundation Prediction?",
    author = "Zangari, Lorenzo  and
      Greco, Candida M.  and
      Picca, Davide  and
      Tagarelli, Andrea",
      publisher = "Association for Computational Linguistics",
      url = "https://aclanthology.org/2025.coling-main.638/",
      pages = "9516--9532",
      abstract = "Moralities, emotions, and events are complex aspects of human cognition, which are often treated separately since capturing their combined effects is challenging, especially due to the lack of annotated data. Leveraging their interrelations hence becomes crucial for advancing the understanding of human moral behaviors. In this work, we propose ME2-BERT, the first holistic framework for fine-tuning a pre-trained language model like BERT to the task of moral foundation prediction. ME2-BERT integrates events and emotions for learning domain-invariant morality-relevant text representations. Our extensive experiments show that ME2-BERT outperforms existing state-of-the-art methods for moral foundation prediction, with an average increase up to 35{\%} in the out-of-domain scenario."
}
```