|
|
--- |
|
|
license: mit |
|
|
datasets: |
|
|
- webis/Touche23-ValueEval |
|
|
language: |
|
|
- en |
|
|
metrics: |
|
|
- f1 |
|
|
tags: |
|
|
- social-values |
|
|
--- |
|
|
# Schwartz Value Classifier |
|
|
This classifier is intended to predict the existence of social values from text snippets. |
|
|
|
|
|
*Disclaimer: this is not the official repo published by the authors of the paper, and may not truly replicate the performance described in the original study* |
|
|
|
|
|
## Value dimensions |
|
|
1. security |
|
|
2. power |
|
|
3. achievement |
|
|
4. hedonism |
|
|
5. stimulation |
|
|
6. self-direction |
|
|
7. universalism |
|
|
8. benevolence |
|
|
9. conformity |
|
|
10. tradition |
|
|
|
|
|
## Datasets |
|
|
This model is finetuned on two datasets: ValueNet and Touche23-ValueEval |
|
|
We follow the original paper to convert both datasets into a binary classification task for each dimension. |
|
|
- ValueNet: a sentence has a positive label if the original label contains 1 (positive) or -1 (negative), and 0 if the original label is 0. |
|
|
- ValueEval: a sentence is assigned a positive label if the original label vector is marked 1 for that dimension. Since the original paper follows a 20-dimension refined categorization, we map them back to 10 dimensions. Therefore, the same sentence appears ten times, once for each dimension. |
|
|
|
|
|
## How to use |
|
|
Start your sentence with a label that indicates which dimension to measure. An example would be: |
|
|
|
|
|
- \<power> [SEP] staying out late after telling my girlfriend I could be home early |
|
|
|
|
|
Please make sure to follow the exact format "<value\_name>" at the beginning of the sentence as this is a special token in the tokenizer: any spaces or different formats will not be encoded correctly. |
|
|
|
|
|
## Performances |
|
|
- F1 score (macro): 0.759 |
|
|
|
|
|
## Training details |
|
|
- Base model: bert-base-uncased |
|
|
- Epochs: 10 w/ early stopping after no F1 increase in 3 epochs |
|
|
- Learning rate: 5e-5 w/ warmup for 0.03 steps and subsequent linear decay |
|
|
- Batch size: 32 |
|
|
- Upsampled training set to maintain 1:1 balance for pos:neg labels |
|
|
|
|
|
## References |
|
|
Do Differences in Values Influence Disagreements in Online Discussions? (EMNLP'23) [link](https://aclanthology.org/2023.emnlp-main.992/) |
