VictorYeste/human-value-detection-deberta-liwc-22

Human Value Detection – DeBERTa + LIWC-22

This model is the DeBERTa + LIWC-22 feature-augmented 19-way value detector from the paper:

Human Values in a Single Sentence: Moral Presence, Hierarchies, and Transformer Ensembles on the Schwartz Continuum
Víctor Yeste, Paolo Rosso (2026)

It is a multi-label classifier over the 19 refined Schwartz basic values, trained on the English, machine-translated portion of the ValueEval'24 / ValuesML corpus, and augmented with LIWC-22 sentence-level features.

• Inputs to the underlying model:
  • DeBERTa token embeddings for the sentence, plus
  • A LIWC-22 feature vector for that sentence.
• Outputs: a probability for each of the 19 Schwartz values.
• Labels: we collapse “attained” and “constrained” into a single binary label per value (value is expressed vs. not expressed).

This checkpoint corresponds to the “Baseline + LIWC-22” feature-augmented model used in the paper, and is also one of the members of the best-performing DeBERTa ensemble.

⚠️ Important: LIWC-22 features come from the proprietary LIWC-22 software.
This model expects LIWC-22 feature vectors as an additional input; it cannot, by itself, compute LIWC-22 scores from raw text.

---

Intended use

• Research on human value detection, moral language, and lexicon-augmented transformers.
• Baseline / starting point for work on:
  • Schwartz value theory in NLP
  • Moral/value-aware text analysis in news and political discourse
  • Multi-label classification under class imbalance
  • Exploring the impact of LIWC-style features in transformer models

The model was not trained or audited for safety-critical or high-stakes decision-making.

---

Labels

The 19 labels follow the refined Schwartz value continuum:

1. Self-direction: thought
2. Self-direction: action
3. Stimulation
4. Hedonism
5. Achievement
6. Power: dominance
7. Power: resources
8. Face
9. Security: personal
10. Security: societal
11. Tradition
12. Conformity: rules
13. Conformity: interpersonal
14. Humility
15. Benevolence: caring
16. Benevolence: dependability
17. Universalism: concern
18. Universalism: nature
19. Universalism: tolerance

---

How to use

Because this model uses a custom architecture (EnhancedDebertaForSequenceClassification) with an additional LIWC-22 feature branch, it is loaded via:

• AutoModelForSequenceClassification(..., trust_remote_code=True)
• and expects an extra lexicon_features tensor of shape [batch_size, num_categories], where num_categories = model.config.num_categories (the LIWC-22 feature dimension used in training).

1. Minimal example (with dummy LIWC vector)

If you do not have LIWC-22 features, you can still load and run the model by passing a zero vector as placeholder.
This will not reproduce the paper’s performance (since the model was trained with non-zero LIWC-22 features), but it will run:

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "VictorYeste/human-value-detection-deberta-liwc-22"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    trust_remote_code=True,  # important for custom model code
)

values = [
    "Self-direction: thought",
    "Self-direction: action",
    "Stimulation",
    "Hedonism",
    "Achievement",
    "Power: dominance",
    "Power: resources",
    "Face",
    "Security: personal",
    "Security: societal",
    "Tradition",
    "Conformity: rules",
    "Conformity: interpersonal",
    "Humility",
    "Benevolence: caring",
    "Benevolence: dependability",
    "Universalism: concern",
    "Universalism: nature",
    "Universalism: tolerance",
]

id2label = {i: label for i, label in enumerate(values)}

def predict_values(text, liwc_vec=None, threshold=0.50):
    # Tokenize sentence
    enc = tokenizer(text, return_tensors="pt", truncation=True)

    # Build LIWC-22 feature vector
    if liwc_vec is None:
        # Fallback: use zeros (does NOT match the paper’s setup)
        liwc_dim = model.config.num_categories
        liwc_vec = [0.0] * liwc_dim

    liwc_tensor = torch.tensor([liwc_vec], dtype=torch.float32)

    with torch.no_grad():
        outputs = model(**enc, lexicon_features=liwc_tensor)

    logits = outputs.logits.squeeze(0)   # (19,)
    probs = torch.sigmoid(logits)        # tensor (19,)
    probs = probs.cpu().numpy()

    active = probs >= threshold
    active_labels = [id2label[i] for i, is_on in enumerate(active) if is_on]

    return {
        "probs": {id2label[i]: float(p) for i, p in enumerate(probs)},
        "labels": active_labels,
    }

example = "We must do more to protect the environment and future generations."
print(predict_values(example))

Again: with liwc_vec = zeros you are effectively using a text-only input to a model whose classification head was trained assuming LIWC-22 features. For proper use, you should provide real LIWC-22 vectors.

2. Using real LIWC-22 features (recommended)

If you have access to the LIWC-22 software and its sentence-level output (e.g., a CSV with columns like Text-ID, Sentence-ID, Text, Segment, and many LIWC-22 features):

1. For each sentence, extract the same columns you used in training (e.g. all columns from index 4 onwards).
2. Build a vector liwc_vec of length model.config.num_categories.
3. Pass it as lexicon_features:

# Suppose liwc_df is a pandas.DataFrame with your LIWC-22 results
row = liwc_df.iloc[0]
# Use exactly the same set of columns you used during training:
columns_to_use = liwc_df.columns[4:]
liwc_vec = row[columns_to_use].astype(float).tolist()

text = row["Text"]
print(predict_values(text, liwc_vec, threshold=0.30))

This setup mirrors the configuration used in the paper (LIWC-22 features + DeBERTa input) and should get you close to the reported performance, assuming the same data and splits.

Because LIWC-22 is proprietary, this repository does not include the LIWC-22 dictionary or any derived feature files. You must compute those features yourself under your own LIWC-22 license.

---

Training data

The model was trained on the English, machine-translated portion of the ValueEval’24 / ValuesML dataset: • Domain: news articles and political manifestos • Unit of analysis: individual sentences • Labels: 19 refined Schwartz values • Each value has attained and constrained annotations in the original data • For this model, these are collapsed into a single binary label per value

Important: the original dataset is distributed under a restricted Data Usage Agreement. You must obtain the data separately from the ValueEval/ValuesML organisers (e.g. via Zenodo) and respect their license.

---

Training setup

•	Base model: microsoft/deberta-base
•	Task: 19-way multi-label classification
•	Objective: binary cross-entropy (BCEWithLogits) over the 19 labels
•	Inputs:
•	DeBERTa sentence embedding
•	LIWC-22 feature vector (dimension = config.num_categories) passed through a 256→128 MLP branch
•	Max sequence length: 512 tokens
•	Optimizer: AdamW
•	Effective batch size: 16 (batch 4 × gradient accumulation 4)
•	Learning rate: 2e-5
•	Weight decay: 0.15
•	Epochs: up to 10 with early stopping on validation macro–F1
•	Hardware: single GPU with ≤ 8 GB VRAM

This is the “Baseline + LIWC-22” feature-augmented configuration described in the paper.

---

Performance (paper reference)

On the English ValueEval’24 sentence-level splits, the paper compares: • Text-only DeBERTa baseline • DeBERTa with LIWC-22 features (this model) • Other feature-augmented variants (prior-sentence context, topics) • Instruction-tuned LLM baselines (7–9B) • A small soft-voting ensemble of DeBERTa-based models

The LIWC-22–augmented model improves over the text-only baseline; for exact macro–F₁ scores and per-label results, please refer to the paper:

Human Values in a Single Sentence: Moral Presence, Hierarchies, and Transformer Ensembles on the Schwartz Continuum Víctor Yeste, Paolo Rosso (2026)

---

Limitations and bias

•	The model is trained on news and political texts; it may not generalise to:
•	Social media
•	Everyday conversations
•	Other genres or languages
•	Values are annotated at the sentence level; many real-world value cues are only clear in broader context.
•	Rare values (e.g., Humility, Hedonism, Universalism: tolerance) have few positive examples and are harder to predict.
•	LIWC-22 features themselves may encode biases from the dictionary and categories.
•	No systematic bias or fairness analysis has been conducted; the model should not be used for profiling individuals or making high-stakes decisions.

If you use this model, please: • Keep humans in the loop. • Treat outputs as noisy indicators, especially for rare labels.

---

License

The model weights and code in this repository are released under the Apache License 2.0.

This does not grant you any rights over: • The underlying training data (ValueEval/ValuesML) • The LIWC-22 dictionary or software

Please obtain and use those under their own licenses and Data Usage Agreements.

---

Citation

If you use this model or the associated code in your research, please cite:

@misc{yeste2026humanvaluessinglesentence,
      title={Human Values in a Single Sentence: Moral Presence, Hierarchies, and Transformer Ensembles on the Schwartz Continuum}, 
      author={Víctor Yeste and Paolo Rosso},
      year={2026},
      eprint={2601.14172},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2601.14172}, 
}

You may also want to cite the ValueEval / ValuesML dataset:

@misc{ValueEval24Zenodo,
  author    = {{The ValuesML Team}},
  title     = {Touch{\'e}24{-}ValueEval},
  year      = {2024},
  month     = {8},
  version   = {2024-08-09},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.13283288},
  url       = {https://doi.org/10.5281/zenodo.13283288}
}