| --- |
| language: |
| - pt |
| - en |
| - it |
| - fr |
| - de |
| - pl |
| - ru |
| license: mit |
| library_name: pytorch |
| pipeline_tag: token-classification |
| base_model: xlm-roberta-base |
| tags: |
| - xlm-roberta |
| - persuasion-techniques |
| - disinformation |
| - cross-lingual |
| - multi-label-ner |
| - portuguese |
| - clef-2024 |
| metrics: |
| - f1 |
| model-index: |
| - name: persuasion-lens-pt |
| results: |
| - task: |
| type: token-classification |
| name: Persuasion Technique Detection |
| metrics: |
| - name: F1-micro |
| type: f1 |
| value: 0.134 |
| - name: F1-macro |
| type: f1 |
| value: 0.103 |
| --- |
| |
| # PersuasionLens-PT: Cross-Lingual Detection of Persuasion Techniques in Portuguese |
|
|
| ## Model Description |
|
|
| PersuasionLens-PT is a span-level multi-label NER model for detecting and classifying **23 persuasion techniques** in European Portuguese news articles. It was developed as part of a Master's thesis at FCUP / INESC TEC, following the [CLEF-2024 CheckThat! Lab Task 3](https://checkthat.gitlab.io/clef2024/task3/) framework. |
|
|
| The model tackles a **zero-resource cross-lingual** scenario: no native Portuguese training data exists, so it learns entirely from six other languages (English, Italian, French, German, Polish, Russian) and machine-translated data, then transfers to Portuguese at inference time. |
|
|
| > **Note**: PersuasionLens corresponds to **Model 2** (final system) in the associated Master's thesis, with the IT→PT-only Stage 2 fine-tuning configuration. |
|
|
| ### Key Features |
|
|
| - Detects 23 persuasion techniques at the **text-span level** (not document or sentence level) |
| - **Cross-lingual transfer** from 6 source languages to European Portuguese |
| - Custom **multi-label sigmoid architecture** (not standard BIO tagging) |
| - Surpasses the CLEF-2024 competition winner by +25% relative F1 |
|
|
|
|
| ## Model Details |
|
|
| - **Architecture**: XLMRMultiLabelNER (custom classification head on XLM-RoBERTa) |
| - **Base Model**: [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) (Conneau et al., 2020) |
| - **Model Type**: Multi-label Token Classification |
| - **Parameters**: ~278M (base) + custom head |
| - **Output**: 46 binary outputs per token (B + I for each of 23 techniques) |
| - **Loss Function**: BCEWithLogitsLoss with per-output `pos_weight` |
| - **Max Sequence Length**: 512 tokens (sliding window with 50% overlap at inference) |
| - **Global Classification Threshold**: 0.70 |
| - **Framework**: PyTorch |
|
|
| > **Important**: This model uses a custom architecture and is **not** compatible with `AutoModelForTokenClassification`. It must be loaded using the custom `XLMRMultiLabelNER` class provided in the [GitHub repository](https://github.com/andrevieira1203/MastersThesis). |
|
|
|
|
| ## Intended Uses |
|
|
| ### Primary Use Cases |
|
|
| - Detecting persuasion and manipulation techniques in Portuguese news articles |
| - Supporting media literacy and fact-checking workflows |
| - Research on cross-lingual transfer learning for low-resource NLP tasks |
| - Studying disinformation patterns in Portuguese-language media |
|
|
| ### Out-of-Scope Uses |
|
|
| - General-purpose NER for Portuguese (use domain-specific models instead) |
| - Real-time production classification without adaptation |
| - Legal or policy decision-making without human review |
| - Languages other than Portuguese (though the cross-lingual architecture could be adapted) |
|
|
|
|
| ## Persuasion Techniques |
|
|
| The model detects 23 techniques from the SemEval/CLEF taxonomy (Piskorski et al., 2023), grouped into four macro-categories: |
|
|
| | Category | Techniques | |
| |---|---| |
| | **Justification** | Appeal to Authority, Appeal to Fear/Prejudice, Appeal to Hypocrisy, Appeal to Popularity, Appeal to Time, Appeal to Values, Flag Waving | |
| | **Simplification** | Causal Oversimplification, Consequential Oversimplification, False Dilemma, Straw Man | |
| | **Call** | Appeal to (Strong) Emotions, Conversation Killer, Slogans | |
| | **Distraction** | Red Herring, Whataboutism | |
| | **Reputation** | Doubt, Guilt by Association, Loaded Language, Name Calling/Labelling, Obfuscation/Vagueness/Confusion, Questioning the Reputation, Repetition | |
|
|
|
|
| ## Performance |
|
|
| Evaluated on the CLEF-2024 Portuguese test set using the official character-level evaluation script with partial span matching: |
|
|
| ### Overall Results |
|
|
| | System | F1-micro | F1-macro | |
| |---|---|---| |
| | **PersuasionLens-PT (this model)** | **0.134** | **0.103** | |
| | PersuasionMultiSpan (organisers, post-competition) | 0.132 | 0.120 | |
| | UniBO (CLEF-2024 1st place) | 0.107 | 0.073 | |
| | Baseline (zero-shot) | 0.002 | — | |
|
|
| ### Training Strategy Comparison |
|
|
| | Configuration | F1-micro | F1-macro | |
| |---|---|---| |
| | **Stage 1 (all 6 languages) + Stage 2 (IT→PT only)** | **0.134** | **0.103** | |
| | Stage 1 (all 6 languages) + Stage 2 (IT→PT + FR→PT) | 0.128 | 0.099 | |
| | Stage 1 (all 6 languages) + Stage 2 (IT→PT + FR→PT + RU→PT) | 0.117 | 0.095 | |
|
|
|
|
| ## Usage |
|
|
| ### Requirements |
|
|
| ```bash |
| pip install torch transformers |
| ``` |
|
|
| ### Loading the Model |
|
|
| This model requires the custom `XLMRMultiLabelNER` class. Clone the repository first: |
|
|
| ```bash |
| git clone https://github.com/andrevieira1203/MastersThesis.git |
| ``` |
|
|
| ```python |
| import torch |
| from transformers import AutoTokenizer |
| |
| # Load tokenizer from HuggingFace |
| tokenizer = AutoTokenizer.from_pretrained("AndreCVieira/persuasion-lens-pt") |
| |
| # Load the custom model class (from the GitHub repository) |
| from model import XLMRMultiLabelNER # adjust import path as needed |
| |
| model = XLMRMultiLabelNER.from_custom_checkpoint("path/to/model_directory") |
| model.eval() |
| ``` |
|
|
| ### Web Demo |
|
|
| A Streamlit demo application is available in the [GitHub repository](https://github.com/andrevieira1203/MastersThesis): |
|
|
| ```bash |
| streamlit run app_streamlit.py |
| ``` |
|
|
| ## Training Procedure |
|
|
| ### Two-Stage Training |
|
|
| 1. **Stage 1 (Multilingual Pre-training)**: Fine-tuning on all 6 source languages (EN, IT, FR, DE, PL, RU) with original + machine-translated data |
| 2. **Stage 2 (Cross-lingual Fine-tuning)**: Further fine-tuning on Italian→Portuguese translated data only, guided by embedding similarity analysis |
|
|
| ### Architecture Design |
|
|
| The key architectural innovation is replacing the standard BIO+softmax formulation with a **multi-label sigmoid head**: |
| - Each token produces 46 independent binary predictions (B and I for each of 23 techniques) |
| - This allows a single token to belong to multiple overlapping persuasion technique spans |
| - Per-output `pos_weight` in BCEWithLogitsLoss addresses the severe class imbalance (most tokens are non-persuasive) |
|
|
| ### Language Selection |
|
|
| An embedding analysis using `paraphrase-multilingual-MiniLM-L12-v2` revealed Italian as the closest language to Portuguese (39.5% nearest neighbours), informing the Stage 2 data selection. |
|
|
| ### Translation |
|
|
| - **Italian, French**: Google Translate with character-level offset alignment |
| - **Russian**: DeepL with character-level offset alignment |
|
|
|
|
| ## File Structure |
|
|
| ``` |
| persuasion-lens-pt/ |
| ├── model.pt # Model weights (PyTorch state dict) |
| ├── multilabel_config.json # Multi-label head configuration (23 techniques, thresholds) |
| ├── config.json # XLM-RoBERTa base config |
| ├── tokenizer.json # Tokenizer |
| ├── tokenizer_config.json # Tokenizer config |
| └── special_tokens_map.json # Special tokens |
| ``` |
|
|
|
|
| ## Limitations and Biases |
|
|
| ### Limitations |
|
|
| 1. **No native Portuguese training data**: The model learns Portuguese entirely through cross-lingual transfer; native annotated data would likely improve performance |
| 2. **Custom architecture**: Not compatible with standard HuggingFace pipelines (`AutoModelForTokenClassification`); requires the custom class |
| 3. **Context length**: Limited to 512 tokens per window (mitigated by sliding window with overlap) |
| 4. **Pickle format**: The `model.pt` file uses PyTorch's pickle-based serialisation |
|
|
| ### Potential Biases |
|
|
| - Translation artifacts may introduce systematic errors not present in native Portuguese text |
| - The CLEF training data reflects specific European media landscapes and may not generalise to all Portuguese-language contexts |
| - Performance varies across the 23 techniques; rare techniques are harder to detect |
|
|
|
|
| ## Citation |
|
|
| If you use this model, please cite: |
|
|
| ```bibtex |
| @mastersthesis{vieira2026persuasionlens, |
| title={Language Models for the Detection of Manipulative Discourse and Disinformation in Text}, |
| author={Vieira, André}, |
| year={2026}, |
| school={Faculdade de Ciências da Universidade do Porto (FCUP)}, |
| type={Master's Thesis} |
| } |
| ``` |
|
|
|
|
| ## Acknowledgments |
|
|
| - Supervisors: Nuno Guimarães and Alípio Jorge (FCUP / INESC TEC) |
| - Built on [XLM-RoBERTa](https://huggingface.co/xlm-roberta-base) by Conneau et al. (2020) |
| - Evaluation framework from [CLEF-2024 CheckThat! Lab Task 3](https://checkthat.gitlab.io/clef2024/task3/) (Piskorski et al., 2024) |
| - GitHub repository: [andrevieira1203/MastersThesis](https://github.com/andrevieira1203/MastersThesis) |
|
|
|
|
|
|
