--- language: - pt - en - it - fr - de - pl - ru license: mit library_name: pytorch pipeline_tag: token-classification base_model: xlm-roberta-base tags: - xlm-roberta - persuasion-techniques - disinformation - cross-lingual - multi-label-ner - portuguese - clef-2024 metrics: - f1 model-index: - name: persuasion-lens-pt results: - task: type: token-classification name: Persuasion Technique Detection metrics: - name: F1-micro type: f1 value: 0.134 - name: F1-macro type: f1 value: 0.103 --- # PersuasionLens-PT: Cross-Lingual Detection of Persuasion Techniques in Portuguese ## Model Description PersuasionLens-PT is a span-level multi-label NER model for detecting and classifying **23 persuasion techniques** in European Portuguese news articles. It was developed as part of a Master's thesis at FCUP / INESC TEC, following the [CLEF-2024 CheckThat! Lab Task 3](https://checkthat.gitlab.io/clef2024/task3/) framework. The model tackles a **zero-resource cross-lingual** scenario: no native Portuguese training data exists, so it learns entirely from six other languages (English, Italian, French, German, Polish, Russian) and machine-translated data, then transfers to Portuguese at inference time. > **Note**: PersuasionLens corresponds to **Model 2** (final system) in the associated Master's thesis, with the IT→PT-only Stage 2 fine-tuning configuration. ### Key Features - Detects 23 persuasion techniques at the **text-span level** (not document or sentence level) - **Cross-lingual transfer** from 6 source languages to European Portuguese - Custom **multi-label sigmoid architecture** (not standard BIO tagging) - Surpasses the CLEF-2024 competition winner by +25% relative F1 ## Model Details - **Architecture**: XLMRMultiLabelNER (custom classification head on XLM-RoBERTa) - **Base Model**: [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) (Conneau et al., 2020) - **Model Type**: Multi-label Token Classification - **Parameters**: ~278M (base) + custom head - **Output**: 46 binary outputs per token (B + I for each of 23 techniques) - **Loss Function**: BCEWithLogitsLoss with per-output `pos_weight` - **Max Sequence Length**: 512 tokens (sliding window with 50% overlap at inference) - **Global Classification Threshold**: 0.70 - **Framework**: PyTorch > **Important**: This model uses a custom architecture and is **not** compatible with `AutoModelForTokenClassification`. It must be loaded using the custom `XLMRMultiLabelNER` class provided in the [GitHub repository](https://github.com/andrevieira1203/MastersThesis). ## Intended Uses ### Primary Use Cases - Detecting persuasion and manipulation techniques in Portuguese news articles - Supporting media literacy and fact-checking workflows - Research on cross-lingual transfer learning for low-resource NLP tasks - Studying disinformation patterns in Portuguese-language media ### Out-of-Scope Uses - General-purpose NER for Portuguese (use domain-specific models instead) - Real-time production classification without adaptation - Legal or policy decision-making without human review - Languages other than Portuguese (though the cross-lingual architecture could be adapted) ## Persuasion Techniques The model detects 23 techniques from the SemEval/CLEF taxonomy (Piskorski et al., 2023), grouped into four macro-categories: | Category | Techniques | |---|---| | **Justification** | Appeal to Authority, Appeal to Fear/Prejudice, Appeal to Hypocrisy, Appeal to Popularity, Appeal to Time, Appeal to Values, Flag Waving | | **Simplification** | Causal Oversimplification, Consequential Oversimplification, False Dilemma, Straw Man | | **Call** | Appeal to (Strong) Emotions, Conversation Killer, Slogans | | **Distraction** | Red Herring, Whataboutism | | **Reputation** | Doubt, Guilt by Association, Loaded Language, Name Calling/Labelling, Obfuscation/Vagueness/Confusion, Questioning the Reputation, Repetition | ## Performance Evaluated on the CLEF-2024 Portuguese test set using the official character-level evaluation script with partial span matching: ### Overall Results | System | F1-micro | F1-macro | |---|---|---| | **PersuasionLens-PT (this model)** | **0.134** | **0.103** | | PersuasionMultiSpan (organisers, post-competition) | 0.132 | 0.120 | | UniBO (CLEF-2024 1st place) | 0.107 | 0.073 | | Baseline (zero-shot) | 0.002 | — | ### Training Strategy Comparison | Configuration | F1-micro | F1-macro | |---|---|---| | **Stage 1 (all 6 languages) + Stage 2 (IT→PT only)** | **0.134** | **0.103** | | Stage 1 (all 6 languages) + Stage 2 (IT→PT + FR→PT) | 0.128 | 0.099 | | Stage 1 (all 6 languages) + Stage 2 (IT→PT + FR→PT + RU→PT) | 0.117 | 0.095 | ## Usage ### Requirements ```bash pip install torch transformers ``` ### Loading the Model This model requires the custom `XLMRMultiLabelNER` class. Clone the repository first: ```bash git clone https://github.com/andrevieira1203/MastersThesis.git ``` ```python import torch from transformers import AutoTokenizer # Load tokenizer from HuggingFace tokenizer = AutoTokenizer.from_pretrained("AndreCVieira/persuasion-lens-pt") # Load the custom model class (from the GitHub repository) from model import XLMRMultiLabelNER # adjust import path as needed model = XLMRMultiLabelNER.from_custom_checkpoint("path/to/model_directory") model.eval() ``` ### Web Demo A Streamlit demo application is available in the [GitHub repository](https://github.com/andrevieira1203/MastersThesis): ```bash streamlit run app_streamlit.py ``` ## Training Procedure ### Two-Stage Training 1. **Stage 1 (Multilingual Pre-training)**: Fine-tuning on all 6 source languages (EN, IT, FR, DE, PL, RU) with original + machine-translated data 2. **Stage 2 (Cross-lingual Fine-tuning)**: Further fine-tuning on Italian→Portuguese translated data only, guided by embedding similarity analysis ### Architecture Design The key architectural innovation is replacing the standard BIO+softmax formulation with a **multi-label sigmoid head**: - Each token produces 46 independent binary predictions (B and I for each of 23 techniques) - This allows a single token to belong to multiple overlapping persuasion technique spans - Per-output `pos_weight` in BCEWithLogitsLoss addresses the severe class imbalance (most tokens are non-persuasive) ### Language Selection An embedding analysis using `paraphrase-multilingual-MiniLM-L12-v2` revealed Italian as the closest language to Portuguese (39.5% nearest neighbours), informing the Stage 2 data selection. ### Translation - **Italian, French**: Google Translate with character-level offset alignment - **Russian**: DeepL with character-level offset alignment ## File Structure ``` persuasion-lens-pt/ ├── model.pt # Model weights (PyTorch state dict) ├── multilabel_config.json # Multi-label head configuration (23 techniques, thresholds) ├── config.json # XLM-RoBERTa base config ├── tokenizer.json # Tokenizer ├── tokenizer_config.json # Tokenizer config └── special_tokens_map.json # Special tokens ``` ## Limitations and Biases ### Limitations 1. **No native Portuguese training data**: The model learns Portuguese entirely through cross-lingual transfer; native annotated data would likely improve performance 2. **Custom architecture**: Not compatible with standard HuggingFace pipelines (`AutoModelForTokenClassification`); requires the custom class 3. **Context length**: Limited to 512 tokens per window (mitigated by sliding window with overlap) 4. **Pickle format**: The `model.pt` file uses PyTorch's pickle-based serialisation ### Potential Biases - Translation artifacts may introduce systematic errors not present in native Portuguese text - The CLEF training data reflects specific European media landscapes and may not generalise to all Portuguese-language contexts - Performance varies across the 23 techniques; rare techniques are harder to detect ## Citation If you use this model, please cite: ```bibtex @mastersthesis{vieira2026persuasionlens, title={Language Models for the Detection of Manipulative Discourse and Disinformation in Text}, author={Vieira, André}, year={2026}, school={Faculdade de Ciências da Universidade do Porto (FCUP)}, type={Master's Thesis} } ``` ## Acknowledgments - Supervisors: Nuno Guimarães and Alípio Jorge (FCUP / INESC TEC) - Built on [XLM-RoBERTa](https://huggingface.co/xlm-roberta-base) by Conneau et al. (2020) - Evaluation framework from [CLEF-2024 CheckThat! Lab Task 3](https://checkthat.gitlab.io/clef2024/task3/) (Piskorski et al., 2024) - GitHub repository: [andrevieira1203/MastersThesis](https://github.com/andrevieira1203/MastersThesis)