--- library_name: transformers tags: - text-classification - spanish - conflict-detection - xlm-roberta language: - es --- # Model Card for XLM-RoBERTa Spanish Conflict Detection Classifier A fine-tuned XLM-RoBERTa model for detecting social conflict mentions in Spanish news articles. The model is trained on the "Conflicto Social en Noticias" dataset and achieves 91.07% macro-F1 score on test data, making it suitable for automated content classification and conflict-related news filtering. ## Model Details ### Model Description This is a binary text classification model based on **FacebookAI/xlm-roberta-base** fine-tuned to detect whether Spanish news articles discuss social conflict or not. The model was trained using a rigorous multi-seed approach (10 random seeds) to ensure robustness and generalization. The classification task is binary: - **CONFLICTO** (1): News articles that discuss social conflict - **NO_CONFLICTO** (0): News articles that do not discuss social conflict The model achieved strong performance across multiple evaluation runs, with consistent metrics indicating reliable predictions on unseen test data. ### Model Details - **Developed by:** Germán Rosati (Factor~Data, SICSS-Buenos Aires) - **Model type:** Transformer-based text classification - **Language(s) (NLP):** Spanish (es) - **License:** MIT - **Finetuned from model:** [FacebookAI/xlm-roberta-base](https://huggingface.co/FacebookAI/xlm-roberta-base) ### Model Sources - **Repository:** https://github.com/gefero/factor_data_tuto_NLP_SICSS - **Dataset:** [agusnieto77/conflicto-social-noticias-4034](https://huggingface.co/datasets/agusnieto77/conflicto-social-noticias-4034) ## Uses ### Direct Use This model can be used for: - **Automated news classification:** Identify news articles discussing social conflict in Spanish-language sources - **Content moderation:** Flag conflict-related content for review or categorization - **News aggregation:** Filter and organize news by conflict relevance - **Research and analytics:** Systematic analysis of conflict coverage in news media - **Social media monitoring:** Detect posts discussing social conflict ### Downstream Use [optional] This model can be integrated into: - News recommendation systems to provide conflict-focused news feeds - Content management systems for automated news categorization - Data pipelines for media analysis research - Misinformation detection systems (as a conflict-detection component) ### Out-of-Scope Use This model is **not suitable for:** - Languages other than Spanish (though XLM-RoBERTa is multilingual, the model was fine-tuned only on Spanish data) - Content moderation decisions without human review (should be used as a scoring/filtering tool, not final arbiter) - Real-time moderation of live content streams without performance testing in your specific domain - Classification of informal text, social media, or user-generated content not resembling news articles (model trained on news) ## Bias, Risks, and Limitations ### Limitations 1. **Language:** Model trained exclusively on Spanish news articles. Performance on other languages or dialects is unknown. 2. **Domain:** Model trained on news articles. Performance on other text types (social media, academic text, etc.) may be degraded. 3. **Temporal bias:** Dataset represents a specific time period. Linguistic evolution and emerging conflict narratives may not be captured. 4. **Class balance:** Dataset contains both conflict and non-conflict examples. Performance may vary based on class distribution in your specific use case. 5. **Truncation:** Text is truncated to 256 tokens (matching model's training setup). Very long articles may lose important context. 6. **Context sensitivity:** "Conflict" detection is based on textual patterns. Sarcasm, irony, or indirect references may be misclassified. ### Risks and Biases - **Labeling bias:** Model inherits any biases present in the original dataset annotation process - **Geographic bias:** News sources and conflict types in training data may not represent all Spanish-speaking regions equally - **Media bias:** Model trained on news articles, which may have their own coverage biases - **Potential overreach:** Model might flag articles mentioning conflict in non-concerning contexts (e.g., historical analysis, conflict resolution discussion) ### Recommendations 1. **Always validate:** Test the model on your specific data before production deployment 2. **Human review:** Use model predictions as a starting point for human review, not as final decisions 3. **Monitor performance:** Track model performance over time and across different domains 4. **Document decisions:** Keep clear records of how the model is being used and any adjustments made 5. **Consider context:** Combine model predictions with other signals for robust classification decisions ## How to Get Started with the Model ### Installation ```bash pip install transformers torch ``` ### Quickstart - Using the Pipeline API ```python from transformers import pipeline # Initialize the model classifier = pipeline( "text-classification", model="gefero/conflict_detection_ROBERTA_based" ) # Classify text texts = [ "El gobierno anunció nuevas políticas de seguridad social", "Miles de personas protestaron en las calles contra las medidas económicas" ] results = classifier(texts) for text, result in zip(texts, results): print(f"Text: {text[:50]}...") print(f"Label: {result['label']} (score: {result['score']:.4f})\n") ``` ### Quickstart - Using the Model Directly ```python from transformers import AutoTokenizer, AutoModelForSequenceClassification import torch model_name = "gefero/conflict_detection_ROBERTA_based" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForSequenceClassification.from_pretrained(model_name) # Prepare input text = "Manifestantes se enfrentan con la policía" inputs = tokenizer( text, truncation=True, padding="max_length", max_length=256, return_tensors="pt" ) # Get predictions with torch.no_grad(): outputs = model(**inputs) logits = outputs.logits predictions = torch.argmax(logits, dim=-1) confidence = torch.softmax(logits, dim=-1).max().item() labels = {0: "NO_CONFLICTO", 1: "CONFLICTO"} print(f"Prediction: {labels[predictions.item()]} (confidence: {confidence:.4f})") ``` ## Training Details ### Training Data - **Dataset:** [agusnieto77/conflicto-social-noticias-4034](https://huggingface.co/datasets/agusnieto77/conflicto-social-noticias-4034) - **Language:** Spanish (es) - **Domain:** News articles - **Splits used:** - Training set: 70% (~2,823 examples) - Development set: 10% (~403 examples) - Test set: 20% (~806 examples) ### Training Procedure #### Preprocessing - **Tokenization:** XLM-RoBERTa tokenizer - **Max length:** 256 tokens (increased from default 128 to capture longer articles) - **Truncation:** Long articles truncated to max_length - **Padding:** Padded to max_length for batch processing #### Training Hyperparameters - **Base model:** FacebookAI/xlm-roberta-base - **Learning rate:** 2e-5 - **Batch size:** 16 (training), 64 (evaluation) - **Epochs:** 5 - **Weight decay:** 0.01 - **Warmup steps:** 0.1 (proportion of total training steps) - **Evaluation strategy:** Evaluate at the end of each epoch - **Best model selection:** Based on macro-F1 on development set - **Hardware:** GPU with FP16 (mixed precision) when available - **Random seeds:** 10 seeds [0, 1, 7, 13, 42, 100, 123, 2024, 31337, 65535] for robust evaluation #### Training Details - **Framework:** Hugging Face Transformers - **Optimizer:** AdamW (default) - **Metric for best model:** macro-F1 (average of precision and recall across both classes) - **Multi-seed training:** Model was trained 10 times with different random seeds to ensure robustness and to provide confidence intervals on performance metrics ## Evaluation ### Testing Data, Factors & Metrics #### Testing Data - **Split:** Test set (20% of original data, stratified split) - **Size:** ~806 examples - **Language:** Spanish - **Domain:** News articles from the original dataset #### Metrics - **Macro-F1** (primary metric): Average F1-score across both classes - Used for model selection during training - Balances precision and recall - Better for imbalanced or binary classification tasks - **Accuracy:** Overall correctness of predictions - **F1-CONFLICTO:** F1-score specifically for the CONFLICTO class (conflict-related news) - **Precision:** True positives / (true positives + false positives) - **Recall:** True positives / (true positives + false negatives) ### Results #### Official Test Results (10 runs with different seeds) | Metric | Mean | Std Dev | Min | Max | |--------|------|---------|-----|-----| | **Test Macro-F1** | 0.9107 | 0.0071 | 0.8999 | 0.9204 | | **Test Accuracy** | 0.9394 | 0.0053 | 0.9319 | 0.9468 | | **Test F1-CONFLICTO** | 0.8602 | 0.0108 | 0.8433 | 0.8746 | | **Test Precision** | 0.8807 | 0.0296 | 0.8307 | 0.9182 | | **Test Recall** | 0.8419 | 0.0247 | 0.8156 | 0.8883 | #### Per-Seed Results | Seed | Dev Macro-F1 | Test Macro-F1 | Test Accuracy | Test F1-CONFLICTO | Test Precision | Test Recall | |------|--------------|---------------|---------------|-------------------|----------------|-------------| | 0 | 0.9113 | 0.9007 | 0.9332 | 0.8439 | 0.8743 | 0.8156 | | 1 | 0.9116 | 0.8999 | 0.9319 | 0.8433 | 0.8605 | 0.8268 | | 7 | 0.8975 | 0.9092 | 0.9381 | 0.8580 | 0.8728 | 0.8436 | | 13 | 0.9187 | 0.9140 | 0.9431 | 0.8639 | 0.9182 | 0.8156 | | 42 | 0.9154 | 0.9127 | 0.9418 | 0.8622 | 0.9074 | 0.8212 | | 100 | 0.9219 | 0.9136 | 0.9394 | 0.8665 | 0.8457 | 0.8883 | | 123 | 0.9146 | 0.9194 | 0.9455 | 0.8736 | 0.8994 | 0.8492 | | 2024 | 0.9098 | 0.9125 | 0.9406 | 0.8629 | 0.8830 | 0.8436 | | 31337 | 0.9180 | 0.9204 | 0.9468 | 0.8746 | 0.9146 | 0.8380 | | 65535 | 0.9168 | 0.9050 | 0.9332 | 0.8533 | 0.8307 | 0.8771 | #### Summary The model demonstrates **excellent performance** with: - **High macro-F1 (0.91):** Balanced and strong predictions on both classes - **High accuracy (0.94):** Correct classification in 94% of cases - **Robust across seeds:** Low standard deviation indicates consistent generalization - **Strong conflict detection (F1-CONFLICTO: 0.86):** Reliably identifies conflict-related news ## Environmental Impact ### Compute Infrastructure #### Hardware - **GPU:** NVIDIA GPU (exact model unspecified, but typical for Colab) - **CPU:** Supporting processors on Colab infrastructure - **RAM:** Standard Colab allocation #### Training Time - **Per seed:** ~5-10 minutes (5 epochs per training run) - **Total:** ~50-100 minutes for 10 complete runs - **Cloud Platform:** Google Colaboratory (free tier) #### Carbon Emissions Estimated CO2 emissions for multi-seed training approach: **Low to minimal** (Colab's data centers use renewable energy sources). Individual training runs are short (~10 min each) and performed on highly optimized infrastructure. For detailed calculations, see [ML Impact Calculator](https://mlco2.github.io/impact#compute). ## Technical Specifications ### Model Architecture and Objective - **Architecture:** Transformer-based sequence classification - Base: XLM-RoBERTa (12 layers, 768 hidden dimensions, 110M parameters) - Task-specific layer: Linear classification head for 2 classes - **Objective:** Binary cross-entropy loss (standard for text classification) - **Multilingual base:** XLM-RoBERTa trained on 100+ languages, fine-tuned here for Spanish-specific conflict detection ### Input/Output - **Input:** Spanish text (news articles) - **Max length:** 256 tokens - **Output:** Class probabilities for [NO_CONFLICTO, CONFLICTO] ## Citation [optional] If you use this model in research, please cite: **BibTeX:** ```bibtex @software{rosati2024conflictdetection, author = {Rosati, Germán}, title = {XLM-RoBERTa Spanish Conflict Detection Classifier}, year = {2024}, publisher = {Hugging Face Hub}, url = {https://huggingface.co/gefero/conflict_detection_ROBERTA_based} } ``` **APA:** Rosati, G. (2024). XLM-RoBERTa Spanish Conflict Detection Classifier [Machine learning model]. Hugging Face Hub. Retrieved from https://huggingface.co/gefero/conflict_detection_ROBERTA_based ## Model Card Authors - **Germán Rosati** (Factor~Data, SICSS-Buenos Aires) ## Model Card Contact - **Email:** german.rosati@gmail.com - **GitHub:** https://github.com/gefero