gefero/conflict_detection_ROBERTA_based

Model Card for XLM-RoBERTa Spanish Conflict Detection Classifier

A fine-tuned XLM-RoBERTa model for detecting social conflict mentions in Spanish news articles. The model is trained on the "Conflicto Social en Noticias" dataset and achieves 91.07% macro-F1 score on test data, making it suitable for automated content classification and conflict-related news filtering.

Model Details

Model Description

This is a binary text classification model based on FacebookAI/xlm-roberta-base fine-tuned to detect whether Spanish news articles discuss social conflict or not. The model was trained using a rigorous multi-seed approach (10 random seeds) to ensure robustness and generalization.

The classification task is binary:

• CONFLICTO (1): News articles that discuss social conflict
• NO_CONFLICTO (0): News articles that do not discuss social conflict

The model achieved strong performance across multiple evaluation runs, with consistent metrics indicating reliable predictions on unseen test data.

Model Details

• Developed by: Germán Rosati (Factor~Data, SICSS-Buenos Aires)
• Model type: Transformer-based text classification
• Language(s) (NLP): Spanish (es)
• License: MIT
• Finetuned from model: FacebookAI/xlm-roberta-base

Model Sources

• Repository: https://github.com/gefero/factor_data_tuto_NLP_SICSS
• Dataset: agusnieto77/conflicto-social-noticias-4034

Uses

Direct Use

This model can be used for:

• Automated news classification: Identify news articles discussing social conflict in Spanish-language sources
• Content moderation: Flag conflict-related content for review or categorization
• News aggregation: Filter and organize news by conflict relevance
• Research and analytics: Systematic analysis of conflict coverage in news media
• Social media monitoring: Detect posts discussing social conflict

Downstream Use [optional]

This model can be integrated into:

• News recommendation systems to provide conflict-focused news feeds
• Content management systems for automated news categorization
• Data pipelines for media analysis research
• Misinformation detection systems (as a conflict-detection component)

Out-of-Scope Use

This model is not suitable for:

• Languages other than Spanish (though XLM-RoBERTa is multilingual, the model was fine-tuned only on Spanish data)
• Content moderation decisions without human review (should be used as a scoring/filtering tool, not final arbiter)
• Real-time moderation of live content streams without performance testing in your specific domain
• Classification of informal text, social media, or user-generated content not resembling news articles (model trained on news)

Bias, Risks, and Limitations

Limitations

1. Language: Model trained exclusively on Spanish news articles. Performance on other languages or dialects is unknown.

2. Domain: Model trained on news articles. Performance on other text types (social media, academic text, etc.) may be degraded.

3. Temporal bias: Dataset represents a specific time period. Linguistic evolution and emerging conflict narratives may not be captured.

4. Class balance: Dataset contains both conflict and non-conflict examples. Performance may vary based on class distribution in your specific use case.

5. Truncation: Text is truncated to 256 tokens (matching model's training setup). Very long articles may lose important context.

6. Context sensitivity: "Conflict" detection is based on textual patterns. Sarcasm, irony, or indirect references may be misclassified.

Risks and Biases

• Labeling bias: Model inherits any biases present in the original dataset annotation process
• Geographic bias: News sources and conflict types in training data may not represent all Spanish-speaking regions equally
• Media bias: Model trained on news articles, which may have their own coverage biases
• Potential overreach: Model might flag articles mentioning conflict in non-concerning contexts (e.g., historical analysis, conflict resolution discussion)

Recommendations

1. Always validate: Test the model on your specific data before production deployment
2. Human review: Use model predictions as a starting point for human review, not as final decisions
3. Monitor performance: Track model performance over time and across different domains
4. Document decisions: Keep clear records of how the model is being used and any adjustments made
5. Consider context: Combine model predictions with other signals for robust classification decisions

How to Get Started with the Model

Installation

pip install transformers torch

Quickstart - Using the Pipeline API

from transformers import pipeline

# Initialize the model
classifier = pipeline(
    "text-classification",
    model="gefero/conflict_detection_ROBERTA_based"
)

# Classify text
texts = [
    "El gobierno anunció nuevas políticas de seguridad social",
    "Miles de personas protestaron en las calles contra las medidas económicas"
]

results = classifier(texts)
for text, result in zip(texts, results):
    print(f"Text: {text[:50]}...")
    print(f"Label: {result['label']} (score: {result['score']:.4f})\n")

Quickstart - Using the Model Directly

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "gefero/conflict_detection_ROBERTA_based"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Prepare input
text = "Manifestantes se enfrentan con la policía"
inputs = tokenizer(
    text,
    truncation=True,
    padding="max_length",
    max_length=256,
    return_tensors="pt"
)

# Get predictions
with torch.no_grad():
    outputs = model(**inputs)
    logits = outputs.logits
    predictions = torch.argmax(logits, dim=-1)
    confidence = torch.softmax(logits, dim=-1).max().item()

labels = {0: "NO_CONFLICTO", 1: "CONFLICTO"}
print(f"Prediction: {labels[predictions.item()]} (confidence: {confidence:.4f})")

Training Details

Training Data

• Dataset: agusnieto77/conflicto-social-noticias-4034
• Language: Spanish (es)
• Domain: News articles
• Splits used:
  • Training set: 70% (~2,823 examples)
  • Development set: 10% (~403 examples)
  • Test set: 20% (~806 examples)

Training Procedure

Preprocessing

• Tokenization: XLM-RoBERTa tokenizer
• Max length: 256 tokens (increased from default 128 to capture longer articles)
• Truncation: Long articles truncated to max_length
• Padding: Padded to max_length for batch processing

Training Hyperparameters

• Base model: FacebookAI/xlm-roberta-base
• Learning rate: 2e-5
• Batch size: 16 (training), 64 (evaluation)
• Epochs: 5
• Weight decay: 0.01
• Warmup steps: 0.1 (proportion of total training steps)
• Evaluation strategy: Evaluate at the end of each epoch
• Best model selection: Based on macro-F1 on development set
• Hardware: GPU with FP16 (mixed precision) when available
• Random seeds: 10 seeds [0, 1, 7, 13, 42, 100, 123, 2024, 31337, 65535] for robust evaluation

Training Details

• Framework: Hugging Face Transformers
• Optimizer: AdamW (default)
• Metric for best model: macro-F1 (average of precision and recall across both classes)
• Multi-seed training: Model was trained 10 times with different random seeds to ensure robustness and to provide confidence intervals on performance metrics

Evaluation

Testing Data, Factors & Metrics

Testing Data

• Split: Test set (20% of original data, stratified split)
• Size: ~806 examples
• Language: Spanish
• Domain: News articles from the original dataset

Metrics

• Macro-F1 (primary metric): Average F1-score across both classes

  • Used for model selection during training
  • Balances precision and recall
  • Better for imbalanced or binary classification tasks

• Accuracy: Overall correctness of predictions

• F1-CONFLICTO: F1-score specifically for the CONFLICTO class (conflict-related news)

• Precision: True positives / (true positives + false positives)

• Recall: True positives / (true positives + false negatives)

Results

Official Test Results (10 runs with different seeds)

Metric	Mean	Std Dev	Min	Max
Test Macro-F1	0.9107	0.0071	0.8999	0.9204
Test Accuracy	0.9394	0.0053	0.9319	0.9468
Test F1-CONFLICTO	0.8602	0.0108	0.8433	0.8746
Test Precision	0.8807	0.0296	0.8307	0.9182
Test Recall	0.8419	0.0247	0.8156	0.8883

Per-Seed Results

Seed	Dev Macro-F1	Test Macro-F1	Test Accuracy	Test F1-CONFLICTO	Test Precision	Test Recall
0	0.9113	0.9007	0.9332	0.8439	0.8743	0.8156
1	0.9116	0.8999	0.9319	0.8433	0.8605	0.8268
7	0.8975	0.9092	0.9381	0.8580	0.8728	0.8436
13	0.9187	0.9140	0.9431	0.8639	0.9182	0.8156
42	0.9154	0.9127	0.9418	0.8622	0.9074	0.8212
100	0.9219	0.9136	0.9394	0.8665	0.8457	0.8883
123	0.9146	0.9194	0.9455	0.8736	0.8994	0.8492
2024	0.9098	0.9125	0.9406	0.8629	0.8830	0.8436
31337	0.9180	0.9204	0.9468	0.8746	0.9146	0.8380
65535	0.9168	0.9050	0.9332	0.8533	0.8307	0.8771

Summary

The model demonstrates excellent performance with:

• High macro-F1 (0.91): Balanced and strong predictions on both classes
• High accuracy (0.94): Correct classification in 94% of cases
• Robust across seeds: Low standard deviation indicates consistent generalization
• Strong conflict detection (F1-CONFLICTO: 0.86): Reliably identifies conflict-related news

Environmental Impact

Compute Infrastructure

Hardware

• GPU: NVIDIA GPU (exact model unspecified, but typical for Colab)
• CPU: Supporting processors on Colab infrastructure
• RAM: Standard Colab allocation

Training Time

• Per seed: ~5-10 minutes (5 epochs per training run)
• Total: ~50-100 minutes for 10 complete runs
• Cloud Platform: Google Colaboratory (free tier)

Carbon Emissions

Estimated CO2 emissions for multi-seed training approach: Low to minimal (Colab's data centers use renewable energy sources). Individual training runs are short (~10 min each) and performed on highly optimized infrastructure.

For detailed calculations, see ML Impact Calculator.

Technical Specifications

Model Architecture and Objective

• Architecture: Transformer-based sequence classification

  • Base: XLM-RoBERTa (12 layers, 768 hidden dimensions, 110M parameters)
  • Task-specific layer: Linear classification head for 2 classes

• Objective: Binary cross-entropy loss (standard for text classification)

• Multilingual base: XLM-RoBERTa trained on 100+ languages, fine-tuned here for Spanish-specific conflict detection

Input/Output

• Input: Spanish text (news articles)
• Max length: 256 tokens
• Output: Class probabilities for [NO_CONFLICTO, CONFLICTO]

Citation [optional]

If you use this model in research, please cite:

BibTeX:

@software{rosati2024conflictdetection,
  author = {Rosati, Germán},
  title = {XLM-RoBERTa Spanish Conflict Detection Classifier},
  year = {2024},
  publisher = {Hugging Face Hub},
  url = {https://huggingface.co/gefero/conflict_detection_ROBERTA_based}
}

APA:

Rosati, G. (2024). XLM-RoBERTa Spanish Conflict Detection Classifier [Machine learning model]. Hugging Face Hub. Retrieved from https://huggingface.co/gefero/conflict_detection_ROBERTA_based

Model Card Authors

• Germán Rosati (Factor~Data, SICSS-Buenos Aires)

Model Card Contact

• Email: german.rosati@gmail.com
• GitHub: https://github.com/gefero