README.md

# Multi-Task BERT Model for Fake News, Hate Speech, and Toxicity Detection

## Model Description
This model is a **multi-task learning framework** based on [BERT (bert-base-german-cased)](https://huggingface.co/bert-base-german-cased), designed to perform binary classification on three tasks simultaneously:
1. Fake News Detection
2. Hate Speech Detection
3. Toxicity Detection

The model utilizes a shared BERT encoder with task-specific fully connected layers for each classification task. It is fine-tuned on task-specific labeled datasets and supports input text in **German**.

### Model Architecture
- **Base Model:** bert-base-german-cased
- **Classifier Heads:**
  - Fully connected layers with 1 output unit for each task (fake news, hate speech, and toxicity).
- **Activation Function:** Sigmoid activation for binary classification.

## Intended Use
This model is intended for applications in:
- Social media monitoring
- Content moderation
- Research on online discourse in German

### Example Use Case
The model can analyze German text to predict whether it contains:
- Fake news (1: True, 0: False)
- Hate speech (1: True, 0: False)
- Toxicity (1: True, 0: False)

## Usage

### Requirements
```bash
pip install torch transformers
```

### Code Example
```python
import torch
from transformers import BertTokenizer, BertModel
import torch.nn as nn

# Load the tokenizer
tokenizer = BertTokenizer.from_pretrained('bert-base-german-cased')

# Define the Multi-Task Model
class MultiTaskModel(nn.Module):
    def __init__(self):
        super(MultiTaskModel, self).__init__()
        self.bert = BertModel.from_pretrained('bert-base-german-cased')
        self.dropout = nn.Dropout(0.3)
        self.fc_fake_news = nn.Linear(self.bert.config.hidden_size, 1)
        self.fc_hate_speech = nn.Linear(self.bert.config.hidden_size, 1)
        self.fc_toxicity = nn.Linear(self.bert.config.hidden_size, 1)

    def forward(self, input_ids, attention_mask):
        outputs = self.bert(input_ids=input_ids, attention_mask=attention_mask)
        pooled_output = outputs[1]  # Get the pooled output
        pooled_output = self.dropout(pooled_output)
        fake_news_output = self.fc_fake_news(pooled_output)
        hate_speech_output = self.fc_hate_speech(pooled_output)
        toxicity_output = self.fc_toxicity(pooled_output)
        return fake_news_output, hate_speech_output, toxicity_output

# Function to load the model
def load_model(device):
    model = MultiTaskModel().to(device)
    model.load_state_dict(torch.load('path_to_your_model.pt'))
    model.eval()
    return model

# Example Usage
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = load_model(device)

text_input = "Mir fallen nur Steuervorteile durch Gender Pay gap ein."
encoding = tokenizer(text_input, return_tensors='pt', padding='max_length', truncation=True, max_length=128)

input_ids = encoding['input_ids'].to(device)
attention_mask = encoding['attention_mask'].to(device)

# Predict
with torch.no_grad():
    outputs_fake_news, outputs_hate_speech, outputs_toxicity = model(input_ids, attention_mask)
    preds_fake_news = torch.sigmoid(outputs_fake_news).squeeze().round().cpu().numpy()
    preds_hate_speech = torch.sigmoid(outputs_hate_speech).squeeze().round().cpu().numpy()
    preds_toxicity = torch.sigmoid(outputs_toxicity).squeeze().round().cpu().numpy()

print(f"Fake News Prediction: {preds_fake_news}")
print(f"Hate Speech Prediction: {preds_hate_speech}")
print(f"Toxicity Prediction: {preds_toxicity}")
```

## Dataset
This model assumes fine-tuning on task-specific datasets for German text. Ensure the training datasets are labeled for fake news, hate speech, and toxicity.

## Evaluation
The model is evaluated using:
- **Binary classification metrics:** F1-score, precision, recall, and accuracy.
- Task-specific benchmarks using separate test sets for each task.

## Limitations and Bias
- **Language Support:** The model only supports German text.
- **Dataset Bias:** Predictions may reflect biases present in the training data.
- **Task-Specific Limitations:** Performance may degrade if the input text contains ambiguous or mixed classifications.

## Citation
If you use this model, please cite:
```bibtex
@article{example2025,
  title={Multi-Task BERT Model for Fake News, Hate Speech, and Toxicity Detection},
  author={Shivang Sinha},
  year={2025}
}