Upload AbuseBert Model

README.md

@@ -1,22 +1,63 @@
----
-license: apache-2.0
-language:
-- en
-base_model:
-- vinai/bertweet-base
-pipeline_tag: text-classification
-tags:
-- text-classification
-- abusive-language
-- hate-speech
-- toxicity
-- bert
-- abusive
-- cyberviolence
-- abusive-language-detection
----
-AbuseBERT is a BERT-based model fine-tuned for abusive language detection.  
-It leverages ten diverse datasets harmonized under a unified taxonomy to improve cross-dataset generalization.  
-The integrated approach mitigates sampling and lexical biases, achieving superior F1 scores on held-out benchmarks compared to models trained on individual datasets.  
-
-AbuseBERT is designed to be **robust and generalizable across platforms, domains, and contexts**, making it well-suited for research, social media moderation studies, and real-world abusive language detection tasks.
+# AbuseBERT
+
+## Model Description
+
+**AbuseBERT** is a **BERT-based classification model** fine-tuned for **abusive language detection**, optimized for **cross-dataset generalization**.
+
+> Abusive language detection models often suffer from poor generalization due to **sampling and lexical biases** in individual datasets. Our approach addresses this by integrating **ten publicly available abusive language datasets**, harmonizing labels and preprocessing textual samples to create a **broader and more representative training distribution**.
+
+**Key Findings:**
+- Individual dataset models: average F1 = **0.60**  
+- Integrated model: F1 = **0.84**  
+- Dataset contribution to performance improvements correlates with **lexical diversity (0.71 correlation)**  
+- Integration exposes models to diverse abuse patterns, enhancing **real-world generalization**
+
+---
+
+## Conclusion / Takeaways
+
+- No single dataset captures the full spectrum of abusive language; each dataset reflects a **limited slice** of the problem space.  
+- Systematically integrating ten heterogeneous datasets significantly improves classification performance on a **held-out benchmark**.  
+- Lexically dissimilar datasets contribute more to **enhancing generalization**.  
+- The integrated model demonstrates superior **cross-dataset performance** compared to models trained on individual datasets.
+
+---
+
+## Paper Reference
+
+Samaneh Hosseini Moghaddam, Kelly Lyons, Frank Rudzicz, Cheryl Regehr, Vivek Goel, Kaitlyn Regehr,  
+“**Enhancing machine learning in abusive language detection with dataset aggregation**,” in *Proc. 35th IEEE Int. Conf. Collaborative Advances in Software Computing (CASC)*, 2025.
+
+---
+
+## Intended Use
+
+**Recommended:**
+- Detecting abusive language in text from social media or online platforms
+- Research on bias mitigation and cross-dataset generalization
+- Supporting safe and inclusive online environments
+
+**Not Recommended:**
+- Fully automated moderation without human oversight
+- High-stakes legal or policy decisions
+
+---
+
+## Usage Example
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+
+# Load model and tokenizer
+tokenizer = AutoTokenizer.from_pretrained("Samanehmoghaddam/AbuseBERT")
+model = AutoModelForSequenceClassification.from_pretrained("Samanehmoghaddam/AbuseBERT")
+
+# Sample input
+text = "Your example text here."
+inputs = tokenizer(text, return_tensors="pt")
+outputs = model(**inputs)
+
+# Predicted label
+predicted_label = torch.argmax(outputs.logits, dim=1).item()
+print(f"Predicted label: {predicted_label}")

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<mask>": 64000
+}

config.json

@@ -0,0 +1,30 @@
+{
+  "_name_or_path": "AbuseBERT",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 130,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "tokenizer_class": "BertweetTokenizer",
+  "torch_dtype": "float32",
+  "transformers_version": "4.32.1",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 64001
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bca7e8d9c0d25798898638c573004a28ca1424017f1988e48c967cfa528a6f14
+size 539674993

special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "64000": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "normalization": false,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "BertweetTokenizer",
+  "unk_token": "<unk>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c84e38a5d3b5bbe5621f040405ed889dbb8bcd4b8bfcd8a930464a4922c70d0
+size 4027