Add narrative classifier (RoBERTa-large, multi-label) + model card

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,206 @@
+---
+language:
+- en
+license: cc-by-nc-4.0
+library_name: transformers
+pipeline_tag: text-classification
+base_model: FacebookAI/roberta-large
+tags:
+- roberta
+- text-classification
+- multi-label-classification
+- disinformation
+- narrative-detection
+- propaganda
+- media-analysis
+metrics:
+- f1
+- precision
+- recall
+---
+
+# Narrative Classifier (RoBERTa-large, multi-label)
+
+A multi-label text classifier that detects **disinformation / propaganda narratives** in news and
+social-media text. Given a piece of text, the model predicts which of **41 predefined narratives**
+(spanning topics such as the war in Ukraine, migration, climate change, COVID-19 / vaccines,
+gender & LGBT+, anti-establishment / anti-EU / anti-NATO framings, etc.) are present.
+
+The model was developed at the **Polish-Japanese Academy of Information Technology (PJAIT / PJATK)**.
+
+- **Architecture:** `RobertaNarrativeModel` — a `roberta-large` encoder + a single linear
+  classification head (`narrative_head`, 1024 → 41) applied to the `<s>` (CLS) token.
+- **Task:** multi-label classification (one input can carry several narratives at once).
+- **Base model:** [`FacebookAI/roberta-large`](https://huggingface.co/FacebookAI/roberta-large)
+- **Parameters:** ~0.4B · **Precision:** FP32 · **Format:** safetensors
+- **Language:** English
+
+> **Note on the architecture.** This repository uses a *custom* model class
+> (`RobertaNarrativeModel`) whose weights are stored under the `transformer.*` and
+> `narrative_head.*` prefixes. It therefore does **not** load directly with
+> `AutoModelForSequenceClassification`. Use the self-contained loading code in the
+> [How to use](#how-to-use) section below.
+
+## Labels
+
+The model outputs 41 labels. The full mapping is in
+[`narrative_labels.json`](./narrative_labels.json) / [`label_config.json`](./label_config.json).
+
+<details>
+<summary>Show all 41 narratives</summary>
+
+| ID | Narrative |
+|----|-----------|
+| 0  | Abortion is evil/immoral/dangerous |
+| 1  | Alternative treatments are more effective than conventional ones |
+| 2  | Climate change is a hoax |
+| 3  | Collapse of Western civilization is imminent |
+| 4  | Conflict is a staged event prepared by outside forces |
+| 5  | Contraception is against nature/dangerous/immoral |
+| 6  | Conventional medicine is ineffective and corrupt |
+| 7  | Conventional medicine is wrong about the causes of diseases |
+| 8  | Elites manipulate elections |
+| 9  | Elites want to take over the world |
+| 10 | European Union is authoritarian |
+| 11 | Feminism is a tool to destroy the natural order and traditional values |
+| 12 | Global elites deliberately cause pandemics and diseases |
+| 13 | Global warming does not exist/is not a serious threat |
+| 14 | Governments fail to take proper action on migration crisis |
+| 15 | Homosexuals are a threat |
+| 16 | Humanity is not responsible for global warming |
+| 17 | LGBT+ is a tool to destroy the natural order and traditional values |
+| 18 | LGBT+ people are mentally ill |
+| 19 | LGBT+ people are privileged |
+| 20 | Media deliberately spreads lies |
+| 21 | Migrants are a burden on the economy |
+| 22 | Migrants are dangerous |
+| 23 | Migrants are destroying local culture and breaking up local communities |
+| 24 | Migration is a conspiracy of global elites |
+| 25 | Most European countries are puppets of the West |
+| 26 | NATO is authoritarian/warmongering |
+| 27 | Official information is a tool to deceive citizens |
+| 28 | Other |
+| 29 | Russia is strong and winning the war |
+| 30 | Sex education is a threat to children |
+| 31 | Solutions to reduce human impact on environment and climate are a conspiracy |
+| 32 | State and international institutions only serve to oppress citizens. |
+| 33 | The West and their allies are immoral/hostile/ineffective |
+| 34 | The energy crisis is artificially created |
+| 35 | Transgender people are a threat |
+| 36 | Ukraine is an evil, aggressive and dangerous country |
+| 37 | Ukrainian refugees are a danger/burden |
+| 38 | Vaccines are dangerous/ineffective/immoral |
+| 39 | Western elites want to destroy the natural order and traditional values |
+| 40 | other |
+
+</details>
+
+## How to use
+
+```python
+import json
+import torch
+from torch import nn
+from transformers import AutoTokenizer, AutoConfig, RobertaModel
+from huggingface_hub import hf_hub_download
+from safetensors.torch import load_file
+
+REPO_ID = "pjait/narrative_classifier"
+
+
+class RobertaNarrativeModel(nn.Module):
+    """roberta-large encoder + a linear head over the <s> (CLS) token."""
+
+    def __init__(self, config, num_labels):
+        super().__init__()
+        self.transformer = RobertaModel(config, add_pooling_layer=False)
+        self.narrative_head = nn.Linear(config.hidden_size, num_labels)
+
+    def forward(self, input_ids, attention_mask=None):
+        out = self.transformer(input_ids=input_ids, attention_mask=attention_mask)
+        cls = out.last_hidden_state[:, 0]          # <s> token representation
+        return self.narrative_head(cls)            # raw logits (multi-label)
+
+
+# --- load config, labels and weights ---------------------------------------
+config = AutoConfig.from_pretrained(REPO_ID)
+tokenizer = AutoTokenizer.from_pretrained(REPO_ID)
+
+with open(hf_hub_download(REPO_ID, "narrative_labels.json")) as f:
+    labels = json.load(f)
+id2narrative = {int(k): v for k, v in labels["id2narrative"].items()}
+num_labels = labels["num_labels"]
+
+model = RobertaNarrativeModel(config, num_labels)
+state_dict = load_file(hf_hub_download(REPO_ID, "model.safetensors"))
+model.load_state_dict(state_dict)
+model.eval()
+
+# --- inference --------------------------------------------------------------
+text = "The vaccines were rushed and are far more dangerous than the virus itself."
+inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
+
+with torch.no_grad():
+    logits = model(**inputs)
+    probs = torch.sigmoid(logits)[0]              # multi-label -> sigmoid
+
+THRESHOLD = 0.5
+predicted = [(id2narrative[i], float(p)) for i, p in enumerate(probs) if p >= THRESHOLD]
+print(sorted(predicted, key=lambda x: -x[1]))
+```
+
+`THRESHOLD` controls precision/recall trade-off; tune it on your own validation data.
+
+## Evaluation
+
+Metrics from [`metrics.txt`](./metrics.txt) (evaluation split, epoch 3):
+
+| Metric | Value |
+|--------|-------|
+| Micro F1 | 0.494 |
+| Macro F1 | 0.185 |
+| Precision | 0.700 |
+| Recall | 0.382 |
+| Subset accuracy | 0.787 |
+| Eval loss | 0.023 |
+
+The gap between micro and macro F1, together with high precision but lower recall, indicates the
+model is conservative and performs unevenly across narratives — likely better on
+well-represented narratives and weaker on rare ones. Treat predictions as a **decision-support
+signal**, not ground truth, and calibrate the threshold for your use case.
+
+## Intended use & limitations
+
+**Intended use.** Research and analysis of disinformation/propaganda narratives in English-language
+media; content moderation triage; media-monitoring dashboards; academic studies of narrative spread.
+
+**Out of scope / cautions.**
+- The model identifies whether text *expresses or discusses* a narrative; it does **not** establish
+  truth, intent, or that the author endorses the narrative (quotation, debunking and reporting can
+  trigger labels).
+- Trained on English; performance on other languages is not guaranteed.
+- Macro F1 is low — rare narratives are unreliable. Do not use for automated, consequential
+  decisions about individuals without human review.
+- Sensitive topics (health, politics, gender, migration). Outputs can reflect biases in the
+  training data. Human oversight is required for any deployment.
+
+## Training
+
+- **Base model:** `FacebookAI/roberta-large` fine-tuned for multi-label narrative classification.
+- **Epochs:** 3 (see `training_args.bin` for the full `TrainingArguments`).
+- **Objective:** multi-label classification (sigmoid + binary cross-entropy over 41 narratives).
+
+## Citation
+
+If you use this model, please cite the Polish-Japanese Academy of Information Technology (PJAIT)
+and the author. *(Add the relevant paper / BibTeX here.)*
+
+```bibtex
+@misc{narrative_classifier_pjait,
+  title  = {Narrative Classifier (RoBERTa-large, multi-label)},
+  author = {Sosnowski, Witold},
+  howpublished = {\url{https://huggingface.co/pjait/narrative_classifier}},
+  note   = {Polish-Japanese Academy of Information Technology (PJAIT)},
+  year   = {2025}
+}
+```

config.json

@@ -0,0 +1,27 @@
+{
+  "_name_or_path": "roberta-large",
+  "architectures": [
+    "RobertaNarrativeModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 50265
+}

label_config.json

@@ -0,0 +1,89 @@
+{
+  "narrative2id": {
+    "Abortion is evil/immoral/dangerous": 0,
+    "Alternative treatments are more effective than conventional ones": 1,
+    "Climate change is a hoax": 2,
+    "Collapse of Western civilization is imminent": 3,
+    "Conflict is a staged event prepared by outside forces": 4,
+    "Contraception is against nature/dangerous/immoral": 5,
+    "Conventional medicine is ineffective and corrupt": 6,
+    "Conventional medicine is wrong about the causes of diseases": 7,
+    "Elites manipulate elections": 8,
+    "Elites want to take over the world": 9,
+    "European Union is authoritarian": 10,
+    "Feminism is a tool to destroy the natural order and traditional values": 11,
+    "Global elites deliberately cause pandemics and diseases": 12,
+    "Global warming does not exist/is not a serious threat": 13,
+    "Governments fail to take proper action on migration crisis": 14,
+    "Homosexuals are a threat": 15,
+    "Humanity is not responsible for global warming": 16,
+    "LGBT+ is a tool to destroy the natural order and traditional values": 17,
+    "LGBT+ people are mentally ill": 18,
+    "LGBT+ people are privileged": 19,
+    "Media deliberately spreads lies": 20,
+    "Migrants are a burden on the economy": 21,
+    "Migrants are dangerous": 22,
+    "Migrants are destroying local culture and breaking up local communities": 23,
+    "Migration is a conspiracy of global elites": 24,
+    "Most European countries are puppets of the West": 25,
+    "NATO is authoritarian/warmongering": 26,
+    "Official information is a tool to deceive citizens": 27,
+    "Other": 28,
+    "Russia is strong and winning the war": 29,
+    "Sex education is a threat to children": 30,
+    "Solutions to reduce human impact on environment and climate are a conspiracy": 31,
+    "State and international institutions only serve to oppress citizens.": 32,
+    "The West and their allies are immoral/hostile/ineffective": 33,
+    "The energy crisis is artificially created": 34,
+    "Transgender people are a threat": 35,
+    "Ukraine is an evil, aggressive and dangerous country": 36,
+    "Ukrainian refugees are a danger/burden": 37,
+    "Vaccines are dangerous/ineffective/immoral": 38,
+    "Western elites want to destroy the natural order and traditional values": 39,
+    "other": 40
+  },
+  "id2narrative": {
+    "0": "Abortion is evil/immoral/dangerous",
+    "1": "Alternative treatments are more effective than conventional ones",
+    "2": "Climate change is a hoax",
+    "3": "Collapse of Western civilization is imminent",
+    "4": "Conflict is a staged event prepared by outside forces",
+    "5": "Contraception is against nature/dangerous/immoral",
+    "6": "Conventional medicine is ineffective and corrupt",
+    "7": "Conventional medicine is wrong about the causes of diseases",
+    "8": "Elites manipulate elections",
+    "9": "Elites want to take over the world",
+    "10": "European Union is authoritarian",
+    "11": "Feminism is a tool to destroy the natural order and traditional values",
+    "12": "Global elites deliberately cause pandemics and diseases",
+    "13": "Global warming does not exist/is not a serious threat",
+    "14": "Governments fail to take proper action on migration crisis",
+    "15": "Homosexuals are a threat",
+    "16": "Humanity is not responsible for global warming",
+    "17": "LGBT+ is a tool to destroy the natural order and traditional values",
+    "18": "LGBT+ people are mentally ill",
+    "19": "LGBT+ people are privileged",
+    "20": "Media deliberately spreads lies",
+    "21": "Migrants are a burden on the economy",
+    "22": "Migrants are dangerous",
+    "23": "Migrants are destroying local culture and breaking up local communities",
+    "24": "Migration is a conspiracy of global elites",
+    "25": "Most European countries are puppets of the West",
+    "26": "NATO is authoritarian/warmongering",
+    "27": "Official information is a tool to deceive citizens",
+    "28": "Other",
+    "29": "Russia is strong and winning the war",
+    "30": "Sex education is a threat to children",
+    "31": "Solutions to reduce human impact on environment and climate are a conspiracy",
+    "32": "State and international institutions only serve to oppress citizens.",
+    "33": "The West and their allies are immoral/hostile/ineffective",
+    "34": "The energy crisis is artificially created",
+    "35": "Transgender people are a threat",
+    "36": "Ukraine is an evil, aggressive and dangerous country",
+    "37": "Ukrainian refugees are a danger/burden",
+    "38": "Vaccines are dangerous/ineffective/immoral",
+    "39": "Western elites want to destroy the natural order and traditional values",
+    "40": "other"
+  },
+  "num_labels": 41
+}

metrics.txt

@@ -0,0 +1,10 @@
+eval_loss: 0.02342209778726101
+eval_narr_micro_f1: 0.49411764705882355
+eval_narr_macro_f1: 0.18548155605115701
+eval_narr_precision: 0.7
+eval_narr_recall: 0.38181818181818183
+eval_narr_subset_acc: 0.7867924528301887
+eval_runtime: 1.7582
+eval_samples_per_second: 301.446
+eval_steps_per_second: 38.107
+epoch: 3.0

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b961b4689633085e2e358a1f32d692677dac2247a477c2b77dacfc00e34a85c0
+size 1421656876

narrative_labels.json

@@ -0,0 +1,89 @@
+{
+  "narrative2id": {
+    "Abortion is evil/immoral/dangerous": 0,
+    "Alternative treatments are more effective than conventional ones": 1,
+    "Climate change is a hoax": 2,
+    "Collapse of Western civilization is imminent": 3,
+    "Conflict is a staged event prepared by outside forces": 4,
+    "Contraception is against nature/dangerous/immoral": 5,
+    "Conventional medicine is ineffective and corrupt": 6,
+    "Conventional medicine is wrong about the causes of diseases": 7,
+    "Elites manipulate elections": 8,
+    "Elites want to take over the world": 9,
+    "European Union is authoritarian": 10,
+    "Feminism is a tool to destroy the natural order and traditional values": 11,
+    "Global elites deliberately cause pandemics and diseases": 12,
+    "Global warming does not exist/is not a serious threat": 13,
+    "Governments fail to take proper action on migration crisis": 14,
+    "Homosexuals are a threat": 15,
+    "Humanity is not responsible for global warming": 16,
+    "LGBT+ is a tool to destroy the natural order and traditional values": 17,
+    "LGBT+ people are mentally ill": 18,
+    "LGBT+ people are privileged": 19,
+    "Media deliberately spreads lies": 20,
+    "Migrants are a burden on the economy": 21,
+    "Migrants are dangerous": 22,
+    "Migrants are destroying local culture and breaking up local communities": 23,
+    "Migration is a conspiracy of global elites": 24,
+    "Most European countries are puppets of the West": 25,
+    "NATO is authoritarian/warmongering": 26,
+    "Official information is a tool to deceive citizens": 27,
+    "Other": 28,
+    "Russia is strong and winning the war": 29,
+    "Sex education is a threat to children": 30,
+    "Solutions to reduce human impact on environment and climate are a conspiracy": 31,
+    "State and international institutions only serve to oppress citizens.": 32,
+    "The West and their allies are immoral/hostile/ineffective": 33,
+    "The energy crisis is artificially created": 34,
+    "Transgender people are a threat": 35,
+    "Ukraine is an evil, aggressive and dangerous country": 36,
+    "Ukrainian refugees are a danger/burden": 37,
+    "Vaccines are dangerous/ineffective/immoral": 38,
+    "Western elites want to destroy the natural order and traditional values": 39,
+    "other": 40
+  },
+  "id2narrative": {
+    "0": "Abortion is evil/immoral/dangerous",
+    "1": "Alternative treatments are more effective than conventional ones",
+    "2": "Climate change is a hoax",
+    "3": "Collapse of Western civilization is imminent",
+    "4": "Conflict is a staged event prepared by outside forces",
+    "5": "Contraception is against nature/dangerous/immoral",
+    "6": "Conventional medicine is ineffective and corrupt",
+    "7": "Conventional medicine is wrong about the causes of diseases",
+    "8": "Elites manipulate elections",
+    "9": "Elites want to take over the world",
+    "10": "European Union is authoritarian",
+    "11": "Feminism is a tool to destroy the natural order and traditional values",
+    "12": "Global elites deliberately cause pandemics and diseases",
+    "13": "Global warming does not exist/is not a serious threat",
+    "14": "Governments fail to take proper action on migration crisis",
+    "15": "Homosexuals are a threat",
+    "16": "Humanity is not responsible for global warming",
+    "17": "LGBT+ is a tool to destroy the natural order and traditional values",
+    "18": "LGBT+ people are mentally ill",
+    "19": "LGBT+ people are privileged",
+    "20": "Media deliberately spreads lies",
+    "21": "Migrants are a burden on the economy",
+    "22": "Migrants are dangerous",
+    "23": "Migrants are destroying local culture and breaking up local communities",
+    "24": "Migration is a conspiracy of global elites",
+    "25": "Most European countries are puppets of the West",
+    "26": "NATO is authoritarian/warmongering",
+    "27": "Official information is a tool to deceive citizens",
+    "28": "Other",
+    "29": "Russia is strong and winning the war",
+    "30": "Sex education is a threat to children",
+    "31": "Solutions to reduce human impact on environment and climate are a conspiracy",
+    "32": "State and international institutions only serve to oppress citizens.",
+    "33": "The West and their allies are immoral/hostile/ineffective",
+    "34": "The energy crisis is artificially created",
+    "35": "Transgender people are a threat",
+    "36": "Ukraine is an evil, aggressive and dangerous country",
+    "37": "Ukrainian refugees are a danger/burden",
+    "38": "Vaccines are dangerous/ineffective/immoral",
+    "39": "Western elites want to destroy the natural order and traditional values",
+    "40": "other"
+  },
+  "num_labels": 41
+}

special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50264": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "mask_token": "<mask>",
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "RobertaTokenizer",
+  "trim_offsets": true,
+  "unk_token": "<unk>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7053bb9d7b515628085d2f02e66e1a12fc9a1fcf26ca67ff374b8418d16ffa6f
+size 5304