Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,124 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- fake-news-detection
|
| 5 |
+
- multimodal
|
| 6 |
+
- clip
|
| 7 |
+
- bangla
|
| 8 |
+
- english
|
| 9 |
+
- supervised-learning
|
| 10 |
+
- fact-checking
|
| 11 |
+
- image-text
|
| 12 |
+
- misinformation
|
| 13 |
+
- fine-tuned
|
| 14 |
+
library_name: transformers
|
| 15 |
+
inference: true
|
| 16 |
+
language:
|
| 17 |
+
- bn
|
| 18 |
+
- en
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
# ๐ก๏ธ Multimodal BN-EN Fake News Scanner
|
| 22 |
+
|
| 23 |
+
A **fine-tuned CLIP model for detecting fake news in Bangla-English (BN-EN) content** using **text and image analysis**.
|
| 24 |
+
|
| 25 |
+
This model was **supervised-trained on real and fake news pairs** to better detect misinformation in South Asian digital content. During inference, it uses **prompt-based similarity** to classify inputs.
|
| 26 |
+
|
| 27 |
+
Deployed at:
|
| 28 |
+
๐ [https://huggingface.co/jarif/Multimodal-BNEN-Fake-News-Scanner-Model](https://huggingface.co/jarif/Multimodal-BNEN-Fake-News-Scanner-Model)
|
| 29 |
+
|
| 30 |
+
---
|
| 31 |
+
|
| 32 |
+
## โ
Key Features
|
| 33 |
+
|
| 34 |
+
- โ
**Supervised fine-tuning** on fake/real news dataset
|
| 35 |
+
- ๐ **Bangla + English text support**
|
| 36 |
+
- ๐ผ๏ธ Analyzes **image authenticity**
|
| 37 |
+
- ๐ Uses **prompt-based classification** with semantic similarity
|
| 38 |
+
- ๐งฉ Built on `openai/clip-vit-base-patch32`, fine-tuned for misinformation detection
|
| 39 |
+
- ๐ Optimized for **South Asian context**
|
| 40 |
+
- ๐ฆ Fully compatible with Hugging Face `transformers`
|
| 41 |
+
|
| 42 |
+
---
|
| 43 |
+
|
| 44 |
+
## ๐ป Inference Code
|
| 45 |
+
|
| 46 |
+
Classify **images** and **text** as **Real** or **Fake** and display results in a clean table using `tabulate`.
|
| 47 |
+
|
| 48 |
+
### Install Dependencies
|
| 49 |
+
|
| 50 |
+
```
|
| 51 |
+
pip install transformers torch pillow tabulate
|
| 52 |
+
|
| 53 |
+
from transformers import CLIPModel, CLIPProcessor
|
| 54 |
+
from PIL import Image
|
| 55 |
+
import torch
|
| 56 |
+
import torch.nn.functional as F
|
| 57 |
+
from tabulate import tabulate
|
| 58 |
+
|
| 59 |
+
# Load your fine-tuned model
|
| 60 |
+
model = CLIPModel.from_pretrained("jarif/Multimodal-BNEN-Fake-News-Scanner-Model")
|
| 61 |
+
processor = CLIPProcessor.from_pretrained("jarif/Multimodal-BNEN-Fake-News-Scanner-Model")
|
| 62 |
+
|
| 63 |
+
# Define class prompts in Bangla
|
| 64 |
+
class_texts = ["เฆเฆเฆฟ เฆซเงเฆ เฆจเฆฟเฆเฆ", "เฆเฆเฆฟ เฆฐเฆฟเฆฏเฆผเงเฆฒ เฆจเฆฟเฆเฆ"] # ["This is fake news", "This is real news"]
|
| 65 |
+
|
| 66 |
+
# --- Image Classification ---
|
| 67 |
+
image = Image.open("your_image.jpg").convert("RGB") # Replace with your image path
|
| 68 |
+
image_inputs = processor(images=image, return_tensors="pt")
|
| 69 |
+
image_emb = model.get_image_features(**image_inputs)
|
| 70 |
+
|
| 71 |
+
# --- Text Classification ---
|
| 72 |
+
text = "เฆชเฆฆเงเฆฎเฆพ เฆจเฆฆเงเฆฐ เฆชเงเฆฐเฆฌเฆฒ เฆธเงเฆฐเงเฆคเง เฆฒเฆเงเฆเฆเฆพเฆ เฆฌเฆฟเฆฒเงเฆจ เฆนเฆฏเฆผเงเฆเงเฅค"
|
| 73 |
+
text_inputs = processor(text=text, return_tensors="pt", padding=True, truncation=True)
|
| 74 |
+
text_emb = model.get_text_features(**text_inputs)
|
| 75 |
+
|
| 76 |
+
# Get embeddings for class prompts
|
| 77 |
+
class_inputs = processor(text=class_texts, return_tensors="pt", padding=True, truncation=True)
|
| 78 |
+
class_embs = model.get_text_features(**class_inputs)
|
| 79 |
+
|
| 80 |
+
# Normalize embeddings (cosine similarity)
|
| 81 |
+
image_emb = F.normalize(image_emb, p=2, dim=-1)
|
| 82 |
+
text_emb = F.normalize(text_emb, p=2, dim=-1)
|
| 83 |
+
class_embs = F.normalize(class_embs, p=2, dim=-1)
|
| 84 |
+
|
| 85 |
+
# Compute similarity
|
| 86 |
+
image_sims = (image_emb @ class_embs.T).squeeze(0)
|
| 87 |
+
text_sims = (text_emb @ class_embs.T).squeeze(0)
|
| 88 |
+
|
| 89 |
+
# Predict
|
| 90 |
+
image_pred = image_sims.argmax().item()
|
| 91 |
+
text_pred = text_sims.argmax().item()
|
| 92 |
+
|
| 93 |
+
image_label = "๐ Fake" if image_pred == 0 else "โ
Real"
|
| 94 |
+
text_label = "๐ Fake" if text_pred == 0 else "โ
Real"
|
| 95 |
+
|
| 96 |
+
# Create result table
|
| 97 |
+
table = [
|
| 98 |
+
["ImageRelation", image_label],
|
| 99 |
+
["Text Relation", text_label]
|
| 100 |
+
]
|
| 101 |
+
|
| 102 |
+
# Print formatted table
|
| 103 |
+
print(tabulate(table, headers=["Modality", "Prediction"], tablefmt="fancy_grid"))
|
| 104 |
+
```
|
| 105 |
+
|
| 106 |
+
|
| 107 |
+
# ๐ Example Output
|
| 108 |
+
```
|
| 109 |
+
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 110 |
+
โ Modality โ Prediction โ
|
| 111 |
+
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโก
|
| 112 |
+
โ Image Relation โ โ
Real โ
|
| 113 |
+
โ Text Relation โ ๐ Fake โ
|
| 114 |
+
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 115 |
+
```
|
| 116 |
+
|
| 117 |
+
# ๐ License
|
| 118 |
+
- This model is licensed under Apache 2.0.
|
| 119 |
+
- Fine-tuned by Sadik Al Jarif for public misinformation detection.
|
| 120 |
+
|
| 121 |
+
# ๐ Acknowledgements
|
| 122 |
+
- OpenAI CLIP
|
| 123 |
+
- Hugging Face Transformers
|
| 124 |
+
- Bangla NLP Community
|