Instructions to use QCRI/MemeIntel with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use QCRI/MemeIntel with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="QCRI/MemeIntel")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("QCRI/MemeIntel")
model = AutoModelForImageTextToText.from_pretrained("QCRI/MemeIntel")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use QCRI/MemeIntel with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "QCRI/MemeIntel"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QCRI/MemeIntel",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/QCRI/MemeIntel

SGLang

How to use QCRI/MemeIntel with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "QCRI/MemeIntel" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QCRI/MemeIntel",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "QCRI/MemeIntel" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QCRI/MemeIntel",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use QCRI/MemeIntel with Docker Model Runner:
```
docker model run hf.co/QCRI/MemeIntel
```

MemeIntel: Explainable Detection of Propagandistic and Hateful Memes

MemeIntel is a Vision-Language Model fine-tuned from meta-llama/Llama-3.2-11B-Vision-Instruct for detecting propaganda in Arabic memes and hateful content in English memes, with explainable reasoning.

Model Description

MemeIntel addresses the challenge of understanding and moderating complex, context-dependent multimodal content on social media. The model performs:

Label Detection: Classifies memes into categories (propaganda/not-propaganda/not-meme/other for Arabic; hateful/not-hateful for English)
Explanation Generation: Provides human-readable explanations for its predictions

The model was trained using a novel multi-stage optimization approach on the MemeXplain dataset.

Usage

from transformers import MllamaForConditionalGeneration, AutoProcessor
from PIL import Image

# Load model and processor
model = MllamaForConditionalGeneration.from_pretrained(
    "QCRI/MemeIntel",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
processor = AutoProcessor.from_pretrained("QCRI/MemeIntel")

# Load your meme image
image = Image.open("path/to/meme.jpg")

Arabic Propaganda Meme Detection (Arabic Explanation)

messages = [
    {"role": "system", "content": "You are an expert social media image analyzer specializing in identifying propaganda in Arabic contexts."},
    {"role": "user", "content": [
        {"type": "image"},
        {"type": "text", "text": "You are an expert social media image analyzer specializing in identifying propaganda in Arabic contexts. I will provide you with Arabic memes and the text extracted from these images. Your task is to classify the image as one of the following: 'propaganda', 'not-propaganda', 'not-meme', or 'other', and provide a brief explanation in Arabic. Start your response with 'Label:' followed by the classification label, then on a new line begin with 'Explanation:' and briefly state your reasoning. Text extracted: لما يقولي انتي مالكيش عزيز\nاعز ما ليا البطاطس المقلية"}
    ]}
]

input_text = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(image, input_text, add_special_tokens=False, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(output[0], skip_special_tokens=True))

Arabic Propaganda Meme Detection (English Explanation)

messages = [
    {"role": "system", "content": "You are an expert social media image analyzer specializing in identifying propaganda in Arabic contexts."},
    {"role": "user", "content": [
        {"type": "image"},
        {"type": "text", "text": "You are an expert social media image analyzer specializing in identifying propaganda in Arabic contexts. I will provide you with Arabic memes and the text extracted from these images. Your task is to classify the image as one of the following: 'propaganda', 'not-propaganda', 'not-meme', or 'other', and provide a brief explanation in English. Start your response with 'Label:' followed by the classification label, then on a new line begin with 'Explanation:' and briefly state your reasoning. Text extracted: وأنا أبكي\n٣\nانت تتمنى وانا البي\n{7"}
    ]}
]

input_text = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(image, input_text, add_special_tokens=False, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(output[0], skip_special_tokens=True))

English Hateful Meme Detection

messages = [
    {"role": "system", "content": "You are an expert social media image analyzer specializing in identifying hateful content in memes"},
    {"role": "user", "content": [
        {"type": "image"},
        {"type": "text", "text": "I will provide you with memes and the text extracted from these images. Your task is to classify the image as one of the following: 'hateful' or 'not-hateful' and provide a brief explanation. Start your response with 'Label:' followed by the classification label, then on a new line begin with 'Explanation:' and briefly state your reasoning. Text extracted: bows here, bows there, bows everywhere"}
    ]}
]

input_text = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(image, input_text, add_special_tokens=False, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(output[0], skip_special_tokens=True))

Prompt Templates

Arabic Meme (Arabic Explanation)

System: You are an expert social media image analyzer specializing in identifying propaganda in Arabic contexts.

User: You are an expert social media image analyzer specializing in identifying propaganda in Arabic contexts. I will provide you with Arabic memes and the text extracted from these images. Your task is to classify the image as one of the following: 'propaganda', 'not-propaganda', 'not-meme', or 'other', and provide a brief explanation in Arabic. Start your response with 'Label:' followed by the classification label, then on a new line begin with 'Explanation:' and briefly state your reasoning. Text extracted: {OCR_TEXT}

Arabic Meme (English Explanation)

System: You are an expert social media image analyzer specializing in identifying propaganda in Arabic contexts.

User: You are an expert social media image analyzer specializing in identifying propaganda in Arabic contexts. I will provide you with Arabic memes and the text extracted from these images. Your task is to classify the image as one of the following: 'propaganda', 'not-propaganda', 'not-meme', or 'other', and provide a brief explanation in English. Start your response with 'Label:' followed by the classification label, then on a new line begin with 'Explanation:' and briefly state your reasoning. Text extracted: {OCR_TEXT}

English Hateful Meme

System: You are an expert social media image analyzer specializing in identifying hateful content in memes

User: I will provide you with memes and the text extracted from these images. Your task is to classify the image as one of the following: 'hateful' or 'not-hateful' and provide a brief explanation. Start your response with 'Label:' followed by the classification label, then on a new line begin with 'Explanation:' and briefly state your reasoning. Text extracted: {OCR_TEXT}

Expected Output Format

The model outputs in the following format:

Label: [classification_label]
Explanation: [reasoning for the classification]

Training

Base Model: meta-llama/Llama-3.2-11B-Vision-Instruct
Training Dataset: QCRI/MemeXplain
Training Method: Multi-stage optimization approach

Performance

MemeIntel achieves state-of-the-art results:

ArMeme (Arabic Propaganda): ~3% absolute improvement over previous SOTA
Hateful Memes (English): ~7% absolute improvement over previous SOTA

Citation

If you use this model, please cite:

@inproceedings{kmainasi-etal-2025-memeintel,
    title = "{M}eme{I}ntel: Explainable Detection of Propagandistic and Hateful Memes",
    author = "Kmainasi, Mohamed Bayan  and
      Hasnat, Abul  and
      Hasan, Md Arid  and
      Shahroor, Ali Ezzat  and
      Alam, Firoj",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.emnlp-main.1539/",
    doi = "10.18653/v1/2025.emnlp-main.1539",
    pages = "30263--30279",
    ISBN = "979-8-89176-332-6",
}