README.md

---
license: mit
language:
- en
- ar
- bn
- de
- es
- hi
- ro
- ru
- zh
base_model: Qwen/Qwen3-VL-8B-Instruct
library_name: transformers
pipeline_tag: image-text-to-text
tags:
- memes
- multimodal
- multilingual
- hate-speech
- vision-language-model
- qwen3-vl
datasets:
- QCRI/MemeLens
---

# MemeLens-VLM

**MemeLens** is a unified multilingual, multitask Vision-Language Model (VLM) for meme understanding. It is fine-tuned from [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) using a **classify-then-explain** training strategy on the [MemeLens dataset](https://huggingface.co/datasets/QCRI/MemeLens), which consolidates 38 public meme datasets across 20 tasks and 9 languages.

**Paper:** [MemeLens: Multilingual Multitask VLMs for Memes](https://arxiv.org/abs/2601.12539)

<p align="center">
  <img src="assets/memelens_task_dataset.png" width="100%" alt="MemeLens Data Construction Overview">
</p>

## Overview

| | |
|---|---|
| **Base model** | Qwen3-VL-8B-Instruct |
| **Training** | Classify-then-explain (multi-stage) |
| **Tasks** | 20 tasks: harm, targets, figurative/pragmatic intent, affect |
| **Languages** | AR, BN, DE, EN, ES, HI, RO, RU, ZH |
| **Datasets** | 38 consolidated public meme datasets |

## Results

### Overall Comparison

| Model / Modality | Acc | M-F1 | W-F1 |
|---|---|---|---|
| Uni-modal (Text) | 65.0 | 0.460 | 0.590 |
| Uni-modal (Image) | 63.6 | 0.472 | 0.600 |
| Multi-modal (Seq-Classification) | 71.0 | 0.580 | 0.680 |
| GPT-4.1 (Zero-Shot) | 61.2 | 0.533 | 0.599 |
| Qwen3-VL-8B-Instruct (Zero-Shot) | 55.1 | 0.482 | 0.539 |
| InternVL3.5-8B (Zero-Shot) | 55.4 | 0.476 | 0.545 |
| Gemma-3-12B (Zero-Shot) | 48.2 | 0.439 | 0.485 |
| Qwen3-2B (Zero-Shot) | 45.6 | 0.394 | 0.431 |
| Phi-3.5-Vision-4.2B (Zero-Shot) | 43.8 | 0.393 | 0.447 |
| **MemeLens (Ours)** | **74.1** | **0.625** | **0.720** |

### Per-Dataset Breakdown

| Dataset | Task | Lang. | Text-Only (Acc/Ma/W) | Image-Only (Acc/Ma/W) | MM-Seq (Acc/Ma/W) | **MemeLens** (Acc/Ma/W) |
|---|---|---|---|---|---|---|
| BanglaAbuse | Abuse | BN | .660/.564/.615 | .680/.628/.663 | .731/.698/.723 | **.787/.759/.782** |
| RoMemes | Deepfake | RO | .634/.259/.493 | .645/.399/.630 | .575/.338/.551 | **.770/.491/.753** |
| HarMeme (Co) | Harmful | EN | .712/.499/.706 | .703/.443/.677 | **.811/.546/.797** | .748/.523/.740 |
| HarMeme | Harmful | EN | .499/.338/.489 | .535/.362/.527 | .590/.400/.590 | **.622/.467/.617** |
| Prop2Hate | Propaganda | AR | .746/.427/.637 | .743/.426/.636 | **.800/.650/.760** | .772/.546/.703 |
| MUTE | Propaganda | BN | .642/.556/.602 | .688/.659/.682 | **.730/.710/.730** | .719/.700/.718 |
| Multi3Hate | Hateful | DE | .590/.371/.438 | .557/.504/.533 | .720/.710/.720 | **.754/.731/.745** |
| MIMIC_Isl | Hateful | EN | .647/.633/.635 | .580/.576/.577 | .510/.340/.350 | **.707/.707/.707** |
| MMHS | Hateful | EN | .631/.387/.488 | .621/.495/.561 | **.630**/.500/**.570** | .614/**.516**/.568 |
| Multi3Hate | Hateful | EN | .574/.573/.573 | .508/.507/.507 | **.770/.770/.770** | .741/.735/.734 |
| FHM | Hateful | EN | .633/.541/.592 | .623/.507/.567 | .760/.740/.760 | **.798/.782/.798** |
| Multi3Hate | Hateful | ES | .557/.358/.399 | .672/.661/.668 | .620/.600/.610 | **.800/.796/.799** |
| Multi3Hate | Hateful | HI | .656/.579/.618 | .574/.528/.559 | **.750/.750/.760** | .754/.724/.744 |
| Multi3Hate | Hateful | ZH | .639/.390/.499 | .574/.470/.535 | .640/.610/.640 | **.770/.740/.765** |
| Memotion | Humor | EN | **.353**/.204/.276 | .325/.235/.297 | .350/.250/.310 | .352/**.248/.316** |
| MET-Meme | Intention | EN | .464/.353/.444 | .383/.295/.363 | .320/.220/.340 | **.524/.442/.514** |
| MET-Meme | Intention | ZH | .621/.443/.611 | .443/.212/.376 | .670/.470/.660 | **.710/.521/.701** |
| MET-Meme | Metaphor | EN | .810/.725/.796 | .814/.724/.797 | **.870/.820/.870** | .867/.821/.863 |
| MET-Meme | Metaphor | ZH | .847/.838/.846 | .678/.636/.662 | **.900/.890/.890** | .866/.859/.865 |
| MAMI | Misogyny | EN | .623/.620/.620 | .628/.611/.611 | .750/.740/.740 | **.849/.849/.849** |
| MIMIC2024 | Misogyny (Cat.) | HI-EN | .470/.146/.412 | .470/.146/.412 | .660/.290/.430 | **.766/.592/.659** |
| MIMIC2024 | Misogyny | HI-EN | .673/.671/.671 | .630/.246/.570 | .850/.850/.850 | **.899/.899/.899** |
| Memotion | Motivational | EN | **.647**/.399/.512 | .608/**.451**/.537 | .640/.450/.540 | .637/.450/**.545** |
| MAMI | Objectification | EN | .670/.503/.590 | .732/.671/.714 | .810/.780/.810 | **.835/.797/.826** |
| Memotion | Offensive | EN | .388/.140/.217 | .371/**.227/.334** | **.390**/.220/.330 | .386/.215/.325 |
| MET-Meme | Offensive | EN | .748/.242/.667 | .742/.233/.658 | .740/**.310/.710** | **.748**/.309/.708 |
| MET-Meme | Offensive | ZH | .803/.485/.787 | .742/.223/.684 | .810/.500/.790 | **.830/.535/.819** |
| RoMemes | Political | RO | .677/.404/.547 | .656/.524/.613 | .830/.780/.820 | **.867/.834/.858** |
| ArMeme | Propaganda | AR | .755/.639/.734 | .735/.554/.685 | **.790/.690/.770** | .789/.679/.765 |
| BanglaAbuse | Sarcasm | BN | .639/.568/.599 | .656/.636/.651 | **.680**/.660/.670 | .674/**.661/.672** |
| Memotion | Sarcasm | EN | .502/.167/.335 | .468/**.195/.352** | **.510**/.170/.340 | .501/.167/.337 |
| MAMI | Shaming | EN | .854/.461/.787 | .834/.610/.819 | .870/.710/.870 | **.898/.719/.883** |
| MAMI | Stereotype | EN | .661/.561/.624 | .729/.336/.707 | .740/.700/.730 | **.784/.739/.772** |
| HarMeme (Co) | Target | EN | .777/.449/.788 | .729/.336/.707 | **.870**/.550/.870 | .823/.420/.840 |
| HarMeme | Target | EN | .485/.314/.479 | .451/.204/.404 | **.590**/.350/**.580** | .562/**.493**/.565 |
| Toxic | Toxic | RU | .826/.493/.771 | .839/.495/.777 | .860/.700/.850 | **.866/.691/.853** |
| MAMI | Violence | EN | .853/.504/.793 | .722/.618/.702 | .910/.770/.890 | **.923/.809/.914** |
| BanglaAbuse | Vulgar | BN | .743/.680/.740 | .743/.680/.740 | .800/.750/.800 | **.827/.772/.821** |
| **Average** | | | 0.650/0.460/0.590 | 0.636/0.472/0.600 | 0.706/0.579/0.678 | **0.741/0.625/0.720** |

Acc = Accuracy, Ma = Macro-F1, W = Weighted-F1. Bold indicates best performance for the dataset's official evaluation metric.

## Supported Tasks

<p align="center">
  <img src="assets/Language_Task.png" width="60%" alt="Task and Language Distribution">
</p>

The model supports 20 meme understanding tasks across multiple categories:

| Category | Tasks |
|---|---|
| **Harm** | Hateful, Harmful, Toxic, Abusive, Vulgar |
| **Targets** | Target identification, Misogyny, Misogyny Categories, Objectification, Shaming, Stereotype, Violence |
| **Figurative/Pragmatic** | Propaganda, Metaphor, Intention, Sarcasm, Deepfake, Political |
| **Affect** | Humor, Offensive, Motivational, Sentiment |

## Usage

```python
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
from PIL import Image

model = Qwen3VLForConditionalGeneration.from_pretrained("QCRI/MemeLens-VLM", torch_dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained("QCRI/MemeLens-VLM")

image = Image.open("path/to/meme.jpg")

messages = [
    {"role": "system", "content": "You are an expert social media image analyzer specializing in identifying hateful content in memes."},
    {"role": "user", "content": [
        {"type": "image"},
        {"type": "text", "text": (
            "Analyze the content of the meme to see if it promotes or displays hateful views against "
            "specific individuals or groups. Look for discriminatory, prejudiced, or hate speech elements "
            "associated with protected characteristics, including race, religion, and gender. "
            "Classify as: 'hateful' if present, or 'not-hateful' if absent.\n\n"
            "IMPORTANT: Your response must strictly follow this format:\n"
            "'Label: <label>\nExplanation: <explanation>'\n"
            "where <label> is your classification and <explanation> provides a brief justification "
            "for your decision based on the visual and textual content.\n\n"
            "Text extracted from meme: <OCR text here>"
        )}
    ]}
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], padding=True, return_tensors="pt").to(model.device)

output_ids = model.generate(**inputs, max_new_tokens=512)
output_text = processor.batch_decode(output_ids[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0]
print(output_text)
```

## Prompt Format

The model uses a **system prompt + user instruction** format. Each task has a specific system prompt and instruction template.

**Output format** (all tasks):
```
Label: <label>
Explanation: <explanation>
```

### Task Prompts Reference

> **Note:** All user prompts must end with the following suffix (omitted below for brevity):
> ```
> IMPORTANT: Your response must strictly follow this format:
> 'Label: <label>
> Explanation: <explanation>'
> where <label> is your classification and <explanation> provides a brief justification
> for your decision based on the visual and textual content.
>
> Text extracted from meme: <OCR text>
> ```

<details>
<summary><b>Arabic (AR)</b></summary>

#### Hateful Content Detection (Prop2Hate)
**System:** `You are an expert social media image analyzer specializing in identifying hateful content in memes.`
**Instruction:** `<image> Analyze the content of the Arabic meme and determine if it includes hateful language. Consider whether it displays hatred, discrimination, or bias based on identity or protected traits. Classify as: 'Hateful' if applicable, otherwise 'not-hateful'.`
**Labels:** `Hateful`, `not-hateful`

#### Propaganda Detection (ArMeme)
**System:** `You are an expert social media analyst specializing in identifying propaganda techniques in memes.`
**Instruction:** `<image> Identify if the given Arabic meme is an example of propaganda by checking for biased or misleading narratives intended to affect political or ideological perspectives. Select: 'propaganda' or 'not-propaganda' as appropriate.`
**Labels:** `propaganda`, `not-propaganda`

</details>

<details>
<summary><b>Bengali (BN)</b></summary>

#### Abuse Detection (BanglaAbuseMeme)
**System:** `You are an expert social media image analyzer specializing in identifying abusive content in memes.`
**Instruction:** `<image> Examine the Bengali meme and decide whether it features abusive content. This includes any form of verbal abuse or personal offense. Classify as: 'abusive' or 'not-abusive'.`
**Labels:** `abusive`, `not-abusive`

#### Hateful Content Detection (MUTE)
**System:** `You are an expert social media image analyzer specializing in identifying hateful content in memes.`
**Instruction:** `<image> Is the provided Bengali meme hateful in nature? Review the meme for any content that expresses discrimination or prejudice. After your review, select one of the two possible classifications. Classify as: 'hateful' or 'not-hateful'.`
**Labels:** `hateful`, `not-hateful`

#### Sarcasm Detection (BanglaAbuseMeme)
**System:** `You are an expert social media image analyzer specializing in identifying abusive content in memes.`
**Instruction:** `<image> Assess the communication style of the Bengali meme. Does it rely on sarcasm to make its point? If the meme is straightforward and means exactly what it says, it is not sarcastic. Classify the content as: 'sarcasm' or 'not-sarcasm'.`
**Labels:** `sarcasm`, `not-sarcasm`

#### Sentiment Analysis (BanglaAbuseMeme)
**System:** `You are an expert social media image analyzer specializing in identifying abusive content in memes.`
**Instruction:** `<image> Examine the Bengali meme and identify the sentiment it conveys. Classify the emotion as: 'positive', 'negative', or 'neutral'.`
**Labels:** `positive`, `negative`, `neutral`

#### Vulgar Content Detection (BanglaAbuseMeme)
**System:** `You are an expert social media image analyzer specializing in identifying abusive content in memes.`
**Instruction:** `<image> Evaluate the Bengali meme to see if vulgarity is present. Search for crude, obscene, or improper language or images. Classify as: 'vulgar' if such content is found, otherwise use 'not-vulgar'.`
**Labels:** `vulgar`, `not-vulgar`

</details>

<details>
<summary><b>Chinese (ZH)</b></summary>

#### Hateful Content Detection (Multi3Hate)
**System:** `You are an expert social media image analyzer specializing in identifying hateful content in memes.`
**Instruction:** `<image> Your objective is to classify the Chinese meme based on its content. Determine if it conveys a hateful message targeting any specific group. Use one of the two provided labels: 'hateful' or 'not-hateful'.`
**Labels:** `hateful`, `not-hateful`

#### Intention Detection (MET-Meme)
**System:** `You are an expert social media image analyzer specializing in detecting intentions in memes.`
**Instruction:** `<image> Please review the Chinese meme and decide on the main reason it was created. Categorize its purpose using one of the provided labels which are: 'Entertaining' for amusement, 'Expressive' for emotional expression, 'Interactive' for engagement, 'Offensive' for provocative intent, or 'Other' for unclear intentions.`
**Labels:** `Entertaining`, `Expressive`, `Interactive`, `Offensive`, `Other`

#### Metaphor Detection (MET-Meme)
**System:** `You are an expert social media image analyzer specializing in identifying metaphors in memes.`
**Instruction:** `<image> Identify whether the Chinese meme exemplifies metaphorical communication by spotting elements of indirect meaning or symbolism, as opposed to stating facts directly. Classify as: 'Metaphorical' if you find figurative elements, otherwise choose 'Literal'.`
**Labels:** `Metaphorical`, `Literal`

#### Offensive Content Detection (MET-Meme)
**System:** `You are an expert social media image analyzer specializing in identifying offensive content in memes.`
**Instruction:** `<image> Identify the degree of offensiveness in the supplied Chinese meme. Choose one of the following precise classifications: 'not-offensive', 'slightly-offensive', 'moderately-offensive', or 'very-offensive'.`
**Labels:** `not-offensive`, `slightly-offensive`, `moderately-offensive`, `very-offensive`

#### Sentiment Category (MET-Meme)
**System:** `You are an expert social media image analyzer specializing in sentiment analysis of memes.`
**Instruction:** `<image> Classify the provided Chinese meme based on its primary emotional content. Your selection must be one of the following: 'Happiness', 'Sorrow', 'Anger', 'Fear', 'Surprise', 'Love', or 'Hate'.`
**Labels:** `Happiness`, `Sorrow`, `Anger`, `Fear`, `Surprise`, `Love`, `Hate`

#### Sentiment Degree (MET-Meme)
**System:** `You are an expert social media image analyzer specializing in sentiment analysis of memes.`
**Instruction:** `<image> Please examine the Chinese meme and classify how strongly its sentiment is expressed. Use the following labels for the intensity: 'slightly' for a low degree, 'moderately' for a medium degree, and 'very' for a high degree of emotional expression.`
**Labels:** `slightly`, `moderately`, `very`

</details>

<details>
<summary><b>English (EN)</b></summary>

#### Harmful Content Detection (HarMeme)
**System:** `You are an expert social media image analyzer specializing in identifying harmful content in memes.`
**Instruction:** `<image> Categorize the meme based on a three-point scale of harmfulness. Scrutinize the image and text to make your judgment. Classify as: 'not-harmful', 'partially-harmful', or 'very-harmful'.`
**Labels:** `not-harmful`, `partially-harmful`, `very-harmful`

#### Harmful Content Detection — COVID-19 (HarMeme)
**System:** `You are an expert social media image analyzer specializing in identifying harmful content in memes.`
**Instruction:** `<image> Assess the meme's content concerning the COVID-19 crisis and decide on its classification. The categories reflect increasing levels of potential harm, from none to severe. Classify the meme as either: 'not-harmful', 'partially-harmful', or 'very-harmful'.`
**Labels:** `not-harmful`, `partially-harmful`, `very-harmful`

#### Hateful Content Detection (FHM)
**System:** `You are an expert social media image analyzer specializing in identifying hateful content in memes.`
**Instruction:** `<image> Analyze the content of the meme to see if it promotes or displays hateful views against specific individuals or groups. Look for discriminatory, prejudiced, or hate speech elements associated with protected characteristics, including race, religion, and gender. Classify as: 'hateful' if present, or 'not-hateful' if absent.`
**Labels:** `hateful`, `not-hateful`

#### Hateful Content Detection (MMHS)
**System:** `You are an expert social media image analyzer specializing in identifying hateful content in memes.`
**Instruction:** `<image> Examine the meme and determine if it contains any form of hate speech. Classify as: 'hateful' if the content is discriminatory or disparaging towards a person or group, and 'not-hateful' if it is not.`
**Labels:** `hateful`, `not-hateful`

#### Hateful Content Detection (Multi3Hate)
**System:** `You are an expert social media image analyzer specializing in identifying hateful content in memes.`
**Instruction:** `<image> Analyze the provided English meme to determine if it contains any form of hate speech, prejudice, or discriminatory content. Based on your analysis, categorize the meme using one of the following labels. Classify as: 'hateful' or 'not-hateful'.`
**Labels:** `hateful`, `not-hateful`

#### Hateful Content Detection — Islamophobia (MIMIC)
**System:** `You are an expert social media image analyzer specializing in identifying hateful content in memes.`
**Instruction:** `<image> Please evaluate the provided meme. Your task is to determine if it communicates hateful or discriminatory messages specifically targeting Islam or its adherents. Classify as: 'hateful' or 'not-hateful'.`
**Labels:** `hateful`, `not-hateful`

#### Humor Classification (Memotion)
**System:** `You are an expert social media image analyzer specializing in emotion recognition in memes.`
**Instruction:** `<image> Classify the level of humor in the meme. Analyze how funny the content is based on its comedic elements. Classify as: 'not-funny' for unfunny content, 'funny' for mildly humorous, 'very-funny' for highly humorous, or 'hilarious' for extremely funny content.`
**Labels:** `not-funny`, `funny`, `very-funny`, `hilarious`

#### Intention Detection (MET-Meme)
**System:** `You are an expert social media image analyzer specializing in detecting intentions in memes.`
**Instruction:** `<image> Categorize the primary purpose of the meme. What was the creator's main goal in making it? Please classify the intention as: 'Entertaining', 'Expressive', 'Interactive', or 'Offensive'.`
**Labels:** `Entertaining`, `Expressive`, `Interactive`, `Offensive`

#### Metaphor Detection (MET-Meme)
**System:** `You are an expert social media image analyzer specializing in identifying metaphors in memes.`
**Instruction:** `<image> Determine whether the meme uses metaphorical or literal language. Analyze if the content relies on figurative language, symbolism, or indirect meaning. Classify as: 'Metaphorical' if it uses figurative language, or 'Literal' if the meaning is direct and straightforward.`
**Labels:** `Metaphorical`, `Literal`

#### Misogyny Detection (MAMI)
**System:** `You are an expert social media image analyzer specializing in identifying misogynistic content in memes.`
**Instruction:** `<image> Scrutinize the meme to detect any indication of misogynous content, including aspects like prejudice, discrimination, objectification, or aversion towards women. Label the meme 'misogynous' when identified, or 'not-misogynous' if none are observed.`
**Labels:** `misogynous`, `not-misogynous`

#### Motivational Detection (Memotion)
**System:** `You are an expert social media image analyzer specializing in emotion recognition in memes.`
**Instruction:** `<image> Assess if the meme fits the motivational category by checking for inspirational, encouraging, or uplifting content. Classify using: 'motivational' for memes with these features, 'not-motivational' otherwise.`
**Labels:** `motivational`, `not-motivational`

#### Objectification Detection (MAMI)
**System:** `You are an expert social media image analyzer specializing in identifying objectification in memes.`
**Instruction:** `<image> Evaluate the meme for signs that it depicts women as objects, prioritizes appearance over character, or undermines their autonomy. Classify as: 'objectification' when such elements are present; otherwise, use 'not-objectification'.`
**Labels:** `objectification`, `not-objectification`

#### Offensive Content Detection (Memotion)
**System:** `You are an expert social media image analyzer specializing in identifying offensive content in memes.`
**Instruction:** `<image> Carefully read the meme and judge how objectionable the material is. Then, classify it as: 'not-offensive', 'slightly-offensive', 'very-offensive', or 'hateful-offensive', based on the degree of offensiveness present.`
**Labels:** `not-offensive`, `slightly-offensive`, `very-offensive`, `hateful-offensive`

#### Offensive Content Detection (MET-Meme)
**System:** `You are an expert social media image analyzer specializing in identifying offensive content in memes.`
**Instruction:** `<image> Judge the degree of offensiveness present in the meme. Evaluate how inappropriate or offensive the content is, and assign one of the following labels: 'not-offensive', 'slightly-offensive', 'moderately-offensive', or 'very-offensive'.`
**Labels:** `not-offensive`, `slightly-offensive`, `moderately-offensive`, `very-offensive`

#### Sarcasm Detection (Memotion)
**System:** `You are an expert social media image analyzer specializing in emotion recognition in memes.`
**Instruction:** `<image> Your objective is to identify the type of sarcasm used in the meme. Determine if the content is straightforward or if it uses irony to convey its message. Please assign a classification based on the complexity of the sarcasm. Classify as: 'not-sarcastic', 'general-sarcasm', 'twisted-meaning', or 'very-twisted'.`
**Labels:** `not-sarcastic`, `general-sarcasm`, `twisted-meaning`, `very-twisted`

#### Sentiment Analysis (Memotion)
**System:** `You are an expert social media image analyzer specializing in sentiment analysis of memes.`
**Instruction:** `<image> Review the meme and determine its general emotional sentiment. Based on your analysis, assign one of the following labels: 'very-negative', 'negative', 'neutral', 'positive', or 'very-positive'.`
**Labels:** `very-negative`, `negative`, `neutral`, `positive`, `very-positive`

#### Sentiment Category (MET-Meme)
**System:** `You are an expert social media image analyzer specializing in sentiment analysis of memes.`
**Instruction:** `<image> For the meme presented, decide which single emotional category it belongs to, considering its overall message and tone. The available classifications are: 'Happiness', 'Sorrow', 'Anger', 'Fear', 'Surprise', 'Love', or 'Hate'.`
**Labels:** `Happiness`, `Sorrow`, `Anger`, `Fear`, `Surprise`, `Love`, `Hate`

#### Sentiment Degree (MET-Meme)
**System:** `You are an expert social media image analyzer specializing in sentiment analysis of memes.`
**Instruction:** `<image> Considering the overall sentiment in the meme, classify its intensity by selecting: 'slightly' for minimal emotional presence, 'moderately' for a balanced intensity, or 'very' if the sentiment is intense and vivid.`
**Labels:** `slightly`, `moderately`, `very`

#### Shaming Detection (MAMI)
**System:** `You are an expert social media image analyzer specializing in identifying shaming content in memes.`
**Instruction:** `<image> Classify whether the meme contains shaming content directed at women. Determine if it aims to humiliate, embarrass, or shame women about their appearance, behavior, or choices. Classify as: 'shaming' if such content is present, or 'not-shaming' if it is not.`
**Labels:** `shaming`, `not-shaming`

#### Stereotype Detection (MAMI)
**System:** `You are an expert social media image analyzer specializing in identifying stereotypes in memes.`
**Instruction:** `<image> Review the meme and determine if it contains stereotypical content about women—look for generalized, simplified, or biased messages. Classify as: 'stereotype' if you find such traits, otherwise 'not-stereotype'.`
**Labels:** `stereotype`, `not-stereotype`

#### Target Identification (HarMeme)
**System:** `You are an expert social media image analyzer specializing in identifying harmful content in memes.`
**Instruction:** `<image> For the provided meme, interpret who or what is being singled out. Is the commentary aimed at one person, a community, an organization, all of society, or no target? Make your classification using: 'individual', 'community', 'organization', 'society', or 'none'.`
**Labels:** `individual`, `community`, `organization`, `society`, `none`

#### Target Identification — COVID-19 (HarMeme)
**System:** `You are an expert social media image analyzer specializing in identifying harmful content in memes.`
**Instruction:** `<image> Review the COVID-19 meme and determine the entity that is depicted as the target within the content. Assign one of the following categories: 'individual', 'community', 'organization', 'society', or 'none'.`
**Labels:** `individual`, `community`, `organization`, `society`, `none`

#### Violence Detection (MAMI)
**System:** `You are an expert social media image analyzer specializing in identifying violent content in memes.`
**Instruction:** `<image> Is there content in this meme that shows, glorifies, or promotes physical, sexual, or psychological violence toward women? Please make a determination. Classify as: 'violence' if yes, and 'not-violence' if no.`
**Labels:** `violence`, `not-violence`

</details>

<details>
<summary><b>German (DE)</b></summary>

#### Hateful Content Detection (Multi3Hate)
**System:** `You are an expert social media image analyzer specializing in identifying hateful content in memes.`
**Instruction:** `<image> Analyze the German meme for hateful content. Determine if it promotes hate, discrimination, or prejudice against protected groups. Classify as: 'hateful' if it contains hate speech, or 'not-hateful' if it does not.`
**Labels:** `hateful`, `not-hateful`

</details>

<details>
<summary><b>Hindi / Hindi-English (HI / HI-EN)</b></summary>

#### Hateful Content Detection (Multi3Hate)
**System:** `You are an expert social media image analyzer specializing in identifying hateful content in memes.`
**Instruction:** `<image> Examine the content of the Hindi meme. Does it contain language or imagery that is discriminatory or incites hatred? Assign one of the following labels: 'hateful' or 'not-hateful'.`
**Labels:** `hateful`, `not-hateful`

#### Misogyny Detection (MIMIC2024)
**System:** `You are an expert social media image analyzer specializing in identifying misogynistic content in memes.`
**Instruction:** `<image> Observe the Hindi-English meme and ascertain if there are signs of misogyny, such as actions or language indicative of objectification, discrimination, prejudice, or hostility towards women. Classify strictly as: 'misogynous' or 'not-misogynous'.`
**Labels:** `misogynous`, `not-misogynous`

#### Misogyny Categories (MIMIC2024)
**System:** `You are an expert social media image analyzer specializing in identifying misogynistic content in memes.`
**Instruction:** `<image> Inspect the Hindi-English meme and decide whether its misogynous nature stems from objectification, prejudice, humiliation, a combination of these, or is unspecified. Select from the labels: 'Objectification', 'Prejudice', 'Humiliation', 'Objectification, Humiliation', 'Objectification, Prejudice', 'Prejudice, Humiliation', or 'Unspecified'.`
**Labels:** `Objectification`, `Prejudice`, `Humiliation`, `Objectification, Humiliation`, `Objectification, Prejudice`, `Prejudice, Humiliation`, `Unspecified`

</details>

<details>
<summary><b>Romanian (RO)</b></summary>

#### Deepfake Detection (RoMemes)
**System:** `You are an expert image analyst specializing in detecting manipulated or deepfake content in memes.`
**Instruction:** `<image> Assess the provided Romanian meme. Is the image content authentic, or has it been altered? If altered, was it done using AI (deepfake) or other methods? Based on your analysis, classify the meme as: 'DeepFake', 'Fake', or 'Real'.`
**Labels:** `DeepFake`, `Fake`, `Real`

#### Emotion Recognition (RoMemes)
**System:** `You are an expert social media image analyzer specializing in emotion recognition in memes.`
**Instruction:** `<image> Determine the emotional impact of the Romanian meme. Analyze its components to select the most accurate emotional descriptor from the options provided. Classify as: 'Joy', 'Sadness', 'Anger', 'Fear', 'Surprise', or 'Love'.`
**Labels:** `Joy`, `Sadness`, `Anger`, `Fear`, `Surprise`, `Love`

#### Political Content Detection (RoMemes)
**System:** `You are an expert social media analyst specializing in analyzing political content in memes.`
**Instruction:** `<image> Assess whether the Romanian meme features political topics, including discussions about politicians, legislation, or political controversies. Classify the meme as either 'political' for relevant content, or 'not-political' if absent.`
**Labels:** `political`, `not-political`

#### Sentiment Analysis (RoMemes)
**System:** `You are an expert social media image analyzer specializing in sentiment analysis of memes.`
**Instruction:** `<image> Investigate the sentiment expressed by the following Romanian meme. Select the appropriate label: 'positive' if the meme is upbeat or approving, 'negative' if it is critical or displeased, or 'neutral' if it remains impartial.`
**Labels:** `positive`, `negative`, `neutral`

</details>

<details>
<summary><b>Russian (RU)</b></summary>

#### Toxicity Detection (Toxic Memes)
**System:** `You are an expert social media image analyzer specializing in identifying toxic content in memes.`
**Instruction:** `<image> Evaluate the Russian meme for the presence of toxicity. This involves identifying any language or imagery that is damaging, disrespectful, or offensive to individuals or groups. Based on your assessment, classify as: 'toxic' or 'not-toxic'.`
**Labels:** `toxic`, `not-toxic`

</details>

<details>
<summary><b>Spanish (ES)</b></summary>

#### Hateful Content Detection (Multi3Hate)
**System:** `You are an expert social media image analyzer specializing in identifying hateful content in memes.`
**Instruction:** `<image> Identify whether the Spanish meme displays hateful attitudes or sentiments, specifically looking for hate speech against protected groups. Classify your findings as: 'hateful' if hate speech is evident, or 'not-hateful' if it is not.`
**Labels:** `hateful`, `not-hateful`

</details>

## Citation

```bibtex
@misc{shahroor2026memelensmultilingualmultitaskvlms,
      title={MemeLens: Multilingual Multitask VLMs for Memes},
      author={Ali Ezzat Shahroor and Mohamed Bayan Kmainasi and Abul Hasnat and Dimitar Dimitrov and Giovanni Da San Martino and Preslav Nakov and Firoj Alam},
      year={2026},
      eprint={2601.12539},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2601.12539},
}
```