| --- |
| language: "nl" |
| tags: |
| - bert |
| - sarcasm-detection |
| - text-classification |
| widget: |
| - text: "We deden een man een nacht in een vat met cola en nu is hij dood" |
| --- |
| |
| # Dutch Sarcasm Detector |
|
|
| Dutch Sarcasm Detector is a text classification model built to detect sarcasm from news article titles. It is fine-tuned on [GroNLP/bert-base-dutch-cased](https://huggingface.co/GroNLP/bert-base-dutch-cased) and the training data consists of ready-made dataset available on Kaggle as well as scraped data from Dutch sarcastic newspaper (De Speld). |
|
|
|
|
| <b>Labels</b>: |
| 0 -> Not Sarcastic; |
| 1 -> Sarcastic |
|
|
| ## Source Data |
| |
| Datasets: |
| - Dutch non-sarcastic data: [Kaggle: Dutch News Articles](https://www.kaggle.com/datasets/maxscheijen/dutch-news-articles) |
| |
| Scraped data: |
| - Dutch sarcastic news from [De Speld](https://speld.nl) |
|
|
|
|
| ## Training Dataset |
| - [helinivan/sarcasm_headlines_multilingual](https://huggingface.co/datasets/helinivan/sarcasm_headlines_multilingual) |
|
|
| ## Codebase: |
| - Git Repo: [Official repository](https://github.com/helinivan/multilingual-sarcasm-detector) |
|
|
| --- |
|
|
| ## Example of classification |
|
|
| ```python |
| from transformers import AutoModelForSequenceClassification |
| from transformers import AutoTokenizer |
| import string |
| |
| def preprocess_data(text: str) -> str: |
| return text.lower().translate(str.maketrans("", "", string.punctuation)).strip() |
| |
| MODEL_PATH = "helinivan/dutch-sarcasm-detector" |
| |
| tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH) |
| model = AutoModelForSequenceClassification.from_pretrained(MODEL_PATH) |
| |
| text = "We deden een man een nacht in een vat met cola en nu is hij dood" |
| tokenized_text = tokenizer([preprocess_data(text)], padding=True, truncation=True, max_length=256, return_tensors="pt") |
| output = model(**tokenized_text) |
| probs = output.logits.softmax(dim=-1).tolist()[0] |
| confidence = max(probs) |
| prediction = probs.index(confidence) |
| results = {"is_sarcastic": prediction, "confidence": confidence} |
| |
| ``` |
|
|
| Output: |
|
|
| ``` |
| {'is_sarcastic': 1, 'confidence': 0.8915400505065918} |
| ``` |
|
|
| ## Performance |
| | Model-Name | F1 | Precision | Recall | Accuracy |
| | ------------- |:-------------| -----| -----| ----| |
| | [helinivan/english-sarcasm-detector ](https://huggingface.co/helinivan/english-sarcasm-detector)| 92.38 | 92.75 | 92.38 | 92.42 |
| | [helinivan/italian-sarcasm-detector ](https://huggingface.co/helinivan/italian-sarcasm-detector) | 88.26 | 87.66 | 89.66 | 88.69 |
| | [helinivan/multilingual-sarcasm-detector ](https://huggingface.co/helinivan/multilingual-sarcasm-detector) | 87.23 | 88.65 | 86.33 | 88.30 |
| | [helinivan/dutch-sarcasm-detector ](https://huggingface.co/helinivan/dutch-sarcasm-detector) | **83.02** | 84.27 | 82.01 | 86.81 |
