Update README.md

README.md

@@ -1,75 +1,152 @@
----
-library_name: transformers
-license: mit
-base_model: jhu-clsp/mmBERT-small
-tags:
-- generated_from_trainer
-metrics:
-- f1
-- precision
-- recall
-model-index:
-- name: mmBERT-small-multilingual-sentiment
-  results: []
----
+---
+library_name: transformers
+license: mit
+base_model: jhu-clsp/mmBERT-small
+tags:
+- sentiment
+- text-classification
+- multilingual
+- modernbert
+- sentiment-analysis
+- product-reviews
+- place-reviews
+- mmbert
+metrics:
+- f1
+- precision
+- recall
+model-index:
+- name: mmBERT-small-multilingual-sentiment
+  results: []
+datasets:
+- clapAI/MultiLingualSentiment
+language:
+- en
+- zh
+- vi
+- ko
+- ja
+- ar
+- de
+- es
+- fr
+- hi
+- id
+- it
+- ms
+- pt
+- ru
+- tr
+pipeline_tag: text-classification
+---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-# mmBERT-small-multilingual-sentiment
+# clapAI/mmBERT-small-multilingual-sentiment
 
-This model is a fine-tuned version of [jhu-clsp/mmBERT-small](https://huggingface.co/jhu-clsp/mmBERT-small) on an unknown dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.4605
-- F1: 81.8318
-- Precision: 81.8361
-- Recall: 81.8321
-
-## Model description
-
-More information needed
-
-## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
-
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 5e-05
-- train_batch_size: 512
-- eval_batch_size: 512
-- seed: 0
-- distributed_type: multi-GPU
-- num_devices: 2
-- gradient_accumulation_steps: 2
-- total_train_batch_size: 2048
-- total_eval_batch_size: 1024
-- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: cosine
-- lr_scheduler_warmup_ratio: 0.01
-- num_epochs: 5
-
-### Training results
-
-| Training Loss | Epoch | Step | Validation Loss | F1      | Precision | Recall  |
-|:-------------:|:-----:|:----:|:---------------:|:-------:|:---------:|:-------:|
-| 1.8064        | 1.0   | 1537 | 0.4447          | 80.9603 | 80.9710   | 81.0960 |
-| 1.6408        | 2.0   | 3074 | 0.4309          | 81.7277 | 81.8109   | 81.6765 |
-| 1.4703        | 3.0   | 4611 | 0.4252          | 82.1472 | 82.1151   | 82.1871 |
-| 1.3121        | 4.0   | 6148 | 0.4393          | 82.0532 | 82.0873   | 82.0275 |
-| 1.1733        | 5.0   | 7685 | 0.4605          | 81.8318 | 81.8361   | 81.8321 |
-
-
-### Framework versions
-
-- Transformers 4.55.0
-- Pytorch 2.8.0+cu128
-- Datasets 3.6.0
-- Tokenizers 0.21.4
+## Introduction
+
+**mmBERT-small-multilingual-sentiment** is a multilingual sentiment classification model, part of
+the [Multilingual-Sentiment](https://huggingface.co/collections/clapAI/multilingual-sentiment-677416a6b23e03f52cb6cc3f)
+collection.
+
+The model is fine-tuned from [jhu-clsp/mmBERT-small](https://huggingface.co/jhu-clsp/mmBERT-small) using the
+multilingual sentiment
+dataset [clapAI/MultiLingualSentiment](https://huggingface.co/datasets/clapAI/MultiLingualSentiment).
+
+Model supports multilingual sentiment classification across 16+ languages, including English, Vietnamese, Chinese,
+French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Arabic, and more.
+
+## Key Highlights
+> 📈 **Improved accuracy**: Achieves **F1 = 82.2**.  
+> 📜 **Long context support**: Handles sequences up to **8192 tokens**.  
+> 🪶 **Efficient size**: Only **140M parameters**, smaller than RoBERTa-base (278M) with better performance.  
+> ⚡ **FlashAttention-2 support**: Enables much faster inference on modern GPUs.
+
+## Evaluation & Performance
+
+Results on the test split
+of [clapAI/MultiLingualSentiment](https://huggingface.co/datasets/clapAI/MultiLingualSentiment)
+
+|                                                      Model                                                      |                            Pretrained Model                            | Parameters | Context-length | F1-score |
+|:---------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------:|:----------:|----------------|:--------:|
+| [clapAI/mmBERT-small-multilingual-sentiment](https://huggingface.co/clapAI/mmBERT-small-multilingual-sentiment) | [jhu-clsp/mmBERT-small](https://huggingface.co/jhu-clsp/mmBERT-small)  |    140M    | 8192           | **82.2** |
+| [modernBERT-base-multilingual-sentiment](https://huggingface.co/clapAI/modernBERT-base-multilingual-sentiment)  | [ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base)  |    150M    | 8192           |  80.16   |
+|    [roberta-base-multilingual-sentiment](https://huggingface.co/clapAI/roberta-base-multilingual-sentiment)     | [XLM-roberta-base](https://huggingface.co/FacebookAI/xlm-roberta-base) |    278M    | 512            |   81.8   | 
+
+## How to use
+
+### Installation
+
+```bash
+pip install torch==2.8
+pip install transformers==4.55.0
+```
+
+`Optional: accelerate inference with FlashAttention-2 (if supported by your GPU):`
+
+```bash
+pip install packaging
+pip install ninja
+MAX_JOBS=4 pip install flash-attn --no-build-isolation
+```
+
+### Example Usage
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+model_id = "clapAI/mmBERT-small-multilingual-sentiment"
+# Load the tokenizer and model
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+dtype = torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16
+model = AutoModelForSequenceClassification.from_pretrained(
+    model_id,
+    torch_dtype=dtype,
+    # Uncomment if device supports FA2
+    # attn_implementation="flash_attention_2" 
+)
+
+model.to(device)
+model.eval()
+
+# Retrieve labels from the model's configuration
+id2label = model.config.id2label
+
+texts = [
+    "I absolutely love the new design of this app!",  # English
+    "الخدمة كانت سيئة للغاية.",
+    "Ich bin sehr zufrieden mit dem Kauf.",  # German
+    "El producto llegó roto y no funciona.",  # Spanish
+    "J'adore ce restaurant, la nourriture est délicieuse!",  # French
+    "Makanannya benar-benar tidak enak.",  # Indonesian
+    "この製品は本当に素晴らしいです！",  # Japanese
+    "고객 서비스가 정말 실망스러웠어요.",  # Korean
+    "Этот фильм просто потрясающий!",  # Russian
+    "Tôi thực sự yêu thích sản phẩm này!",  # Vietnamese
+    "质量真的很差。"  # Chinese
+]
+
+for text in texts:
+    inputs = tokenizer(text, return_tensors="pt").to(device)
+    with torch.inference_mode():
+        outputs = model(**inputs)
+        prediction = id2label[outputs.logits.argmax(dim=-1).item()]
+    print(f"Text: {text} | Prediction: {prediction}")
+```
+
+## Citation
+
+If you use this model, please consider citing:
+
+```bibtex
+@misc{clapAI_mmbert_small_multilingual_sentiment,
+      title={mmBERT-small-multilingual-sentiment: A Multilingual Sentiment Classification Model},
+      author={clapAI},
+      howpublished={\url{https://huggingface.co/clapAI/mmBERT-small-multilingual-sentiment}},
+      year={2025},
+}