---
language:
- afr
- als
- amh
- arb
- ars
- ary
- arz
- asm
- azj
- bel
- ben
- bew
- bos
- bul
- cat
- ces
- ckb
- cmn
- cym
- dan
- deu
- div
- ekk
- ell
- eng
- epo
- eus
- fao
- fas
- fil
- fin
- fra
- fry
- gle
- glg
- guj
- hau
- heb
- hin
- hrv
- hun
- hye
- ind
- isl
- ita
- jpn
- kan
- kat
- kaz
- khk
- khm
- kin
- kir
- kmr
- kor
- lao
- lat
- lit
- ltz
- lvs
- mal
- mar
- mkd
- mlt
- mya
- nld
- nno
- nob
- npi
- nrm
- ory
- pan
- pbt
- plt
- pol
- por
- ron
- rus
- sin
- slk
- slv
- snd
- som
- spa
- srp
- swe
- swh
- tam
- tel
- tgk
- tha
- tur
- ukr
- urd
- uzn
- vie
- xho
- yue
- zsm
license: mit
base_model:
- intfloat/multilingual-e5-small
datasets:
- agentlans/multilingual-document-classification
metrics:
- f1
- loss
model-index:
- name: multilingual-e5-small-domain-classifier
  results:
  - task:
      type: text-classification
      name: Text Classification
    metrics:
    - type: f1
      value: 0.7709
      name: Evaluation F1
    - type: loss
      value: 0.9974
      name: Evaluation Loss
---
# multilingual-e5-small Domain Classifier

A fine-tuned version of the **bert** architecture (`BertForSequenceClassification`) optimized for the `text-classification` task.

- **Model type:** bert
- **Problem Type:** single_label_classification
- **Number of Labels:** 26
- **Vocabulary Size:** 250037
- **License:** MIT

## Use

To get started with this model in Python using the Hugging Face Transformers library, run the following code:

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "agentlans/multilingual-e5-small-domain-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Replace this with your input text."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
predicted_class_name = model.config.id2label[predicted_class_id]

print(f"Predicted Class ID: {predicted_class_id}")
print(f"Predicted Class Name: {predicted_class_name}")
```

## Intended Uses & Limitations

### Intended Use
This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:

| Label ID | Label Name |
|---|---|
| 0 | Adult |
| 1 | Arts_and_Entertainment |
| 2 | Autos_and_Vehicles |
| 3 | Beauty_and_Fitness |
| 4 | Books_and_Literature |
| 5 | Business_and_Industrial |
| 6 | Computers_and_Electronics |
| 7 | Finance |
| 8 | Food_and_Drink |
| 9 | Games |
| 10 | Health |
| 11 | Hobbies_and_Leisure |
| 12 | Home_and_Garden |
| 13 | Internet_and_Telecom |
| 14 | Jobs_and_Education |
| 15 | Law_and_Government |
| 16 | News |
| 17 | Online_Communities |
| 18 | People_and_Society |
| 19 | Pets_and_Animals |
| 20 | Real_Estate |
| 21 | Science |
| 22 | Sensitive_Subjects |
| 23 | Shopping |
| 24 | Sports |
| 25 | Travel_and_Transportation |

## Training Details

### Hyperparameters
The following hyperparameters were used during fine-tuning:
- **Learning Rate:** 5e-05
- **Train Batch Size:** 8
- **Eval Batch Size:** 8
- **Optimizer:** OptimizerNames.ADAMW_TORCH_FUSED
- **Number of Epochs:** 3.0
- **Mixed Precision:** BF16

<details>
<summary><b>Show Advanced Training Configuration</b></summary>

#### Optimization & Regularization
- **Gradient Accumulation Steps:** 1
- **Learning Rate Scheduler:** SchedulerType.LINEAR
- **Warmup Steps:** 0
- **Warmup Ratio:** None
- **Weight Decay:** 0.0
- **Max Gradient Norm:** 1.0

#### Hardware & Reproducibility
- **Number of GPUs:** 1
- **Seed:** 42

</details>

## Training Results & Evaluation

During fine-tuning, the model achieved the following results on the evaluation set:

| Metric | Value |
|---|---|
| **Train Loss** | 0.6686 |
| **Validation Loss** | 0.9974 |
| **Validation F1 Score** | 0.7709 |
| **Total FLOPs** | 7.9086e+15 |

### Speed Performance
- **Training Runtime:** 1639.5865 seconds
- **Train Samples per Second:** 292.775
- **Evaluation Runtime:** 10.8576 seconds
- **Eval Samples per Second:** 1842.216


<details>
<summary><b>Show Detailed Training Logs</b></summary>

### Training Logs History

| Step | Epoch | Learning Rate | Training Loss | Validation Loss | Validation F1 |
|---|---|---|---|---|---|
| 500 | 0.025 | 4.9584e-05 | 2.602 | N/A | N/A |
| 1000 | 0.05 | 4.9168e-05 | 1.8965 | N/A | N/A |
| 1500 | 0.075 | 4.8751e-05 | 1.604 | N/A | N/A |
| 2000 | 0.1 | 4.8334e-05 | 1.3957 | N/A | N/A |
| 2500 | 0.125 | 4.7918e-05 | 1.322 | N/A | N/A |
| 3000 | 0.15 | 4.7501e-05 | 1.2218 | N/A | N/A |
| 3500 | 0.175 | 4.7084e-05 | 1.195 | N/A | N/A |
| 4000 | 0.2 | 4.6668e-05 | 1.1313 | N/A | N/A |
| 4500 | 0.225 | 4.6251e-05 | 1.0902 | N/A | N/A |
| 5000 | 0.25 | 4.5835e-05 | 1.0637 | N/A | N/A |
| 5500 | 0.275 | 4.5418e-05 | 1.0626 | N/A | N/A |
| 6000 | 0.3 | 4.5001e-05 | 1.0054 | N/A | N/A |
| 6500 | 0.325 | 4.4585e-05 | 1.0253 | N/A | N/A |
| 7000 | 0.35 | 4.4168e-05 | 1.0127 | N/A | N/A |
| 7500 | 0.375 | 4.3751e-05 | 0.9714 | N/A | N/A |
| 8000 | 0.4 | 4.3335e-05 | 0.9589 | N/A | N/A |
| 8500 | 0.425 | 4.2918e-05 | 0.9808 | N/A | N/A |
| 9000 | 0.45 | 4.2502e-05 | 0.9392 | N/A | N/A |
| 9500 | 0.475 | 4.2085e-05 | 0.9304 | N/A | N/A |
| 10000 | 0.5 | 4.1668e-05 | 0.9369 | N/A | N/A |
| 10500 | 0.525 | 4.1252e-05 | 0.9181 | N/A | N/A |
| 11000 | 0.55 | 4.0835e-05 | 0.8996 | N/A | N/A |
| 11500 | 0.575 | 4.0418e-05 | 0.9111 | N/A | N/A |
| 12000 | 0.6 | 4.0002e-05 | 0.9033 | N/A | N/A |
| 12500 | 0.625 | 3.9585e-05 | 0.917 | N/A | N/A |
| 13000 | 0.65 | 3.9169e-05 | 0.8872 | N/A | N/A |
| 13500 | 0.675 | 3.8752e-05 | 0.8604 | N/A | N/A |
| 14000 | 0.7 | 3.8335e-05 | 0.8628 | N/A | N/A |
| 14500 | 0.725 | 3.7919e-05 | 0.8929 | N/A | N/A |
| 15000 | 0.75 | 3.7502e-05 | 0.8585 | N/A | N/A |
| 15500 | 0.775 | 3.7085e-05 | 0.9014 | N/A | N/A |
| 16000 | 0.8 | 3.6669e-05 | 0.8581 | N/A | N/A |
| 16500 | 0.825 | 3.6252e-05 | 0.8622 | N/A | N/A |
| 17000 | 0.85 | 3.5836e-05 | 0.873 | N/A | N/A |
| 17500 | 0.875 | 3.5419e-05 | 0.8446 | N/A | N/A |
| 18000 | 0.9 | 3.5002e-05 | 0.819 | N/A | N/A |
| 18500 | 0.925 | 3.4586e-05 | 0.8458 | N/A | N/A |
| 19000 | 0.95 | 3.4169e-05 | 0.8458 | N/A | N/A |
| 19500 | 0.975 | 3.3752e-05 | 0.8497 | N/A | N/A |
| 20000 | 1.0 | 3.3336e-05 | 0.7989 | N/A | N/A |
| 20002 | 1.0 | N/A | N/A | 0.8514 | 0.7452 |
| 20500 | 1.025 | 3.2919e-05 | 0.6034 | N/A | N/A |
| 21000 | 1.05 | 3.2503e-05 | 0.6148 | N/A | N/A |
| 21500 | 1.075 | 3.2086e-05 | 0.614 | N/A | N/A |
| 22000 | 1.1 | 3.1669e-05 | 0.5895 | N/A | N/A |
| 22500 | 1.125 | 3.1253e-05 | 0.6483 | N/A | N/A |
| 23000 | 1.15 | 3.0836e-05 | 0.6331 | N/A | N/A |
| 23500 | 1.175 | 3.0419e-05 | 0.5885 | N/A | N/A |
| 24000 | 1.2 | 3.0003e-05 | 0.6082 | N/A | N/A |
| 24500 | 1.225 | 2.9586e-05 | 0.6312 | N/A | N/A |
| 25000 | 1.25 | 2.9170e-05 | 0.6033 | N/A | N/A |
| 25500 | 1.275 | 2.8753e-05 | 0.6006 | N/A | N/A |
| 26000 | 1.3 | 2.8336e-05 | 0.6283 | N/A | N/A |
| 26500 | 1.325 | 2.7920e-05 | 0.6319 | N/A | N/A |
| 27000 | 1.35 | 2.7503e-05 | 0.5913 | N/A | N/A |
| 27500 | 1.375 | 2.7086e-05 | 0.6037 | N/A | N/A |
| 28000 | 1.4 | 2.6670e-05 | 0.6025 | N/A | N/A |
| 28500 | 1.425 | 2.6253e-05 | 0.6067 | N/A | N/A |
| 29000 | 1.45 | 2.5837e-05 | 0.6075 | N/A | N/A |
| 29500 | 1.475 | 2.5420e-05 | 0.6035 | N/A | N/A |
| 30000 | 1.5 | 2.5003e-05 | 0.5826 | N/A | N/A |
| 30500 | 1.525 | 2.4587e-05 | 0.5905 | N/A | N/A |
| 31000 | 1.55 | 2.4170e-05 | 0.563 | N/A | N/A |
| 31500 | 1.575 | 2.3753e-05 | 0.5795 | N/A | N/A |
| 32000 | 1.6 | 2.3337e-05 | 0.603 | N/A | N/A |
| 32500 | 1.625 | 2.2920e-05 | 0.5805 | N/A | N/A |
| 33000 | 1.65 | 2.2504e-05 | 0.6108 | N/A | N/A |
| 33500 | 1.675 | 2.2087e-05 | 0.6077 | N/A | N/A |
| 34000 | 1.7 | 2.1670e-05 | 0.5751 | N/A | N/A |
| 34500 | 1.725 | 2.1254e-05 | 0.5833 | N/A | N/A |
| 35000 | 1.75 | 2.0837e-05 | 0.5895 | N/A | N/A |
| 35500 | 1.775 | 2.0420e-05 | 0.5541 | N/A | N/A |
| 36000 | 1.8 | 2.0004e-05 | 0.5423 | N/A | N/A |
| 36500 | 1.825 | 1.9587e-05 | 0.5566 | N/A | N/A |
| 37000 | 1.85 | 1.9171e-05 | 0.5493 | N/A | N/A |
| 37500 | 1.875 | 1.8754e-05 | 0.5602 | N/A | N/A |
| 38000 | 1.9 | 1.8337e-05 | 0.5878 | N/A | N/A |
| 38500 | 1.925 | 1.7921e-05 | 0.5681 | N/A | N/A |
| 39000 | 1.95 | 1.7504e-05 | 0.5464 | N/A | N/A |
| 39500 | 1.975 | 1.7087e-05 | 0.5917 | N/A | N/A |
| 40000 | 2.0 | 1.6671e-05 | 0.5443 | N/A | N/A |
| 40004 | 2.0 | N/A | N/A | 0.8536 | 0.7652 |
| 40500 | 2.025 | 1.6254e-05 | 0.3501 | N/A | N/A |
| 41000 | 2.05 | 1.5838e-05 | 0.3785 | N/A | N/A |
| 41500 | 2.075 | 1.5421e-05 | 0.4034 | N/A | N/A |
| 42000 | 2.1 | 1.5004e-05 | 0.385 | N/A | N/A |
| 42500 | 2.125 | 1.4588e-05 | 0.3758 | N/A | N/A |
| 43000 | 2.15 | 1.4171e-05 | 0.3713 | N/A | N/A |
| 43500 | 2.175 | 1.3754e-05 | 0.413 | N/A | N/A |
| 44000 | 2.2 | 1.3338e-05 | 0.3787 | N/A | N/A |
| 44500 | 2.225 | 1.2921e-05 | 0.3805 | N/A | N/A |
| 45000 | 2.25 | 1.2505e-05 | 0.3757 | N/A | N/A |
| 45500 | 2.275 | 1.2088e-05 | 0.3887 | N/A | N/A |
| 46000 | 2.3 | 1.1671e-05 | 0.3789 | N/A | N/A |
| 46500 | 2.325 | 1.1255e-05 | 0.3742 | N/A | N/A |
| 47000 | 2.35 | 1.0838e-05 | 0.3805 | N/A | N/A |
| 47500 | 2.375 | 1.0421e-05 | 0.3936 | N/A | N/A |
| 48000 | 2.4 | 1.0005e-05 | 0.38 | N/A | N/A |
| 48500 | 2.425 | 9.5882e-06 | 0.3941 | N/A | N/A |
| 49000 | 2.45 | 9.1716e-06 | 0.4054 | N/A | N/A |
| 49500 | 2.475 | 8.7550e-06 | 0.3659 | N/A | N/A |
| 50000 | 2.5 | 8.3383e-06 | 0.3917 | N/A | N/A |
| 50500 | 2.525 | 7.9217e-06 | 0.3876 | N/A | N/A |
| 51000 | 2.55 | 7.5051e-06 | 0.3628 | N/A | N/A |
| 51500 | 2.575 | 7.0885e-06 | 0.3918 | N/A | N/A |
| 52000 | 2.6 | 6.6718e-06 | 0.359 | N/A | N/A |
| 52500 | 2.625 | 6.2552e-06 | 0.3634 | N/A | N/A |
| 53000 | 2.65 | 5.8386e-06 | 0.3737 | N/A | N/A |
| 53500 | 2.675 | 5.4220e-06 | 0.4022 | N/A | N/A |
| 54000 | 2.7 | 5.0053e-06 | 0.3562 | N/A | N/A |
| 54500 | 2.725 | 4.5887e-06 | 0.349 | N/A | N/A |
| 55000 | 2.75 | 4.1721e-06 | 0.3573 | N/A | N/A |
| 55500 | 2.775 | 3.7555e-06 | 0.335 | N/A | N/A |
| 56000 | 2.8 | 3.3388e-06 | 0.3679 | N/A | N/A |
| 56500 | 2.825 | 2.9222e-06 | 0.3266 | N/A | N/A |
| 57000 | 2.85 | 2.5056e-06 | 0.3453 | N/A | N/A |
| 57500 | 2.875 | 2.0890e-06 | 0.3682 | N/A | N/A |
| 58000 | 2.9 | 1.6723e-06 | 0.3417 | N/A | N/A |
| 58500 | 2.925 | 1.2557e-06 | 0.3192 | N/A | N/A |
| 59000 | 2.95 | 8.3908e-07 | 0.3375 | N/A | N/A |
| 59500 | 2.975 | 4.2246e-07 | 0.3669 | N/A | N/A |
| 60000 | 3.0 | 5.8328e-09 | 0.332 | N/A | N/A |
| 60006 | 3.0 | N/A | N/A | 0.9974 | 0.7709 |

</details>


## Framework Versions

- **Transformers:** 5.0.0.dev0
- **PyTorch:** 2.9.1+cu128