README.md

---
language:
- afr
- als
- amh
- arb
- ars
- ary
- arz
- asm
- azj
- bel
- ben
- bew
- bos
- bul
- cat
- ces
- ckb
- cmn
- cym
- dan
- deu
- div
- ekk
- ell
- eng
- epo
- eus
- fao
- fas
- fil
- fin
- fra
- fry
- gle
- glg
- guj
- hau
- heb
- hin
- hrv
- hun
- hye
- ind
- isl
- ita
- jpn
- kan
- kat
- kaz
- khk
- khm
- kin
- kir
- kmr
- kor
- lao
- lat
- lit
- ltz
- lvs
- mal
- mar
- mkd
- mlt
- mya
- nld
- nno
- nob
- npi
- nrm
- ory
- pan
- pbt
- plt
- pol
- por
- ron
- rus
- sin
- slk
- slv
- snd
- som
- spa
- srp
- swe
- swh
- tam
- tel
- tgk
- tha
- tur
- ukr
- urd
- uzn
- vie
- xho
- yue
- zsm
license: mit
base_model:
- intfloat/multilingual-e5-small
datasets:
- agentlans/multilingual-document-classification
metrics:
- f1
- loss
model-index:
- name: multilingual-e5-small-doc-type-v2-classifier
  results:
  - task:
      type: text-classification
      name: Text Classification
    metrics:
    - type: f1
      value: 0.809
      name: Evaluation F1
    - type: loss
      value: 0.8624
      name: Evaluation Loss
---
# multilingual-e5-small Document Type V2 Classifier

A fine-tuned version of the **bert** architecture (`BertForSequenceClassification`) optimized for the `text-classification` task.

- **Model type:** bert
- **Problem Type:** single_label_classification
- **Number of Labels:** 25
- **Vocabulary Size:** 250037
- **License:** MIT

## Use

To get started with this model in Python using the Hugging Face Transformers library, run the following code:

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "agentlans/multilingual-e5-small-doc-type-v2-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Replace this with your input text."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
predicted_class_name = model.config.id2label[predicted_class_id]

print(f"Predicted Class ID: {predicted_class_id}")
print(f"Predicted Class Name: {predicted_class_name}")
```

## Intended Uses & Limitations

### Intended Use
This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:

| Label ID | Label Name |
|---|---|
| 0 | About (Org.) |
| 1 | About (Personal) |
| 2 | Academic Writing |
| 3 | Audio Transcript |
| 4 | Comment Section |
| 5 | Content Listing |
| 6 | Creative Writing |
| 7 | Customer Support |
| 8 | Documentation |
| 9 | FAQ |
| 10 | Knowledge Article |
| 11 | Legal Notices |
| 12 | Listicle |
| 13 | News (Org.) |
| 14 | News Article |
| 15 | Nonfiction Writing |
| 16 | Other/Unclassified |
| 17 | Personal Blog |
| 18 | Product Page |
| 19 | Q&A Forum |
| 20 | Spam / Ads |
| 21 | Structured Data |
| 22 | Truncated |
| 23 | Tutorial |
| 24 | User Review |

## Training Details

### Hyperparameters
The following hyperparameters were used during fine-tuning:
- **Learning Rate:** 5e-05
- **Train Batch Size:** 8
- **Eval Batch Size:** 8
- **Optimizer:** OptimizerNames.ADAMW_TORCH_FUSED
- **Number of Epochs:** 3.0
- **Mixed Precision:** BF16

<details>
<summary><b>Show Advanced Training Configuration</b></summary>

#### Optimization & Regularization
- **Gradient Accumulation Steps:** 1
- **Learning Rate Scheduler:** SchedulerType.LINEAR
- **Warmup Steps:** 0
- **Warmup Ratio:** None
- **Weight Decay:** 0.0
- **Max Gradient Norm:** 1.0

#### Hardware & Reproducibility
- **Number of GPUs:** 1
- **Seed:** 42

</details>

## Training Results & Evaluation

During fine-tuning, the model achieved the following results on the evaluation set:

| Metric | Value |
|---|---|
| **Train Loss** | 0.5709 |
| **Validation Loss** | 0.8624 |
| **Validation F1 Score** | 0.809 |
| **Total FLOPs** | 7.9082e+15 |

### Speed Performance
- **Training Runtime:** 1693.148 seconds
- **Train Samples per Second:** 283.503
- **Evaluation Runtime:** 11.4879 seconds
- **Eval Samples per Second:** 1741.655


<details>
<summary><b>Show Detailed Training Logs</b></summary>

### Training Logs History

| Step | Epoch | Learning Rate | Training Loss | Validation Loss | Validation F1 |
|---|---|---|---|---|---|
| 500 | 0.025 | 4.9584e-05 | 1.8537 | N/A | N/A |
| 1000 | 0.05 | 4.9168e-05 | 1.3289 | N/A | N/A |
| 1500 | 0.075 | 4.8751e-05 | 1.1698 | N/A | N/A |
| 2000 | 0.1 | 4.8334e-05 | 1.0996 | N/A | N/A |
| 2500 | 0.125 | 4.7918e-05 | 1.0552 | N/A | N/A |
| 3000 | 0.15 | 4.7501e-05 | 1.0462 | N/A | N/A |
| 3500 | 0.175 | 4.7084e-05 | 1.0004 | N/A | N/A |
| 4000 | 0.2 | 4.6668e-05 | 0.9812 | N/A | N/A |
| 4500 | 0.225 | 4.6251e-05 | 0.9245 | N/A | N/A |
| 5000 | 0.25 | 4.5834e-05 | 0.9282 | N/A | N/A |
| 5500 | 0.275 | 4.5418e-05 | 0.9167 | N/A | N/A |
| 6000 | 0.3 | 4.5001e-05 | 0.8886 | N/A | N/A |
| 6500 | 0.325 | 4.4584e-05 | 0.8826 | N/A | N/A |
| 7000 | 0.35 | 4.4168e-05 | 0.8443 | N/A | N/A |
| 7500 | 0.375 | 4.3751e-05 | 0.8374 | N/A | N/A |
| 8000 | 0.4 | 4.3334e-05 | 0.8271 | N/A | N/A |
| 8500 | 0.425 | 4.2918e-05 | 0.8306 | N/A | N/A |
| 9000 | 0.45 | 4.2501e-05 | 0.8561 | N/A | N/A |
| 9500 | 0.475 | 4.2085e-05 | 0.7851 | N/A | N/A |
| 10000 | 0.5 | 4.1668e-05 | 0.7841 | N/A | N/A |
| 10500 | 0.525 | 4.1251e-05 | 0.7678 | N/A | N/A |
| 11000 | 0.55 | 4.0835e-05 | 0.7538 | N/A | N/A |
| 11500 | 0.575 | 4.0418e-05 | 0.735 | N/A | N/A |
| 12000 | 0.6 | 4.0001e-05 | 0.774 | N/A | N/A |
| 12500 | 0.625 | 3.9585e-05 | 0.7368 | N/A | N/A |
| 13000 | 0.65 | 3.9168e-05 | 0.7435 | N/A | N/A |
| 13500 | 0.675 | 3.8751e-05 | 0.7035 | N/A | N/A |
| 14000 | 0.7 | 3.8335e-05 | 0.7552 | N/A | N/A |
| 14500 | 0.725 | 3.7918e-05 | 0.7443 | N/A | N/A |
| 15000 | 0.75 | 3.7501e-05 | 0.7461 | N/A | N/A |
| 15500 | 0.775 | 3.7085e-05 | 0.7352 | N/A | N/A |
| 16000 | 0.8 | 3.6668e-05 | 0.6946 | N/A | N/A |
| 16500 | 0.825 | 3.6252e-05 | 0.6939 | N/A | N/A |
| 17000 | 0.85 | 3.5835e-05 | 0.7509 | N/A | N/A |
| 17500 | 0.875 | 3.5418e-05 | 0.6992 | N/A | N/A |
| 18000 | 0.9 | 3.5002e-05 | 0.7043 | N/A | N/A |
| 18500 | 0.925 | 3.4585e-05 | 0.6977 | N/A | N/A |
| 19000 | 0.95 | 3.4168e-05 | 0.6952 | N/A | N/A |
| 19500 | 0.975 | 3.3752e-05 | 0.708 | N/A | N/A |
| 20000 | 1.0 | 3.3335e-05 | 0.6695 | N/A | N/A |
| 20001 | 1.0 | N/A | N/A | 0.6958 | 0.7876 |
| 20500 | 1.025 | 3.2918e-05 | 0.5363 | N/A | N/A |
| 21000 | 1.05 | 3.2502e-05 | 0.547 | N/A | N/A |
| 21500 | 1.075 | 3.2085e-05 | 0.5733 | N/A | N/A |
| 22000 | 1.1 | 3.1668e-05 | 0.5454 | N/A | N/A |
| 22500 | 1.125 | 3.1252e-05 | 0.5235 | N/A | N/A |
| 23000 | 1.15 | 3.0835e-05 | 0.5291 | N/A | N/A |
| 23500 | 1.175 | 3.0418e-05 | 0.5537 | N/A | N/A |
| 24000 | 1.2 | 3.0002e-05 | 0.555 | N/A | N/A |
| 24500 | 1.225 | 2.9585e-05 | 0.5338 | N/A | N/A |
| 25000 | 1.25 | 2.9169e-05 | 0.5615 | N/A | N/A |
| 25500 | 1.275 | 2.8752e-05 | 0.5155 | N/A | N/A |
| 26000 | 1.3 | 2.8335e-05 | 0.5353 | N/A | N/A |
| 26500 | 1.325 | 2.7919e-05 | 0.5317 | N/A | N/A |
| 27000 | 1.35 | 2.7502e-05 | 0.5429 | N/A | N/A |
| 27500 | 1.375 | 2.7085e-05 | 0.5311 | N/A | N/A |
| 28000 | 1.4 | 2.6669e-05 | 0.5345 | N/A | N/A |
| 28500 | 1.425 | 2.6252e-05 | 0.5287 | N/A | N/A |
| 29000 | 1.45 | 2.5835e-05 | 0.5204 | N/A | N/A |
| 29500 | 1.475 | 2.5419e-05 | 0.5121 | N/A | N/A |
| 30000 | 1.5 | 2.5002e-05 | 0.52 | N/A | N/A |
| 30500 | 1.525 | 2.4585e-05 | 0.5094 | N/A | N/A |
| 31000 | 1.55 | 2.4169e-05 | 0.5169 | N/A | N/A |
| 31500 | 1.575 | 2.3752e-05 | 0.5226 | N/A | N/A |
| 32000 | 1.6 | 2.3335e-05 | 0.5281 | N/A | N/A |
| 32500 | 1.625 | 2.2919e-05 | 0.5246 | N/A | N/A |
| 33000 | 1.65 | 2.2502e-05 | 0.532 | N/A | N/A |
| 33500 | 1.675 | 2.2086e-05 | 0.5068 | N/A | N/A |
| 34000 | 1.7 | 2.1669e-05 | 0.4971 | N/A | N/A |
| 34500 | 1.725 | 2.1252e-05 | 0.5122 | N/A | N/A |
| 35000 | 1.75 | 2.0836e-05 | 0.489 | N/A | N/A |
| 35500 | 1.775 | 2.0419e-05 | 0.479 | N/A | N/A |
| 36000 | 1.8 | 2.0002e-05 | 0.4919 | N/A | N/A |
| 36500 | 1.825 | 1.9586e-05 | 0.4974 | N/A | N/A |
| 37000 | 1.85 | 1.9169e-05 | 0.5045 | N/A | N/A |
| 37500 | 1.875 | 1.8752e-05 | 0.525 | N/A | N/A |
| 38000 | 1.9 | 1.8336e-05 | 0.4748 | N/A | N/A |
| 38500 | 1.925 | 1.7919e-05 | 0.4831 | N/A | N/A |
| 39000 | 1.95 | 1.7502e-05 | 0.5091 | N/A | N/A |
| 39500 | 1.975 | 1.7086e-05 | 0.4821 | N/A | N/A |
| 40000 | 2.0 | 1.6669e-05 | 0.4862 | N/A | N/A |
| 40002 | 2.0 | N/A | N/A | 0.7491 | 0.797 |
| 40500 | 2.025 | 1.6253e-05 | 0.357 | N/A | N/A |
| 41000 | 2.05 | 1.5836e-05 | 0.333 | N/A | N/A |
| 41500 | 2.075 | 1.5419e-05 | 0.374 | N/A | N/A |
| 42000 | 2.1 | 1.5003e-05 | 0.3698 | N/A | N/A |
| 42500 | 2.125 | 1.4586e-05 | 0.3759 | N/A | N/A |
| 43000 | 2.15 | 1.4169e-05 | 0.3543 | N/A | N/A |
| 43500 | 2.175 | 1.3753e-05 | 0.3695 | N/A | N/A |
| 44000 | 2.2 | 1.3336e-05 | 0.3385 | N/A | N/A |
| 44500 | 2.225 | 1.2919e-05 | 0.3583 | N/A | N/A |
| 45000 | 2.25 | 1.2503e-05 | 0.3445 | N/A | N/A |
| 45500 | 2.275 | 1.2086e-05 | 0.3575 | N/A | N/A |
| 46000 | 2.3 | 1.1669e-05 | 0.3382 | N/A | N/A |
| 46500 | 2.325 | 1.1253e-05 | 0.3732 | N/A | N/A |
| 47000 | 2.35 | 1.0836e-05 | 0.3454 | N/A | N/A |
| 47500 | 2.375 | 1.0419e-05 | 0.3563 | N/A | N/A |
| 48000 | 2.4 | 1.0003e-05 | 0.3302 | N/A | N/A |
| 48500 | 2.425 | 9.5862e-06 | 0.3421 | N/A | N/A |
| 49000 | 2.45 | 9.1695e-06 | 0.3119 | N/A | N/A |
| 49500 | 2.475 | 8.7529e-06 | 0.3578 | N/A | N/A |
| 50000 | 2.5 | 8.3362e-06 | 0.3584 | N/A | N/A |
| 50500 | 2.525 | 7.9196e-06 | 0.3142 | N/A | N/A |
| 51000 | 2.55 | 7.5030e-06 | 0.3124 | N/A | N/A |
| 51500 | 2.575 | 7.0863e-06 | 0.3262 | N/A | N/A |
| 52000 | 2.6 | 6.6697e-06 | 0.3072 | N/A | N/A |
| 52500 | 2.625 | 6.2530e-06 | 0.3274 | N/A | N/A |
| 53000 | 2.65 | 5.8364e-06 | 0.3131 | N/A | N/A |
| 53500 | 2.675 | 5.4197e-06 | 0.3281 | N/A | N/A |
| 54000 | 2.7 | 5.0031e-06 | 0.3108 | N/A | N/A |
| 54500 | 2.725 | 4.5864e-06 | 0.3189 | N/A | N/A |
| 55000 | 2.75 | 4.1698e-06 | 0.3367 | N/A | N/A |
| 55500 | 2.775 | 3.7531e-06 | 0.2969 | N/A | N/A |
| 56000 | 2.8 | 3.3365e-06 | 0.3332 | N/A | N/A |
| 56500 | 2.825 | 2.9199e-06 | 0.3197 | N/A | N/A |
| 57000 | 2.85 | 2.5032e-06 | 0.312 | N/A | N/A |
| 57500 | 2.875 | 2.0866e-06 | 0.3275 | N/A | N/A |
| 58000 | 2.9 | 1.6699e-06 | 0.2933 | N/A | N/A |
| 58500 | 2.925 | 1.2533e-06 | 0.3123 | N/A | N/A |
| 59000 | 2.95 | 8.3662e-07 | 0.3045 | N/A | N/A |
| 59500 | 2.975 | 4.1998e-07 | 0.2928 | N/A | N/A |
| 60000 | 3.0 | 3.3332e-09 | 0.3199 | N/A | N/A |
| 60003 | 3.0 | N/A | N/A | 0.8624 | 0.809 |

</details>


## Framework Versions

- **Transformers:** 5.0.0.dev0
- **PyTorch:** 2.9.1+cu128