---
base_model: aitf-komdigi/KomdigiITS-8B-DFK-CPT
tags:
- text-generation-inference
- transformers
- unsloth
- mistral3
- trl
license: apache-2.0
language:
- en
datasets:
- Brodip/datasetSFT_tlr
---


## Dataset Details

### Tasks & Capabilities
* **Text Classification DFK** (Disinformasi, Fitnah, Ujaran Kebencian)

### Dataset Splits
* **Train Samples:** 20,790
* **Validation Samples:** 2,599
* **Testing Samples:** 2,599

### Target Labels
1. `DISINFORMASI`
2. `FITNAH`
3. `UJARAN KEBENCIAN`
4. `FAKTA`
5. `BUKAN DFK`

## Training Configurations

### Supervised Fine-Tuning (SFT) Pipeline Parameters
| Parameter | Value |
| :--- | :--- |
| **Max Sequence Length** | 2048 tokens |
| **Batch Size** | 2 |
| **Gradient Accumulation Steps** | 16 |
| **Effective Batch Size** | 32 |
| **Learning Rate (LR)** | 2e-4 |
| **LR Scheduler** | Linear |
| **Optimizer** | AdamW 8-bit |
| **Number of Epochs** | 1 |
| **Warmup Steps** | 20 |
| **Weight Decay** | 0.01 |
| **Max Gradient Norm** | 1.0 |
| **Evaluation Steps** | Every 200 steps |

### LoRA (PEFT) Hyperparameters
| Parameter | Value |
| :--- | :--- |
| **r (Rank)** | 16 |
| **lora_alpha** | 32 |
| **lora_dropout** | 0 |
| **bias** | none |
| **use_rslora** | False |
| **Gradient Checkpointing** | unsloth |
| **Target Modules** | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `down_proj`, `up_proj` |

# Uploaded  model

- **Developed by:** hnuka
- **License:** apache-2.0
- **Finetuned from model :** aitf-komdigi/KomdigiITS-8B-DFK-CPT

This mistral3 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth)

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)