README.md
Browse files
README.md
CHANGED
|
@@ -8,8 +8,13 @@ tags:
|
|
| 8 |
- qlora
|
| 9 |
- peft
|
| 10 |
- llama3
|
|
|
|
| 11 |
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
|
| 12 |
pipeline_tag: text-classification
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
---
|
| 14 |
# llama3.1-8b-Instruct-qlora-hatexplain
|
| 15 |
|
|
@@ -50,7 +55,7 @@ python scripts/qlora_inference.py \
|
|
| 50 |
## Training Notes
|
| 51 |
|
| 52 |
- LLaMA-Factory SFT stage with LoRA rank 8, alpha 16, dropout 0.05.
|
| 53 |
-
- Cutoff length
|
| 54 |
- QLoRA (4-bit) backbone for efficient fine-tuning on a single GPU.
|
| 55 |
|
| 56 |
-
Refer to `config/llama31_hatexplain_qlora_sft.yaml` for the full set of hyperparameters.
|
|
|
|
| 8 |
- qlora
|
| 9 |
- peft
|
| 10 |
- llama3
|
| 11 |
+
- llama-factory
|
| 12 |
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
|
| 13 |
pipeline_tag: text-classification
|
| 14 |
+
datasets:
|
| 15 |
+
- Hate-speech-CNERG/hatexplain
|
| 16 |
+
metrics:
|
| 17 |
+
- accuracy
|
| 18 |
---
|
| 19 |
# llama3.1-8b-Instruct-qlora-hatexplain
|
| 20 |
|
|
|
|
| 55 |
## Training Notes
|
| 56 |
|
| 57 |
- LLaMA-Factory SFT stage with LoRA rank 8, alpha 16, dropout 0.05.
|
| 58 |
+
- Cutoff length 1024, cosine scheduler, 3 epochs, learning rate 2e-5.
|
| 59 |
- QLoRA (4-bit) backbone for efficient fine-tuning on a single GPU.
|
| 60 |
|
| 61 |
+
Refer to `config/llama31_hatexplain_qlora_sft.yaml` for the full set of hyperparameters.
|