truettwo commited on
Commit
bddfd97
·
verified ·
1 Parent(s): 47be8d4

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +27 -5
README.md CHANGED
@@ -1,5 +1,27 @@
1
- ---
2
- license: mit
3
- tags:
4
- - unsloth
5
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ license: llama3.2
4
+ base_model: meta-llama/Llama-3.2-1B
5
+ tags:
6
+ - lora
7
+ - qlora
8
+ - question-answering
9
+ - squad
10
+ datasets:
11
+ - squad_v2
12
+ ---
13
+
14
+ # Llama-3.2-1B + LoRA — SQuAD 2.0 QA
15
+
16
+ ## Результаты
17
+
18
+ | Метод | F1 | Exact Match |
19
+ |---|---|---|
20
+ | Zero-shot | 8.28% | 4.0% |
21
+ | Few-shot (5) | 0.0% | 0.0% |
22
+ | **LoRA fine-tuned** | **61.63%** | **43.0%** |
23
+
24
+ ## Конфигурация LoRA
25
+ - `r=16`, `alpha=32`, `dropout=0.05`
26
+ - `max_steps=500`, `lr=2e-4`, `warmup=50`
27
+ - 4-bit NF4 QLoRA (unsloth)