| language: en | |
| license: llama3.2 | |
| base_model: meta-llama/Llama-3.2-1B | |
| tags: | |
| - lora | |
| - qlora | |
| - question-answering | |
| - squad | |
| datasets: | |
| - squad_v2 | |
| # Llama-3.2-1B + LoRA — SQuAD 2.0 QA | |
| ## Результаты | |
| | Метод | F1 | Exact Match | | |
| |---|---|---| | |
| | Zero-shot | 8.28% | 4.0% | | |
| | Few-shot (5) | 0.0% | 0.0% | | |
| | **LoRA fine-tuned** | **61.63%** | **43.0%** | | |
| ## Конфигурация LoRA | |
| - `r=16`, `alpha=32`, `dropout=0.05` | |
| - `max_steps=500`, `lr=2e-4`, `warmup=50` | |
| - 4-bit NF4 QLoRA (unsloth) |