README.md

metadata

language: en
license: llama3.2
base_model: meta-llama/Llama-3.2-1B
tags:
  - lora
  - qlora
  - question-answering
  - squad
datasets:
  - squad_v2

Llama-3.2-1B + LoRA — SQuAD 2.0 QA

Результаты

Метод	F1	Exact Match
Zero-shot	8.28%	4.0%
Few-shot (5)	0.0%	0.0%
LoRA fine-tuned	61.63%	43.0%

Конфигурация LoRA

• r=16, alpha=32, dropout=0.05
• max_steps=500, lr=2e-4, warmup=50
• 4-bit NF4 QLoRA (unsloth)