🦙 LLaMA 3.2 1B + SAMSum LoRA Adapter
This is a LoRA adapter trained on the SAMSum dataset for dialogue summarization using meta-llama/Llama-3.2-1B as the base model.
🛠️ Training Details
- Base model:
meta-llama/Llama-3.2-1B - LoRA config: r=8, alpha=32, dropout=0.01
- Epochs: 1
- Batch size: 1 (accumulation: 4)
- Precision: 8-bit (bitsandbytes)
- Device: Google Colab (T4 16GB)
🔧 Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B", device_map="auto")
model = PeftModel.from_pretrained(base, "aptro/Llama-3.2-1B-samsun")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B")
License
This model follows the LLaMA 2 Community License Agreement.
- Downloads last month
- 3
Model tree for aptro/Llama-3.2-1B-samsun
Base model
meta-llama/Llama-3.2-1B