metadata
base_model: GSAI-ML/LLaDA-1.5
tags:
- lora
- peft
- countdown
COUNTDOWN LoRA Adapter for LLaDA-1.5
This is a LoRA adapter fine-tuned on GSAI-ML/LLaDA-1.5 for the countdown task.
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
# 加载基座模型
base_model = AutoModelForCausalLM.from_pretrained("GSAI-ML/LLaDA-1.5")
tokenizer = AutoTokenizer.from_pretrained("GSAI-ML/LLaDA-1.5")
# 加载 adapter
model = PeftModel.from_pretrained(base_model, "mousezhang/countdown-llada1.5")
Training Config
- Base Model: GSAI-ML/LLaDA-1.5
- Task: countdown
- LoRA Rank: 128
- LoRA Alpha: 64
- Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj