metadata
base_model: GSAI-ML/LLaDA-8B-Instruct
library_name: peft
pipeline_tag: text-generation
tags:
- peft
- lora
- llada
- math
Math LoRA Adapter for LLaDA-8B-Instruct
This repository contains a PEFT LoRA adapter fine-tuned for math reasoning on top of
GSAI-ML/LLaDA-8B-Instruct.
The repository stores adapter weights only. It does not include the base model weights.
Files
adapter_model.safetensors: LoRA adapter weights.adapter_config.json: PEFT LoRA configuration.tokenizer.json,tokenizer_config.json,special_tokens_map.json: tokenizer files used with the adapter.trainer_state.json,training_args.bin: training metadata kept for traceability.
Optimizer, scheduler, and RNG checkpoint files are intentionally not included.
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model_id = "GSAI-ML/LLaDA-8B-Instruct"
adapter_id = "mousezhang/math-llada8b"
tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_id,
trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_id)
Adapter Details
- Base model:
GSAI-ML/LLaDA-8B-Instruct - Adapter type: LoRA via PEFT
- Task/domain: math reasoning
- PEFT task type:
CAUSAL_LM - LoRA rank:
128 - LoRA alpha:
128 - LoRA dropout:
0.05 - Target modules:
q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
Training and Evaluation
This model card does not report benchmark scores. Evaluation results should be treated as not provided unless published separately by the model author.
Limitations
This adapter inherits the limitations and license terms of the base model. It is intended for research and experimental use, and outputs should be checked carefully before use in high-stakes settings.