mousezhang/math-llada8b

Math LoRA Adapter for LLaDA-8B-Instruct

This repository contains a PEFT LoRA adapter fine-tuned for math reasoning on top of GSAI-ML/LLaDA-8B-Instruct.

The repository stores adapter weights only. It does not include the base model weights.

Files

• adapter_model.safetensors: LoRA adapter weights.
• adapter_config.json: PEFT LoRA configuration.
• tokenizer.json, tokenizer_config.json, special_tokens_map.json: tokenizer files used with the adapter.
• trainer_state.json, training_args.bin: training metadata kept for traceability.

Optimizer, scheduler, and RNG checkpoint files are intentionally not included.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "GSAI-ML/LLaDA-8B-Instruct"
adapter_id = "mousezhang/math-llada8b"

tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_id)

Adapter Details

• Base model: GSAI-ML/LLaDA-8B-Instruct
• Adapter type: LoRA via PEFT
• Task/domain: math reasoning
• PEFT task type: CAUSAL_LM
• LoRA rank: 128
• LoRA alpha: 128
• LoRA dropout: 0.05
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Training and Evaluation

This model card does not report benchmark scores. Evaluation results should be treated as not provided unless published separately by the model author.

Limitations

This adapter inherits the limitations and license terms of the base model. It is intended for research and experimental use, and outputs should be checked carefully before use in high-stakes settings.