hailsbop/mistral7BXHV2L

Mistral + LoRA Fine-Tuning

Lightweight fine-tuning setup for adapting a Mistral-architecture model using LoRA.
The project keeps everything modular: base model, adapters, training config, and generation settings.

Features

• LoRA adapters applied to attention and MLP blocks
• Minimal training overhead with notebook-friendly resource use
• Drop-in loading for inference or further fine-tuning

Requirements

pip install transformers accelerate peft datasets bitsandbytes

Training

Start training with:

accelerate launch train.py --config config.yaml

Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "mistral-base"
lora = "./lora-out"

tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, lora)

prompt = "Your prompt here"
out = model.generate(
    **tok(prompt, return_tensors="pt").to(model.device),
    max_new_tokens=256
)
print(tok.decode(out[0]))

Files

• config.yaml – training + LoRA configuration
• train.py – simple training harness
• lora-out/ – adapter weights after training
• generation_config.json – generation defaults
• tokenizer_config.json / special_tokens_map.json – tokenizer metadata

License

MIT