metadata
library_name: transformers
tags: []
Model Card for Model ID
LLM trained with LoRA for VK NLP course.
Model Details
Model Description
The model is LLM OuteAI/Lite-Oute-1-300M-Instruct.
LLM trained with LoRA for k, v matricies.
Model Sources
- Pretrained Model: OuteAI/Lite-Oute-1-300M-Instruct
- Train Data: cardiffnlp/tweet_eval
Getting start
from safetensors.torch import load_file
from huggingface_hub import hf_hub_download
device = torch.device("cuda")
model = AutoModelForCausalLM.from_pretrained(f"dmitry315/llm-course-hw3-lora", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(f"dmitry315/llm-course-hw3-lora")
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"
# Примените peft к модели
apply_peft_to_module(model, LinearWithLoRA, r=8, alpha=16, target_submodules=["k_proj", "v_proj"])
model.to(device)
path = hf_hub_download(f"dmitry315/llm-course-hw3-lora", "model.safetensors")
state_dict = load_file(path)
model.load_state_dict(state_dict, strict=False)