README.md

---
library_name: transformers
tags: []
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

LLM trained with LoRA for VK NLP course.

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->
The model is LLM [OuteAI/Lite-Oute-1-300M-Instruct](https://huggingface.co/OuteAI/Lite-Oute-1-300M-Instruct).

LLM trained with LoRA for k, v matricies.

### Model Sources

<!-- Provide the basic links for the model. -->

- **Pretrained Model:** [OuteAI/Lite-Oute-1-300M-Instruct](https://huggingface.co/OuteAI/Lite-Oute-1-300M-Instruct)
- **Train Data:** [cardiffnlp/tweet_eval](https://huggingface.co/datasets/cardiffnlp/tweet_eval)

## Getting start

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

```
from safetensors.torch import load_file
from huggingface_hub import hf_hub_download
device = torch.device("cuda")

model = AutoModelForCausalLM.from_pretrained(f"dmitry315/llm-course-hw3-lora", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(f"dmitry315/llm-course-hw3-lora")
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"

# Примените peft к модели
apply_peft_to_module(model, LinearWithLoRA, r=8, alpha=16, target_submodules=["k_proj", "v_proj"])
model.to(device)

path = hf_hub_download(f"dmitry315/llm-course-hw3-lora", "model.safetensors")
state_dict = load_file(path)

model.load_state_dict(state_dict, strict=False)
```