File size: 1,285 Bytes
f87f77c 6817222 f87f77c 6817222 f87f77c 6817222 f87f77c 6817222 f87f77c 6817222 f87f77c 6817222 f87f77c 6817222 ba90a37 6817222 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
---
library_name: transformers
tags: []
---
# Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->
LLM trained with QLoRA for VK NLP course.
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
The model is LLM [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0).
LLM trained with QLoRA for k, v matricies with PEFT lib.
### Model Sources
<!-- Provide the basic links for the model. -->
- **Pretrained Model:** [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)
- **Train Data:** [cardiffnlp/tweet_eval](https://huggingface.co/datasets/cardiffnlp/tweet_eval)
## Getting start
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
```
from safetensors.torch import load_file
from huggingface_hub import hf_hub_download
device = torch.device("cuda")
model = AutoModelForCausalLM.from_pretrained(f"{REPO_NAME}-tinyllama-qlora", device_map="auto")
model.to(device)
tokenizer = AutoTokenizer.from_pretrained(f"{REPO_NAME}-tinyllama-qlora")
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"
``` |