| | --- |
| | library_name: transformers |
| | base_model: unsloth/tinyllama-bnb-4bit |
| | license: mit |
| | datasets: |
| | - yahma/alpaca-cleaned |
| | language: |
| | - en |
| | pipeline_tag: text-generation |
| | tags: |
| | - Instruct |
| | - TinyLlama |
| | --- |
| | |
| | # Steps to try the model: |
| |
|
| | ### prompt Template |
| | ```python |
| | alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. |
| | |
| | ### Instruction: |
| | {} |
| | |
| | ### Input: |
| | {} |
| | |
| | ### Response: |
| | {}""" |
| | ``` |
| | ### load the model |
| |
|
| | ```python |
| | from transformers import AutoTokenizer, AutoModelForCausalLM |
| | |
| | tokenizer = AutoTokenizer.from_pretrained("damerajee/tinyllama-sft-small-v2") |
| | model = AutoModelForCausalLM.from_pretrained("damerajee/tinyllama-sft-small-v2") |
| | ``` |
| | ### Inference |
| |
|
| | ```python |
| | inputs = tokenizer( |
| | [ |
| | alpaca_prompt.format( |
| | "best places to visit in india", # instruction |
| | "", # input |
| | "", # output |
| | ) |
| | ]*1, return_tensors = "pt") |
| | |
| | outputs = model.generate(**inputs, max_new_tokens = 128, use_cache = True) |
| | tokenizer.batch_decode(outputs) |
| | ``` |
| |
|
| |
|
| |
|
| | # Model Information |
| | The base model [unsloth/tinyllama-bnb-4bit](https://huggingface.co/unsloth/tinyllama-bnb-4bit) was Instruct finetuned using [Unsloth](https://github.com/unslothai/unsloth) |
| |
|
| | # Model Limitations |
| | The model was trained on a very small dataset so it might not be as good ,will be training on larger dataset soon |
| | # Training Details |
| | The model was trained for 1 epoch on a free goggle colab which took about 1 hour and 30 mins approximately |
