Ramikan-BR/tinyllama-coder-py-4bit-v5

Uploaded model

• Developed by: Ramikan-BR
• License: apache-2.0
• Finetuned from model : unsloth/tinyllama-chat-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

---

Inference tests after refinement

Test 1: Continuing the Fibonacci sequence

alpaca_prompt = "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n### Input:\nContinue the fibonnaci sequence.\n\n### Output:"

from unsloth import FastLanguageModel
FastLanguageModel.for_inference(model) # Enable native 2x faster inference

inputs = tokenizer([alpaca_prompt.format("Continue the fibonnaci sequence.", "1, 1, 2, 3, 5, 8", "")], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)
print(tokenizer.batch_decode(outputs))
Output:
['<s> Below is an instruction that describes a task. Write a response that appropriately completes the request.\n### Input:\nContinue the fibonnaci sequence.\n\n### Output:\n1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89</s>']

**Test 2: Famous tall tower in Paris**
alpaca_prompt = "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n### Input:\nWhat is a famous tall tower in Paris?\n\n### Output:"

from unsloth import FastLanguageModel
FastLanguageModel.for_inference(model) # Enable native 2x faster inference

inputs = tokenizer([alpaca_prompt.format("What is a famous tall tower in Paris?", "", "")], return_tensors="pt").to("cuda")
from transformers import TextStreamer
text_streamer = TextStreamer(tokenizer)
_ = model.generate(**inputs, streamer=text_streamer, max_new_tokens=64)
Output:
Eiffel Tower, located in Paris, is a famous tall tower that stands at 320 meters (98 feet) tall. It was built in 189002 as a symbol of the city's modernization and progress, and it remains an iconic landmark to this
For the first time, the AI answered both questions correctly, despite the response about the Eiffel Tower containing errors about the year and not finishing the response. I will continue refining the AI with the data-oss_instruct-decontaminated_python.jsonl dataset. This version of the dataset only contains Python code, and since I can only train on the free Colab GPU, I was forced to split the dataset into 10 parts and refine the AI for two epochs with each part (up to this point, we are on the fifth part of the dataset)... Thanks to the Unsloth team, without you, I wouldn't have even achieved any relevant training on an AI since I don't have a GPU!