shuvo-halder/tinyllama-guanaco-trained

TinyLlama-1.1B Fine-Tuned (LoRA)

This model is a fine-tuned version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 on the mlabonne/guanaco-llama2-1k dataset using LoRA (Low-Rank Adaptation).

Model Details

• Developed by: [shuvo-halder00]
• Model type: Causal Language Model (Fine-tuned adapter)
• Language(s): English
• License: Apache-2.0
• Finetuned from model: TinyLlama/TinyLlama-1.1B-Chat-v1.0

Uses

Direct Use

This model is intended for general conversational tasks, instruction following, and answering user queries based on the instruction fine-tuning dataset.

Limitations & Biases

• The model is only 1.1B parameters, so it may occasionally generate incorrect facts (hallucinations).
• It inherits the biases present in the base model and the Guanaco dataset.
• Not recommended for critical production tasks (medical, legal, or high-stakes financial advice).

How to Get Started with the Model

Use the code below to load and run this model using 🤗 Transformers and PEFT:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
adapter_id = "[shuvo-halder00]/[tinyllama-guanaco-trained]"

# Load base model
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load fine-tuned LoRA adapter
model = PeftModel.from_pretrained(model, adapter_id)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)

prompt = "### Human: What are the benefits of learning programming?\n### Assistant:"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

outputs = model.generate(
    **inputs, 
    max_new_tokens=150, 
    temperature=0.7, 
    do_sample=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))