# lora-tinyllama

## Overview
`lora-tinyllama` is a fine-tuned version of the `tinyllama-1.1b` model, created using **LoRA (Low-Rank Adaptation)**. This model specializes in adapting the `tinyllama-1.1b` base for specific tasks with minimal computational overhead.

### Key Features
- **Model Size**: ~90MB (LoRA adapter weights only).
- **Efficiency**: Keeps the base model frozen and adds small trainable layers.
- **Flexibility**: Requires the original `tinyllama-1.1b` base model for usage.
- **Purpose**: Designed for specialized NLP tasks, leveraging the compact and powerful nature of the base model.

---

## Usage Instructions

### Prerequisites
Before using `lora-tinyllama`, ensure you have:
1. The base model: `tinyllama-1.1b`.
2. The fine-tuned LoRA weights: `lora-tinyllama`.

---

### Loading the Model
Here’s how to load and use `lora-tinyllama` with the base model:

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Step 1: Load the base model
base_model_path = "path/to/tinyllama-1.1b"
base_model = AutoModelForCausalLM.from_pretrained(base_model_path)

# Step 2: Load the LoRA weights
lora_model_path = "path/to/lora-tinyllama"
lora_model = PeftModel.from_pretrained(base_model, lora_model_path)

# Step 3: Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_path)

# Step 4: Use the model for inference
inputs = tokenizer("Hello, world!", return_tensors="pt")
outputs = lora_model.generate(inputs["input_ids"])
print(tokenizer.decode(outputs[0]))