| # lora-tinyllama |
|
|
| ## Overview |
| `lora-tinyllama` is a fine-tuned version of the `tinyllama-1.1b` model, created using **LoRA (Low-Rank Adaptation)**. This model specializes in adapting the `tinyllama-1.1b` base for specific tasks with minimal computational overhead. |
|
|
| ### Key Features |
| - **Model Size**: ~90MB (LoRA adapter weights only). |
| - **Efficiency**: Keeps the base model frozen and adds small trainable layers. |
| - **Flexibility**: Requires the original `tinyllama-1.1b` base model for usage. |
| - **Purpose**: Designed for specialized NLP tasks, leveraging the compact and powerful nature of the base model. |
|
|
| --- |
|
|
| ## Usage Instructions |
|
|
| ### Prerequisites |
| Before using `lora-tinyllama`, ensure you have: |
| 1. The base model: `tinyllama-1.1b`. |
| 2. The fine-tuned LoRA weights: `lora-tinyllama`. |
|
|
| --- |
|
|
| ### Loading the Model |
| Here’s how to load and use `lora-tinyllama` with the base model: |
|
|
| ```python |
| from peft import PeftModel |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| # Step 1: Load the base model |
| base_model_path = "path/to/tinyllama-1.1b" |
| base_model = AutoModelForCausalLM.from_pretrained(base_model_path) |
| |
| # Step 2: Load the LoRA weights |
| lora_model_path = "path/to/lora-tinyllama" |
| lora_model = PeftModel.from_pretrained(base_model, lora_model_path) |
| |
| # Step 3: Load the tokenizer |
| tokenizer = AutoTokenizer.from_pretrained(base_model_path) |
| |
| # Step 4: Use the model for inference |
| inputs = tokenizer("Hello, world!", return_tensors="pt") |
| outputs = lora_model.generate(inputs["input_ids"]) |
| print(tokenizer.decode(outputs[0])) |
| |
