# CLI-LoRA-TinyLLaMA

Fine-tuned **TinyLLaMA-1.1B** model using **QLoRA** on a custom CLI Q&A dataset (Git, Bash, tar/gzip, grep, venv) for the Fenrir Security Internship Task.

---

## 🔧 Project Overview

- **Base model**: [TinyLLaMA/TinyLLaMA-1.1B-Chat-v1.0](https://huggingface.co/TinyLLaMA/TinyLLaMA-1.1B-Chat-v1.0)
- **Fine-tuning method**: QLoRA
- **Library**: `transformers`, `peft`, `trl`, `datasets`
- **Training file**: [`training.ipynb`](./training.ipynb)

---

## 🧠 Objective

To fine-tune a small language model on real-world command-line Q&A data (no LLM-generated text) and build a command-line chatbot agent capable of providing accurate CLI support.

---

## 📂 Files Included

- `training.ipynb`: Full training notebook (cleaned, token-free)
- `adapter_config.json`: LoRA adapter configuration
- `adapter_model.safetensors`: Trained adapter weights
- `eval_logs.json`: Sample evaluation results (accuracy, loss, etc.)
- `README.md`: This file

---

## 📊 Results

| Metric       | Value         |
|--------------|---------------|
| Training Loss| *<your value>* |
| Eval Accuracy| *<your value>* |
| Epochs       | *<your value>* |

---

## 📎 Sample Q&A

```bash
Q: How to stash changes in Git?
A: Use `git stash` to save your changes temporarily. Retrieve later using `git stash pop`.