---
license: apache-2.0
tags:
  - qlora
  - tinyllama
  - cli
  - command-line
  - fine-tuning
  - low-resource
  - internship
  - fenrir
model_type: TinyLlamaForCausalLM
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
datasets:
  - custom-cli-qa
library_name: peft
pipeline_tag: text-generation
---

# 🔧 CLI LoRA TinyLLaMA Fine-Tuning (Fenrir Internship Project)

🚀 This repository presents a **LoRA fine-tuned version of TinyLLaMA-1.1B-Chat** trained on a custom dataset of CLI Q&A. Developed as part of a 24-hour AI/ML internship task by **Fenrir Security Pvt Ltd**, this lightweight model functions as a domain-specific command-line assistant.

---

## 📁 Dataset

A curated collection of 200+ real-world CLI Q&A pairs covering:

- Git (branching, stash, merge, rebase)
- Bash (variables, loops, file manipulation)
- `grep`, `tar`, `gzip` (command syntax, flags)
- Python environments (`venv`, pip)

Stored in `cli_questions.json`.

---

## ⚙️ Model Details

| Field              | Value                                      |
|-------------------|--------------------------------------------|
| Base Model         | `TinyLlama/TinyLlama-1.1B-Chat-v1.0`       |
| Fine-Tuning Method | QLoRA via `peft`                           |
| Epochs             | 3 (with early stopping)                    |
| Adapter Size       | ~7MB (LoRA weights only)                   |
| Hardware           | Local CPU (low-resource)                   |
| Tokenizer          | Inherited from base model                  |

---

## 📊 Evaluation

| Metric                     | Result         |
|----------------------------|----------------|
| Accuracy on Eval Set       | ~92%           |
| Manual Review              | High relevance |
| Hallucination Rate         | Very low       |
| Inference Time (CPU)       | < 1s / query   |

All results are stored in `eval_results.json`.

---

## 🧠 Files Included

- `adapter_model.safetensors` — fine-tuned LoRA weights
- `adapter_config.json` — LoRA hyperparameters
- `training.ipynb` — complete training notebook
- `agent.py` — CLI interface to test the model
- `cli_questions.json` — training dataset
- `eval_results.json` — eval results
- `requirements.txt` — dependencies

---

## 📦 Inference Example

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")

peft_model = PeftModel.from_pretrained(base_model, "Harish2002/cli-lora-tinyllama")
peft_model.eval()

prompt = "How do I initialize a new Git repository?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = peft_model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))