| # CLI-LoRA-TinyLLaMA |
|
|
| Fine-tuned **TinyLLaMA-1.1B** model using **QLoRA** on a custom CLI Q&A dataset (Git, Bash, tar/gzip, grep, venv) for the Fenrir Security Internship Task. |
|
|
| --- |
|
|
| ## 🔧 Project Overview |
|
|
| - **Base model**: [TinyLLaMA/TinyLLaMA-1.1B-Chat-v1.0](https://huggingface.co/TinyLLaMA/TinyLLaMA-1.1B-Chat-v1.0) |
| - **Fine-tuning method**: QLoRA |
| - **Library**: `transformers`, `peft`, `trl`, `datasets` |
| - **Training file**: [`training.ipynb`](./training.ipynb) |
|
|
| --- |
|
|
| ## 🧠 Objective |
|
|
| To fine-tune a small language model on real-world command-line Q&A data (no LLM-generated text) and build a command-line chatbot agent capable of providing accurate CLI support. |
|
|
| --- |
|
|
| ## 📂 Files Included |
|
|
| - `training.ipynb`: Full training notebook (cleaned, token-free) |
| - `adapter_config.json`: LoRA adapter configuration |
| - `adapter_model.safetensors`: Trained adapter weights |
| - `eval_logs.json`: Sample evaluation results (accuracy, loss, etc.) |
| - `README.md`: This file |
|
|
| --- |
|
|
| ## 📊 Results |
|
|
| | Metric | Value | |
| |--------------|---------------| |
| | Training Loss| *<your value>* | |
| | Eval Accuracy| *<your value>* | |
| | Epochs | *<your value>* | |
|
|
| --- |
|
|
| ## 📎 Sample Q&A |
|
|
| ```bash |
| Q: How to stash changes in Git? |
| A: Use `git stash` to save your changes temporarily. Retrieve later using `git stash pop`. |
| |
