Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -17,41 +17,68 @@ library_name: peft
|
|
| 17 |
pipeline_tag: text-generation
|
| 18 |
---
|
| 19 |
|
| 20 |
-
# CLI LoRA
|
| 21 |
|
| 22 |
-
π This
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## π Dataset
|
| 25 |
-
|
| 26 |
-
-
|
| 27 |
-
|
| 28 |
-
-
|
| 29 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
## βοΈ Model Details
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
-
|
| 37 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
## π Evaluation
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
|
| 44 |
## π§ Files Included
|
| 45 |
-
- `adapter_model.safetensors`
|
| 46 |
-
- `adapter_config.json`
|
| 47 |
-
- `README.md` (you are here)
|
| 48 |
-
- (Optional) `eval_logs.json`, `training.ipynb`
|
| 49 |
|
| 50 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
```python
|
| 53 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 54 |
-
from peft import PeftModel
|
| 55 |
|
| 56 |
base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
|
| 57 |
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
|
|
|
|
| 17 |
pipeline_tag: text-generation
|
| 18 |
---
|
| 19 |
|
| 20 |
+
# π§ CLI LoRA TinyLLaMA Fine-Tuning (Fenrir Internship Project)
|
| 21 |
|
| 22 |
+
π This repository presents a **LoRA fine-tuned version of TinyLLaMA-1.1B-Chat** trained on a custom dataset of CLI Q&A. Developed as part of a 24-hour AI/ML internship task by **Fenrir Security Pvt Ltd**, this lightweight model functions as a domain-specific command-line assistant.
|
| 23 |
+
|
| 24 |
+
---
|
| 25 |
|
| 26 |
## π Dataset
|
| 27 |
+
|
| 28 |
+
A curated collection of 200+ real-world CLI Q&A pairs covering:
|
| 29 |
+
|
| 30 |
+
- Git (branching, stash, merge, rebase)
|
| 31 |
+
- Bash (variables, loops, file manipulation)
|
| 32 |
+
- `grep`, `tar`, `gzip` (command syntax, flags)
|
| 33 |
+
- Python environments (`venv`, pip)
|
| 34 |
+
|
| 35 |
+
Stored in `cli_questions.json`.
|
| 36 |
+
|
| 37 |
+
---
|
| 38 |
|
| 39 |
## βοΈ Model Details
|
| 40 |
+
|
| 41 |
+
| Field | Value |
|
| 42 |
+
|-------------------|--------------------------------------------|
|
| 43 |
+
| Base Model | `TinyLlama/TinyLlama-1.1B-Chat-v1.0` |
|
| 44 |
+
| Fine-Tuning Method | QLoRA via `peft` |
|
| 45 |
+
| Epochs | 3 (with early stopping) |
|
| 46 |
+
| Adapter Size | ~7MB (LoRA weights only) |
|
| 47 |
+
| Hardware | Local CPU (low-resource) |
|
| 48 |
+
| Tokenizer | Inherited from base model |
|
| 49 |
+
|
| 50 |
+
---
|
| 51 |
|
| 52 |
## π Evaluation
|
| 53 |
+
|
| 54 |
+
| Metric | Result |
|
| 55 |
+
|----------------------------|----------------|
|
| 56 |
+
| Accuracy on Eval Set | ~92% |
|
| 57 |
+
| Manual Review | High relevance |
|
| 58 |
+
| Hallucination Rate | Very low |
|
| 59 |
+
| Inference Time (CPU) | < 1s / query |
|
| 60 |
+
|
| 61 |
+
All results are stored in `eval_results.json`.
|
| 62 |
+
|
| 63 |
+
---
|
| 64 |
|
| 65 |
## π§ Files Included
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
|
| 67 |
+
- `adapter_model.safetensors` β fine-tuned LoRA weights
|
| 68 |
+
- `adapter_config.json` β LoRA hyperparameters
|
| 69 |
+
- `training.ipynb` β complete training notebook
|
| 70 |
+
- `agent.py` β CLI interface to test the model
|
| 71 |
+
- `cli_questions.json` β training dataset
|
| 72 |
+
- `eval_results.json` β eval results
|
| 73 |
+
- `requirements.txt` β dependencies
|
| 74 |
+
|
| 75 |
+
---
|
| 76 |
+
|
| 77 |
+
## π¦ Inference Example
|
| 78 |
|
| 79 |
```python
|
| 80 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 81 |
+
from peft import PeftModel
|
| 82 |
|
| 83 |
base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
|
| 84 |
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
|