Update README.md

README.md

@@ -1,3 +1,69 @@
----
-license: mit
----
+---
+license: mit
+tags:
+  - tinyllama
+  - sciq
+  - multiple-choice
+  - peft
+  - lora
+  - 4bit
+  - quantization
+  - instruction-tuning
+datasets:
+  - allenai/sciq
+language:
+  - en
+library_name: transformers
+pipeline_tag: text-generation
+---
+
+# 🧠 TinyLLaMA-1.1B LoRA Fine-tuned on SciQ Dataset
+
+This is a **TinyLLaMA-1.1B** model fine-tuned using **LoRA (Low-Rank Adaptation)** on the [SciQ](https://huggingface.co/datasets/allenai/sciq) multiple-choice question answering dataset. It uses **4-bit quantization** via `bitsandbytes` to reduce memory usage and improve inference efficiency.
+
+## 🧪 Use Cases
+
+This model is suitable for:
+
+- Educational QA bots
+- MCQ-style reasoning
+- Lightweight inference on constrained hardware (e.g., GPUs with <8GB VRAM)
+
+## 🛠️ Training Details
+
+- Base Model: `TinyLlama/TinyLlama-1.1B-Chat-v1.0`
+- Dataset: `allenai/sciq` (Science QA)
+- Method: Parameter-Efficient Fine-Tuning using LoRA
+- Quantization: 4-bit using `bitsandbytes`
+- Framework: 🤗 Transformers + PEFT + Datasets
+
+## 🧬 Model Architecture
+
+- Model: Causal Language Model
+- Fine-tuned layers: `q_proj`, `v_proj` (via LoRA)
+- Quantization: 4-bit (bnb config)
+
+## 📊 Evaluation
+
+- Accuracy: **100%** on a 1000-sample SciQ subset
+- Eval Loss: ~0.19
+
+## 💡 How to Use
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("your-username/tinyllama-sciq-lora")
+tokenizer = AutoTokenizer.from_pretrained("your-username/tinyllama-sciq-lora")
+
+prompt = """Question: What is the boiling point of water?\nChoices:\nA. 50°C\nB. 75°C\nC. 90°C\nD. 100°C\nAnswer:"""
+inputs = tokenizer(prompt, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=20)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+
+## 🔐 License
+This model is released under the MIT License.
+
+## 🙌 Credits
+FineTuned By - [Uditanshu Pandey](https://huggingface.co/TechyCode) 
+Based on - [TinyLLaMA-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)