Updated README.md File

README.md

@@ -1,21 +1,21 @@
 ---
-base_model: unsloth/tinyllama-chat-bnb-4bit
-library_name: peft
+base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
+library_name: llama.cpp
 pipeline_tag: text-generation
 tags:
-- base_model:adapter:unsloth/tinyllama-chat-bnb-4bit
-- lora
-- sft
-- transformers
-- trl
-- unsloth
+- tinyllama
+- email-reply
+- gguf
+- ollama
+- local-ai
 license: mit
 language:
 - en
 ---
-# TinyLlama Email Reply Generator (LoRA)
 
-A **LoRA fine-tuned TinyLlama model** for generating professional email replies from incoming emails.
+# TinyLlama Email Reply Generator
+
+A **fine-tuned TinyLlama model** for generating professional email replies from incoming emails.
 
 The adapter was trained on the **Enron Email Reply Dataset** to learn professional communication patterns.
 
@@ -24,13 +24,62 @@ The adapter was trained on the **Enron Email Reply Dataset** to learn profession
 ## Model Overview
 
 * **Base Model:** TinyLlama/TinyLlama-1.1B-Chat-v1.0
-* **Fine-tuning Method:** LoRA (Low Rank Adaptation)
+* **Format:** GGUF (Q4_K_M quantization)
+* **Size:** ~667 MB
 * **Task:** Email reply generation
 * **Language:** English
-* **Framework:** Unsloth + Transformers
-* **Deployment:** Designed for local inference with FastAPI or Ollama
 
-This adapter allows users to generate contextual email replies without relying on large commercial APIs.
+---
+
+## Quick Start with GGUF
+
+### Using Ollama (Recommended)
+
+```bash
+# Pull the model from Hugging Face
+huggingface-cli download AshankGupta/tinyllama-email-reply tinyllama-chat.Q4_K_M.gguf --local-dir ./model
+
+# Or download directly
+curl -L -o model.gguf "https://huggingface.co/AshankGupta/tinyllama-email-reply/resolve/main/tinyllama-chat.Q4_K_M.gguf"
+
+# Create Ollama model
+ollama create tinyllama-email-reply -f Modelfile
+
+# Run
+ollama run tinyllama-email-reply
+```
+
+### Using llama.cpp
+
+```bash
+# Download GGUF file from Hugging Face
+curl -L -o tinyllama-email-reply.Q4_K_M.gguf \
+  "https://huggingface.co/AshankGupta/tinyllama-email-reply/resolve/main/tinyllama-chat.Q4_K_M.gguf"
+
+# Run inference
+./llama.cpp/llama-cli -m tinyllama-email-reply.Q4_K_M.gguf -p "Write a professional email reply to: Can you send the invoice by tomorrow?"
+```
+
+### Using Python
+
+```python
+from llama_cpp import Llama
+
+model = Llama(
+    model_path="tinyllama-chat.Q4_K_M.gguf",
+    n_ctx=1024,
+)
+
+prompt = """You are an AI assistant that writes professional email replies.
+
+Email:
+Can you send the invoice by tomorrow?
+
+Reply:"""
+
+output = model(prompt, max_tokens=120)
+print(output["choices"][0]["text"])
+```
 
 ---
 
@@ -44,12 +93,6 @@ This model is designed for:
 * Local AI workflows
 * Research on small language models
 
-Example applications include:
-
-* Gmail Smart Reply style systems
-* Chrome extensions for automated responses
-* Offline AI assistants
-
 ---
 
 ## Training Dataset
@@ -58,7 +101,7 @@ The model was trained using the **Enron Email Reply Dataset**, which contains re
 
 Dataset characteristics:
 
-* ~15,000 email–reply pairs
+* ~15,000 email-reply pairs
 * Business and professional communication
 * Cleaned and formatted into instruction-style prompts
 
@@ -84,56 +127,7 @@ Sure, I will send the report by tomorrow.
 * **Sequence length:** 512 tokens
 * **Optimizer:** AdamW
 * **Base architecture:** TinyLlama 1.1B
-
-Only a small percentage of parameters were trained via LoRA adapters, enabling efficient training on consumer GPUs.
-
----
-
-## Usage
-
-Install dependencies:
-
-```
-pip install unsloth transformers accelerate bitsandbytes
-```
-
-Load the model:
-
-```python
-from unsloth import FastLanguageModel
-
-model, tokenizer = FastLanguageModel.from_pretrained(
-    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
-    load_in_4bit=True
-)
-
-model.load_adapter("ashankgupta/tinyllama-email-reply")
-```
-
-Example inference:
-
-```python
-email = """
-Hi,
-
-Can you send the invoice by tomorrow?
-"""
-
-prompt = f"""
-You are an AI assistant that writes professional email replies.
-
-Email:
-{email}
-
-Reply:
-"""
-
-inputs = tokenizer(prompt, return_tensors="pt")
-
-output = model.generate(**inputs, max_new_tokens=120)
-
-print(tokenizer.decode(output[0], skip_special_tokens=True))
-```
+* **Quantization:** Q4_K_M
 
 ---