Update README.md

README.md

@@ -15,6 +15,33 @@ library_name: adapter-transformers
 
 
 
+
+
+
+
+# Llama-3.2-3B- 
+
+![License](https://img.shields.io/badge/License-Apache%202.0-blue)
+![Python](https://img.shields.io/badge/Python-3.8%2B-green)
+![Framework](https://img.shields.io/badge/Framework-Unsloth-ff69b4)
+![Model](https://img.shields.io/badge/Model-Llama_3.2_3B-orange)
+
+
+
+This repository contains code to fine-tune the **Llama-3.2-3B-Instruct** model using Unsloth for efficient training. The model is optimized for conversational tasks and supports 4-bit quantization, LoRA adapters, and GGUF export.
+
+## Model Overview
+- **Base Model**: [`Llama-3.2-3B-Instruct`](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct)
+- **Fine-Tuning Dataset**: [FineTome-100k](https://huggingface.co/datasets/mlabonne/FineTome-100k) (converted to Llama-3.1 chat format)
+- **Features**:
+  - 4-bit quantization for reduced memory usage
+  - LoRA adapters (1-10% parameter updates)
+  - Sequence length: 2048 (RoPE scaling supported)
+  - Optimized for Tesla T4 GPUs
+
+## 🚀 Quick Start
+
+
 ### Load this model as:
 ```python
 from llama_cpp import Llama
@@ -51,6 +78,130 @@ if __name__ == "__main__":
     response = generate_text(prompt)
     print(f"Prompt: {prompt}\nResponse: {response}")
 ```
+### Installation
+```bash
+pip install unsloth
+pip install --force-reinstall --no-cache-dir --no-deps git+https://github.com/unslothai/unsloth.git
+```
+
+### Load Model
+```python
+from unsloth import FastLanguageModel
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name="unsloth/Llama-3.2-3B-Instruct",
+    max_seq_length=2048,
+    dtype=None,  # Auto-detect (bf16 for Ampere+ GPUs)
+    load_in_4bit=True,
+)
+```
+
+### Run Inference
+```python
+messages = [{"role": "user", "content": "Continue the Fibonacci sequence: 1, 1, 2, 3, 5, 8,"}]
+inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
+
+outputs = model.generate(
+    inputs, 
+    max_new_tokens=64,
+    temperature=1.5,
+    min_p=0.1,
+)
+print(tokenizer.decode(outputs[0]))
+```
+
+## 🛠️ Training
+
+### Data Preparation
+The dataset is standardized to Llama-3.1 chat format:
+```python
+from unsloth.chat_templates import get_chat_template, standardize_sharegpt
+
+tokenizer = get_chat_template(tokenizer, "llama-3.1")  # Adds system prompts
+dataset = load_dataset("mlabonne/FineTome-100k", split="train")
+dataset = standardize_sharegpt(dataset)  # Converts to role/content format
+```
+
+### LoRA Configuration
+```python
+model = FastLanguageModel.get_peft_model(
+    model,
+    r=16,  # LoRA rank
+    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
+    lora_alpha=16,
+    use_gradient_checkpointing="unsloth",  # 30% less VRAM
+)
+```
+
+### Training Arguments
+```python
+from trl import SFTTrainer
+
+trainer = SFTTrainer(
+    model=model,
+    train_dataset=dataset,
+    dataset_text_field="text",
+    max_seq_length=2048,
+    args=TrainingArguments(
+        per_device_train_batch_size=2,
+        gradient_accumulation_steps=4,
+        learning_rate=2e-4,
+        max_steps=60,  # Demo: set to 60 steps. For full training, use num_train_epochs=1
+        fp16=not is_bfloat16_supported(),
+        bf16=is_bfloat16_supported(),
+        optim="adamw_8bit",
+    ),
+)
+```
+
+## 💾 Saving & Deployment
+
+### Save LoRA Adapters
+```python
+model.save_pretrained("llama3_2_3B")
+tokenizer.save_pretrained("llama3_2_3B")
+```
+
+### Export to GGUF (for llama.cpp)
+```python
+model.save_pretrained_gguf(
+    "model", 
+    tokenizer,
+    quantization_method="q4_k_m",  # Recommended quantization
+)
+```
+
+### Upload to Hugging Face Hub
+```python
+model.push_to_hub_gguf(
+    "your-username/llama3_2_3B",
+    tokenizer,
+    quantization_method=["q4_k_m", "q8_0"],  # Multiple formats
+    token="hf_your_token_here",
+)
+```
+
+## 📊 Performance
+| Metric               | Value          |
+|----------------------|----------------|
+| Training Time (60 steps) | ~7.5 minutes   |
+| Peak VRAM Usage      | 6.5 GB         |
+| Quantized Size (Q4_K_M) | ~1.9 GB     |
+
+## 📜 Notes
+- **Knowledge Cutoff**: December 2023 (updated to July 2024 via fine-tuning)
+- Use `temperature=1.5` and `min_p=0.1` for best results ([reference](https://x.com/menhguin/status/1826132708508213629))
+- For 2x faster inference, enable `FastLanguageModel.for_inference(model)`
+
+## 🤝 Contributing
+- Report issues 
+- Star the repo if you find this useful! ⭐
+
+## License
+Apache 2.0. See [LICENSE on top of Model Card]
+```
+
+```
 
 ### Key Fixes Added:
 1. **Model Download**: Uses `huggingface_hub` to properly download the GGUF file