Upload 2 files

Llama_3B_Fine_Tuning.ipynb

@@ -0,0 +1,645 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 🚀 **Maring Language Translation Fine-Tuning**\n",
+    "\n",
+    "**Model:** Llama-3.2-3B-Instruct  \n",
+    "**Task:** English ↔ Maring Translation  \n",
+    "**Hardware:** Tesla T4 (Free Colab)  \n",
+    "**Time:** ~18-22 minutes  \n",
+    "\n",
+    "## 📋 **Steps:**\n",
+    "1. ✅ Mount Google Drive\n",
+    "2. ✅ Upload training data  \n",
+    "3. ✅ Install dependencies  \n",
+    "4. ✅ Load model  \n",
+    "5. ✅ Train model  \n",
+    "6. ✅ Save to Drive  \n",
+    "7. ✅ Test inference\n",
+    "\n",
+    "## 🎯 **Optimizations Applied:**\n",
+    "- **Batch Size:** 4 (better learning)\n",
+    "- **LoRA Rank:** 32 (higher capacity)\n",
+    "- **Mixed Precision:** fp16 (memory efficient)\n",
+    "- **Beam Search:** 3 beams (better quality)\n",
+    "- **Temperature:** 0.3 (natural responses)\n",
+    "- **Max Tokens:** 100 (detailed answers)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🗂️ **STEP 1: Mount Google Drive**\n",
+    "\n",
+    "**⚠️ IMPORTANT:** This saves all models directly to your Google Drive for fast download!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from google.colab import drive\n",
+    "drive.mount('/content/drive')\n",
+    "\n",
+    "# Create folder for our models\n",
+    "!mkdir -p \"/content/drive/MyDrive/Maring_Translation_Model\"\n",
+    "print(\"✅ Google Drive mounted successfully!\")\n",
+    "print(\"📁 Models will be saved to: /content/drive/MyDrive/Maring_Translation_Model\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 📤 **STEP 2: Upload Training Data**\n",
+    "\n",
+    "**Upload your `training_data_improved.json` file to Colab:**\n",
+    "1. Click the 📁 folder icon (left sidebar)\n",
+    "2. Click the 📤 upload button  \n",
+    "3. Select your `training_data_improved.json` file\n",
+    "4. Wait for green checkmark ✅"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "# Check if training data exists\n",
+    "if os.path.exists('training_data_improved.json'):\n",
+    "    file_size = os.path.getsize('training_data_improved.json')\n",
+    "    print(f\"✅ training_data_improved.json found! Size: {file_size:,} bytes\")\n",
+    "\n",
+    "    # Copy to Drive as backup\n",
+    "    !cp training_data_improved.json \"/content/drive/MyDrive/Maring_Translation_Model/\"\n",
+    "    print(\"📋 Training data backed up to Google Drive\")\n",
+    "else:\n",
+    "    print(\"❌ training_data_improved.json NOT found!\")\n",
+    "    print(\"📤 Please upload the file using the folder icon on the left\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🔧 **STEP 3: Install Dependencies**\n",
+    "\n",
+    "**This installs Unsloth and all required libraries.**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "import os, re\n",
+    "if \"COLAB_\" not in \"\".join(os.environ.keys()):\n",
+    "    !pip install unsloth\n",
+    "else:\n",
+    "    import torch\n",
+    "    v = re.match(r'[\\d]{1,}\\.[\\d]{1,}', str(torch.__version__)).group(0)\n",
+    "    xformers = 'xformers==' + {'2.10':'0.0.34','2.9':'0.0.33.post1','2.8':'0.0.32.post2'}.get(v, \"0.0.34\")\n",
+    "    !pip install sentencepiece protobuf \"datasets==4.3.0\" \"huggingface_hub>=0.34.0\" hf_transfer\n",
+    "    !pip install --no-deps unsloth_zoo bitsandbytes accelerate {xformers} peft trl triton unsloth\n",
+    "\n",
+    "!pip install transformers==4.56.2\n",
+    "!pip install --no-deps trl==0.22.2\n",
+    "\n",
+    "print(\"✅ All dependencies installed successfully!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🤖 **STEP 4: Load Model & Setup LoRA**\n",
+    "\n",
+    "**This loads Llama-3.2-3B-Instruct and prepares it for training.**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from unsloth import FastLanguageModel\n",
+    "import torch\n",
+    "\n",
+    "# Model configuration\n",
+    "max_seq_length = 2048\n",
+    "dtype = None  # Auto-detect for T4\n",
+    "load_in_4bit = True\n",
+    "\n",
+    "print(\"🔄 Loading Llama-3.2-3B-Instruct...\")\n",
+    "model, tokenizer = FastLanguageModel.from_pretrained(\n",
+    "    model_name = \"unsloth/Llama-3.2-3B-Instruct\",\n",
+    "    max_seq_length = max_seq_length,\n",
+    "    dtype = dtype,\n",
+    "    load_in_4bit = load_in_4bit,\n",
+    ")\n",
+    "\n",
+    "print(\"⚡ Adding LoRA adapters...\")\n",
+    "model = FastLanguageModel.get_peft_model(\n",
+    "    model,\n",
+    "    r = 8,  # CRITICAL FIX: Lower rank for better generalization with 11k examples\n",
+    "    lora_alpha = 16,  # Keep alpha higher than rank for stability\n",
+    "    target_modules = [\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
+    "                      \"gate_proj\", \"up_proj\", \"down_proj\"],\n",
+    "    lora_dropout = 0,\n",
+    "    bias = \"none\",\n",
+    "    use_gradient_checkpointing = \"unsloth\",\n",
+    "    random_state = 3407,\n",
+    "    use_rslora = False,\n",
+    "    loftq_config = None,\n",
+    ")\n",
+    "\n",
+    "print(\"✅ Model loaded and LoRA adapters added!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 📊 **STEP 5: Load & Prepare Training Data**\n",
+    "\n",
+    "**This loads your JSON data and formats it for training.**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from unsloth.chat_templates import get_chat_template, standardize_sharegpt\n",
+    "import json\n",
+    "from datasets import Dataset\n",
+    "import collections\n",
+    "\n",
+    "# FIXED: Use correct chat template for Llama-3.2\n",
+    "tokenizer = get_chat_template(\n",
+    "    tokenizer,\n",
+    "    chat_template = \"llama-3.1\",\n",
+    ")\n",
+    "\n",
+    "# FIXED: Define system prompt for KomoAI\n",
+    "SYSTEM_PROMPT = \"You are KomoAI 3B, an expert translator between English and Maring language developed by Moshilning Koninga. Provide accurate translations and cultural information about the Maring tribe.\"\n",
+    "\n",
+    "def load_training_data(file_path):\n",
+    "    \"\"\"Load training data from JSON file\"\"\"\n",
+    "    data = []\n",
+    "    try:\n",
+    "        with open(file_path, 'r', encoding='utf-8') as f:\n",
+    "            for line in f:\n",
+    "                line = line.strip()\n",
+    "                if line:\n",
+    "                    data.append(json.loads(line))\n",
+    "    except FileNotFoundError:\n",
+    "        print(f\"❌ Error: {file_path} not found\")\n",
+    "        return []\n",
+    "    except json.JSONDecodeError as e:\n",
+    "        print(f\"❌ Error parsing JSON: {e}\")\n",
+    "        return []\n",
+    "    return data\n",
+    "\n",
+    "def convert_to_chat_format(data):\n",
+    "    \"\"\"Convert instruction/input/output to chat format\"\"\"\n",
+    "    conversations = []\n",
+    "    for item in data:\n",
+    "        instruction = item['instruction']\n",
+    "        input_text = item['input']\n",
+    "        output_text = item['output']\n",
+    "\n",
+    "        # FIXED: Use colon to match inference format\n",
+    "        user_message = f\"{instruction}: {input_text}\"\n",
+    "        conversation = [\n",
+    "            {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n",
+    "            {\"role\": \"user\", \"content\": user_message},\n",
+    "            {\"role\": \"assistant\", \"content\": output_text}\n",
+    "        ]\n",
+    "        conversations.append({\"conversations\": conversation})\n",
+    "    return conversations\n",
+    "\n",
+    "def formatting_prompts_func(examples):\n",
+    "    convos = examples[\"conversations\"]\n",
+    "    texts = [tokenizer.apply_chat_template(convo, tokenize = False, add_generation_prompt = False) for convo in convos]\n",
+    "    return { \"text\" : texts }\n",
+    "\n",
+    "# Load and process data\n",
+    "print(\"📂 Loading training data...\")\n",
+    "training_data = load_training_data('training_data_improved.json')\n",
+    "print(f\"📊 Loaded {len(training_data)} training examples\")\n",
+    "\n",
+    "# CRITICAL FIX: Check for duplicates before training\n",
+    "user_messages = [f\"{item['instruction']}: {item['input']}\" for item in training_data]\n",
+    "counts = collections.Counter(user_messages)\n",
+    "dupes = [(msg, cnt) for msg, cnt in counts.items() if cnt > 5]\n",
+    "print(f\"✅ Unique examples: {len(counts)}\")\n",
+    "if dupes:\n",
+    "    print(f\"⚠️ Found {len(dupes)} duplicated patterns:\")\n",
+    "    for msg, cnt in dupes[:3]:\n",
+    "        print(f\"  - '{msg}' appears {cnt} times\")\n",
+    "\n",
+    "if len(training_data) > 0:\n",
+    "    # Convert to chat format\n",
+    "    chat_data = convert_to_chat_format(training_data)\n",
+    "    dataset = Dataset.from_list(chat_data)\n",
+    "\n",
+    "    # Apply standardization and formatting\n",
+    "    dataset = standardize_sharegpt(dataset)\n",
+    "    dataset = dataset.map(formatting_prompts_func, batched = True)\n",
+    "\n",
+    "    print(f\"✅ Dataset ready: {len(dataset)} examples\")\n",
+    "    print(f\"📝 Sample: {dataset[0]['conversations'][1]['content'][:50]}...\")\n",
+    "else:\n",
+    "    print(\"❌ No training data loaded!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🏋️ **STEP 6: Start Training**\n",
+    "\n",
+    "**⏱️ This will take 18-22 minutes. Watch the progress bar!**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from trl import SFTConfig, SFTTrainer\n",
+    "from transformers import DataCollatorForSeq2Seq\n",
+    "from unsloth.chat_templates import train_on_responses_only\n",
+    "\n",
+    "# Create trainer\n",
+    "trainer = SFTTrainer(\n",
+    "    model = model,\n",
+    "    tokenizer = tokenizer,\n",
+    "    train_dataset = dataset,\n",
+    "    dataset_text_field = \"text\",\n",
+    "    max_seq_length = max_seq_length,\n",
+    "    data_collator = DataCollatorForSeq2Seq(tokenizer = tokenizer),\n",
+    "    packing = False,\n",
+    "    args = SFTConfig(\n",
+    "        per_device_train_batch_size = 4,\n",
+    "        gradient_accumulation_steps = 2,\n",
+    "        warmup_steps = 50,  # OPTIMIZED: Proper warmup (not too high, not too low)\n",
+    "        num_train_epochs = 6,\n",
+    "        learning_rate = 1e-4,  # Slightly lower for stability\n",
+    "        logging_steps = 10,\n",
+    "        optim = \"adamw_8bit\",\n",
+    "        weight_decay = 0.01,  # Better regularization\n",
+    "        lr_scheduler_type = \"cosine\",  # Better final convergence\n",
+    "        seed = 3407,\n",
+    "        output_dir = \"outputs\",\n",
+    "        report_to = \"none\",\n",
+    "        # CRITICAL FIX: Remove fp16 and gradient_checkpointing - Unsloth handles these\n",
+    "    ),\n",
+    ")\n",
+    "\n",
+    "# FIXED: Apply response masking for Llama-3.2 format\n",
+    "trainer = train_on_responses_only(\n",
+    "    trainer,\n",
+    "    instruction_part = \"<|start_header_id|>user<|end_header_id|>\\n\\n\",\n",
+    "    response_part = \"<|start_header_id|>assistant<|end_header_id|>\\n\\n\",\n",
+    ")\n",
+    "\n",
+    "# Calculate actual total steps dynamically\n",
+    "total_steps = (len(dataset) // 8) * 6  # Dynamic calculation\n",
+    "\n",
+    "print(\"🚀 Starting training...\")\n",
+    "print(f\"📊 Training 6 epochs on {len(dataset)} examples\")\n",
+    "print(f\"🔄 Total steps: ~{total_steps}\")\n",
+    "print(f\"🔥 Warmup: 50 steps ({50/total_steps*100:.1f}% of training)\")\n",
+    "\n",
+    "# Start training\n",
+    "trainer_stats = trainer.train()\n",
+    "\n",
+    "print(\"🎉 Training completed!\")\n",
+    "print(f\"⏱️ Training time: {trainer_stats.metrics['train_runtime']:.1f} seconds\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 💾 **STEP 7: Save Models to Google Drive**\n",
+    "\n",
+    "**All models will be saved to your Google Drive for fast download!**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Define save paths in Google Drive\n",
+    "drive_path = \"/content/drive/MyDrive/Maring_Translation_Model\"\n",
+    "lora_path = f\"{drive_path}/llama_maring_lora\"\n",
+    "gguf_path = f\"{drive_path}/llama_maring_gguf\"\n",
+    "\n",
+    "print(\"💾 Saving models to Google Drive...\")\n",
+    "\n",
+    "# FIXED: Clear cache before saving to prevent OOM\n",
+    "import torch\n",
+    "torch.cuda.empty_cache()\n",
+    "\n",
+    "# 1. Save LoRA adapters (main trained model)\n",
+    "print(\"📁 Saving LoRA adapters...\")\n",
+    "model.save_pretrained(lora_path)\n",
+    "tokenizer.save_pretrained(lora_path)\n",
+    "print(f\"✅ LoRA saved to: {lora_path}\")\n",
+    "\n",
+    "# 2. Save GGUF format (for fast local inference)\n",
+    "print(\"\\n🔄 Converting to GGUF format...\")\n",
+    "model.save_pretrained_gguf(\n",
+    "    gguf_path,\n",
+    "    tokenizer,\n",
+    "    quantization_method = \"q4_k_m\"  # Recommended for good balance\n",
+    ")\n",
+    "print(f\"✅ GGUF saved to: {gguf_path}\")\n",
+    "\n",
+    "# 3. Save 16-bit merged model (full model) - only if enough space\n",
+    "print(\"\\n🔄 Saving 16-bit merged model...\")\n",
+    "try:\n",
+    "    model.save_pretrained_merged(\n",
+    "        f\"{drive_path}/llama_maring_16bit\",\n",
+    "        tokenizer,\n",
+    "        save_method = \"merged_16bit\"\n",
+    "    )\n",
+    "    print(f\"✅ 16-bit model saved to: {drive_path}/llama_maring_16bit\")\n",
+    "except RuntimeError as e:\n",
+    "    if \"no disk space\" in str(e):\n",
+    "        print(\"⚠️ Not enough space for 16-bit model (this is normal)\")\n",
+    "        print(\"✅ LoRA and GGUF files are sufficient for most uses\")\n",
+    "    else:\n",
+    "        raise e\n",
+    "\n",
+    "print(\"\\n🎉 All models saved to Google Drive!\")\n",
+    "print(\"📂 Check your Drive: MyDrive/Maring_Translation_Model\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🧪 **STEP 8: Test the Trained Model**\n",
+    "\n",
+    "**Test your fine-tuned translation model!**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from unsloth.chat_templates import get_chat_template\n",
+    "from transformers import TextStreamer\n",
+    "\n",
+    "# Setup for inference\n",
+    "tokenizer = get_chat_template(tokenizer, chat_template = \"llama-3.1\")\n",
+    "FastLanguageModel.for_inference(model)\n",
+    "\n",
+    "print(\"🧪 Testing translation capabilities...\\n\")\n",
+    "\n",
+    "# Test 1: English to Maring\n",
+    "print(\"📝 Test 1: English to Maring\")\n",
+    "messages = [\n",
+    "    {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n",
+    "    {\"role\": \"user\", \"content\": \"Translate to Maring: gate\"}\n",
+    "]\n",
+    "inputs = tokenizer.apply_chat_template(\n",
+    "    messages, tokenize = True, add_generation_prompt = True, return_tensors = \"pt\"\n",
+    ").to(\"cuda\")\n",
+    "\n",
+    "# CRITICAL FIX: Use beam search for translation (deterministic)\n",
+    "outputs = model.generate(\n",
+    "    input_ids = inputs, \n",
+    "    max_new_tokens = 64,  # Reduced for single-word translations\n",
+    "    use_cache = True,\n",
+    "    num_beams = 3,  # Beam search for quality\n",
+    "    do_sample = False,  # CRITICAL: Must be False with num_beams > 1\n",
+    "    repetition_penalty = 1.1,\n",
+    ")\n",
+    "# IMPROVED: Better output parsing with special token removal\n",
+    "result = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0].split(\"assistant\")[-1].strip()\n",
+    "# Remove any trailing special tokens\n",
+    "result = result.replace(\"<|eot_id|>\", \"\").replace(\"<|end_of_text|>\", \"\").strip()\n",
+    "print(f\"🔤 Result: {result}\\n\")\n",
+    "\n",
+    "# Test 2: Maring to English\n",
+    "print(\"📝 Test 2: Maring to English\")\n",
+    "messages = [\n",
+    "    {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n",
+    "    {\"role\": \"user\", \"content\": \"Translate to English: Pater\"}\n",
+    "]\n",
+    "inputs = tokenizer.apply_chat_template(\n",
+    "    messages, tokenize = True, add_generation_prompt = True, return_tensors = \"pt\"\n",
+    ").to(\"cuda\")\n",
+    "\n",
+    "# CRITICAL FIX: Use beam search for translation (deterministic)\n",
+    "outputs = model.generate(\n",
+    "    input_ids = inputs, \n",
+    "    max_new_tokens = 64,  # Reduced for single-word translations\n",
+    "    use_cache = True,\n",
+    "    num_beams = 3,  # Beam search for quality\n",
+    "    do_sample = False,  # CRITICAL: Must be False with num_beams > 1\n",
+    "    repetition_penalty = 1.1,\n",
+    ")\n",
+    "result = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0].split(\"assistant\")[-1].strip()\n",
+    "# Remove any trailing special tokens\n",
+    "result = result.replace(\"<|eot_id|>\", \"\").replace(\"<|end_of_text|>\", \"\").strip()\n",
+    "print(f\"🔤 Result: {result}\\n\")\n",
+    "\n",
+    "# Test 3: Cultural question (use sampling for variety)\n",
+    "print(\"📝 Test 3: Cultural Question\")\n",
+    "messages = [\n",
+    "    {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n",
+    "    {\"role\": \"user\", \"content\": \"What is the prefix for the first-born son in Maring?\"}\n",
+    "]\n",
+    "inputs = tokenizer.apply_chat_template(\n",
+    "    messages, tokenize = True, add_generation_prompt = True, return_tensors = \"pt\"\n",
+    ").to(\"cuda\")\n",
+    "\n",
+    "text_streamer = TextStreamer(tokenizer, skip_prompt = True)\n",
+    "print(\"🔤 Streaming result: \")\n",
+    "# CRITICAL FIX: Use sampling for Q&A (not translation)\n",
+    "_ = model.generate(\n",
+    "    input_ids = inputs, \n",
+    "    streamer = text_streamer, \n",
+    "    max_new_tokens = 150,  # More tokens for detailed answers\n",
+    "    use_cache = True, \n",
+    "    num_beams = 1,  # Single beam for Q&A\n",
+    "    do_sample = True,  # CRITICAL: Enable sampling for variety\n",
+    "    temperature = 0.3,  # Temperature for sampling\n",
+    "    repetition_penalty = 1.1,\n",
+    ")\n",
+    "\n",
+    "print(\"\\n✅ Model testing completed!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 💬 **Interactive Model Testing**\n",
+    "\n",
+    "Use the cell below to test your fine-tuned model with custom prompts. Type your question or translation request and press Enter."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from unsloth.chat_templates import get_chat_template\n",
+    "from transformers import TextStreamer\n",
+    "\n",
+    "print(\"✨ Welcome to the Interactive Translation Tester! ✨\")\n",
+    "print(\"----------------------------------------------------\")\n",
+    "print(\"Type your translation request in English or Maring.\\n\")\n",
+    "print(\"Examples: 'Translate to Maring: hello' or 'Translate to English: Kumka'\\n\")\n",
+    "print(\"Type 'quit' to exit.\\n\")\n",
+    "\n",
+    "# Loop to allow continuous testing\n",
+    "while True:\n",
+    "    user_input = input(\"➡️ Your input: \")\n",
+    "    if user_input.lower().strip() == 'quit':\n",
+    "        break\n",
+    "    elif not user_input.strip():\n",
+    "        print(\"Please enter some text.\")\n",
+    "        continue\n",
+    "\n",
+    "    # Detect if this is a translation or Q&A\n",
+    "    is_translation = any(word in user_input.lower() for word in ['translate', 'convert', 'say', 'meaning of', 'what does'])\n",
+    "    \n",
+    "    # Format the input with system prompt\n",
+    "    messages = [\n",
+    "        {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n",
+    "        {\"role\": \"user\", \"content\": user_input}\n",
+    "    ]\n",
+    "    inputs = tokenizer.apply_chat_template(\n",
+    "        messages, tokenize = True, add_generation_prompt = True, return_tensors = \"pt\"\n",
+    "    ).to(\"cuda\")\n",
+    "\n",
+    "    # Generate the response\n",
+    "    print(\"🤖 Model's Response:\")\n",
+    "    text_streamer = TextStreamer(tokenizer, skip_prompt = True, clean_up_tokenization_spaces = True)\n",
+    "    \n",
+    "    # CRITICAL FIX: Use different parameters based on task type\n",
+    "    if is_translation:\n",
+    "        # For translations: use beam search (deterministic)\n",
+    "        _ = model.generate(\n",
+    "            input_ids = inputs,\n",
+    "            streamer = text_streamer,\n",
+    "            max_new_tokens = 64,  # Shorter for translations\n",
+    "            use_cache = True,\n",
+    "            num_beams = 3,  # Beam search\n",
+    "            do_sample = False,  # No sampling with beams\n",
+    "            repetition_penalty = 1.1,\n",
+    "        )\n",
+    "    else:\n",
+    "        # For Q&A: use sampling (more variety)\n",
+    "        _ = model.generate(\n",
+    "            input_ids = inputs,\n",
+    "            streamer = text_streamer,\n",
+    "            max_new_tokens = 150,  # Longer for detailed answers\n",
+    "            use_cache = True,\n",
+    "            num_beams = 1,  # Single beam\n",
+    "            do_sample = True,  # Enable sampling\n",
+    "            temperature = 0.3,  # Temperature for variety\n",
+    "            repetition_penalty = 1.1,\n",
+    "        )\n",
+    "    \n",
+    "    print(\"\\n----------------------------------------------------\")\n",
+    "\n",
+    "print(\"👋 Interactive testing ended. Thank you!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 📂 **STEP 9: Download Models from Google Drive**\n",
+    "\n",
+    "**Your models are now in Google Drive! Download them anytime:**\n",
+    "\n",
+    "### 📁 **Files Created:**\n",
+    "- `llama_maring_lora/` - **LoRA adapters** (small, for loading with base model)\n",
+    "- `llama_maring_gguf.q4_k_m.gguf` - **GGUF format** (ready for local inference)\n",
+    "- `llama_maring_16bit/` - **Full 16-bit model** (larger, highest quality)\n",
+    "\n",
+    "### 🚀 **How to Download:**\n",
+    "1. Open Google Drive (drive.google.com)\n",
+    "2. Go to `My Drive > Maring_Translation_Model`\n",
+    "3. Right-click any file/folder → Download\n",
+    "\n",
+    "### 💡 **Recommended Usage:**\n",
+    "- **For development:** Use `llama_maring_lora/` (small, fast)\n",
+    "- **For deployment:** Use `llama_maring_gguf.q4_k_m.gguf` (single file)\n",
+    "- **For best quality:** Use `llama_maring_16bit/` (larger, most accurate)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🎉 **TRAINING COMPLETE!**\n",
+    "\n",
+    "### ✅ **What You Accomplished:**\n",
+    "- ✅ Fine-tuned Llama-3.2-3B on English-Maring translation\n",
+    "- ✅ Used improved training data with consistent translations\n",
+    "- ✅ Saved models in multiple formats\n",
+    "- ✅ Tested translation capabilities\n",
+    "- ✅ All files safely stored in Google Drive\n",
+    "\n",
+    "### 🚀 **Next Steps:**\n",
+    "1. Download models from Google Drive\n",
+    "2. Use GGUF file with llama.cpp for local inference\n",
+    "3. Deploy LoRA adapters with Hugging Face\n",
+    "4. Share your translation model!\n",
+    "\n",
+    "**🎊 Congratulations on your improved Maring translation model!**"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}