Update README.md

README.md

@@ -1,7 +1,11 @@
 ---
+base_model: microsoft/Phi-3-mini-4k-instruct
+library_name: transformers
+license: mit
 language:
 - en
-license: apache-2.0
+datasets:
+- b-mc2/sql-create-context
 tags:
 - sql
 - text-to-sql
@@ -9,21 +13,15 @@ tags:
 - phi-3
 - fine-tuned
 - text-generation
-base_model: microsoft/Phi-3-mini-4k-instruct
+- phi3
+pipeline_tag: text-generation
 ---
 
 # Phi-3 Mini SQL Generator — Merged Model
 
-This is the **merged (standalone) version** for `Shizu0n/phi3-mini-sql-generator` on top of `microsoft/Phi-3-mini-4k-instruct`.
-
-The LoRA adapter weights have been merged directly into the base model, making this a standard `AutoModelForCausalLM` — no PEFT dependency required for inference.
-
-## Why two versions?
-
-| Repo | Purpose |
-|---|---|
-| `Shizu0n/phi3-mini-sql-generator` | Original QLoRA adapter — documents the training pipeline |
-| `Shizu0n/phi3-mini-sql-generator-merged` | Merged standalone model — used for deployment and inference |
+Merged standalone version of [Shizu0n/phi3-mini-sql-generator](https://huggingface.co/Shizu0n/phi3-mini-sql-generator)
+— LoRA adapter weights fused into [Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct).
+No PEFT dependency required for inference.
 
 ## Evaluation — Base vs Fine-tuned
 
@@ -34,29 +32,25 @@ Evaluated on 200 held-out examples from [b-mc2/sql-create-context](https://huggi
 | Phi-3-mini-4k-instruct (base) | 2.0% |
 | **This model (fine-tuned)** | **73.5%** |
 
+> Exact match: normalized SQL comparison (lowercase, strip whitespace/semicolons).
+
+## Why two versions?
+
+| Repo | Purpose |
+|---|---|
+| [`Shizu0n/phi3-mini-sql-generator`](https://huggingface.co/Shizu0n/phi3-mini-sql-generator) | QLoRA adapter — documents the training pipeline |
+| `Shizu0n/phi3-mini-sql-generator-merged` | Merged standalone — used for deployment and inference |
+
 ## Training Details
 
 - **Dataset:** b-mc2/sql-create-context — 1,000 train / 200 validation examples
-- **Method:** QLoRA (4-bit NF4, LoRA rank 16, alpha 32)
+- **Method:** QLoRA (4-bit NF4, LoRA rank 16, alpha 32, target modules: qkv_proj/o_proj/gate_up_proj/down_proj)
 - **Hardware:** NVIDIA T4 (Google Colab free tier)
 - **Training time:** ~21 min
 - **Final train loss:** 0.6526
+- **Best checkpoint:** step 250 (by eval loss)
 
-## Validation
-
-The merge was accepted only after all three smoke tests returned a concrete SQL query:
-
-1. PEFT adapter loaded on the base model.
-2. Local merged directory after `merge_and_unload()` and `save_pretrained()`.
-3. Downloaded model from this Hugging Face repo with `force_download=True`.
-
-Reference smoke output:
-
-```sql
-SELECT AVG(salary), department FROM employees GROUP BY department
-```
-
-## Inference example
+## Inference Example
 
 ```python
 import torch
@@ -64,26 +58,20 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
 
 model_id = "Shizu0n/phi3-mini-sql-generator-merged"
 
-tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=False)
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
 model = AutoModelForCausalLM.from_pretrained(
     model_id,
-    dtype=torch.float16,
+    torch_dtype=torch.float16,
     device_map="auto",
     trust_remote_code=False,
     attn_implementation="eager",
 )
-model.config.use_cache = False
+model.eval()
 
 prompt = (
-    "Given the following SQL table, write a SQL query.
-
-"
-    "Table: employees (id, name, department, salary)
-
-"
-    "Question: What is the average salary per department?
-
-SQL:"
+    "Given the following SQL table, write a SQL query.\n\n"
+    "Table: employees (id, name, department, salary)\n\n"
+    "Question: What is the average salary per department?\n\nSQL:"
 )
 
 inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
@@ -96,13 +84,24 @@ with torch.inference_mode():
         repetition_penalty=1.1,
         pad_token_id=tokenizer.eos_token_id,
     )
+prompt_len = inputs["input_ids"].shape[-1]
+print(tokenizer.decode(outputs[0][prompt_len:], skip_special_tokens=True))
+```
 
-text = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
-print(text.strip())
+Expected output:
+```sql
+SELECT AVG(salary), department FROM employees GROUP BY department
 ```
 
-## Notes
+## Validation
+
+Merge accepted after three smoke tests:
+1. PEFT adapter loaded on base model
+2. Local merged directory after `merge_and_unload()` + `save_pretrained()`
+3. Downloaded from this repo with `force_download=True`
+
+## Limitations
 
-- Use `trust_remote_code=False`; the built-in Transformers Phi-3 implementation avoids stale remote-code failures.
-- Do not patch `rope_scaling` manually.
-- Do not mutate `_tied_weights_keys` before saving.
+- Fine-tuned on 1,000 examples — best suited for simple to medium complexity SELECT queries
+- Not tested on dialect-specific SQL (PostgreSQL/MySQL-specific functions)
+- May struggle with multi-table JOINs and nested subqueries