Instructions to use Hokaigo/Qwen2.5-1.5B-SQL-Critic-LoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Hokaigo/Qwen2.5-1.5B-SQL-Critic-LoRA with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Hokaigo/Qwen2.5-1.5B-SQL-Critic-LoRA", device_map="auto") - PEFT
How to use Hokaigo/Qwen2.5-1.5B-SQL-Critic-LoRA with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Qwen2.5-1.5B-SQL-Critic-LoRA
Model Description
This is a LoRA adapter for the Qwen/Qwen2.5-1.5B-Instruct model, fine-tuned to act as an AI SQL Critic.
The model is trained to compare an original SQL query with an optimized version based on a provided DDL schema, and strictly output a JSON object evaluating their semantic equivalence.
- Base Model: Qwen/Qwen2.5-1.5B-Instruct
- Task: Semantic SQL Comparison & JSON Formatting
- Language: English
- Training Method: Parameter-Efficient Fine-Tuning (PEFT) using LoRA.
Intended Use
This adapter is intended to be used over the base Qwen2.5-1.5B-Instruct model to enforce strict JSON formatting and improve logic in identifying missing SQL clauses (like WHERE, GROUP BY) between two queries.
Training & Evaluation Results
Before fine-tuning, the base model's accuracy on the semantic comparison task was 52.3% (random guessing level). After applying this LoRA adapter, the evaluation metrics improved significantly:
- Equivalence Accuracy: 99.1%
- Valid JSON Output: 100%
Figure 1: Training and validation loss over 40 steps, demonstrating successful convergence without overfitting.
Catastrophic Forgetting Check
The adapter successfully preserves the base model's general knowledge. When tested on math logic, geography, and grammar correction without a strict JSON system prompt, the model outputs fluent natural language and correctly solves the tasks, proving that the base weights were not structurally compromised.
How to use
You can load this adapter using the peft library in Python:
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model_name = "Qwen/Qwen2.5-1.5B-Instruct"
adapter_name = "Hokaigo/Qwen2.5-1.5B-SQL-Critic-LoRA"
model = AutoModelForCausalLM.from_pretrained(base_model_name)
model = PeftModel.from_pretrained(model, adapter_name)
tokenizer = AutoTokenizer.from_pretrained(adapter_name)