Neeharika20/qwen3-4b-nl2sql

Qwen3-4B NL → SQL (Spider Fine-Tuned)

Model Overview

• Model name: qwen3-4b-nl2sql
• Base model: Qwen/Qwen3-4B-Instruct-2507
• Model type: Decoder-only Transformer (Causal Language Model)
• Task: Natural Language → SQL Query Generation
• Language: English
• License: Apache-2.0
• Author: Neeharika

✅ This repository contains merged model weights and can be used directly with
AutoModelForCausalLM.from_pretrained.

This model is a fine-tuned version of Qwen3-4B, optimized to translate natural language questions into executable SQL queries using database schema context. It is designed for structured data querying and analytics use cases rather than open-ended conversation.

---

Intended Use

Primary Use Cases

• Natural language interfaces for SQL databases
• Backend services that auto-generate SQL
• Data analytics assistants
• Research in semantic parsing and text-to-SQL

Out-of-Scope Uses

• General-purpose chatbots
• Code generation beyond SQL
• Autonomous decision-making in high-risk domains (medical, legal, financial)

---

Training Data

Dataset

• Name: Spider Dataset
• Type: Public benchmark for cross-domain text-to-SQL tasks
• Domains: Multiple real-world databases with diverse schemas
• Size: ~7k–10k question–SQL pairs (after preprocessing)

Data Characteristics

• Multi-table joins
• Nested and correlated subqueries
• Aggregations (COUNT, AVG, SUM, GROUP BY, HAVING)
• Schema-dependent reasoning

Preprocessing

• Converted samples into instruction-style format
• Injected full database schema into prompts
• Cleaned malformed or ambiguous samples
• Normalized SQL formatting

---

Prompt Format

The model was trained using schema-aware instruction prompts:

### Instruction:
Convert the question into an SQL query.

### Database Schema:
<Table and column definitions>

### Question:
<User query>

### SQL:

Training Procedure

Frameworks & Libraries

• Hugging Face Transformers
• Hugging Face TRL (Supervised Fine-Tuning)
• Unsloth (memory-efficient training)
• PyTorch

Fine-Tuning Details

• Method: Supervised Fine-Tuning (SFT)
• Quantization during training: 4-bit (bnb)
• Precision: Mixed precision
• Optimization: LoRA adapters (merged post-training)

Key Hyperparameters

• Max sequence length: 2048
• Learning rate: 2e-4
• Optimizer: AdamW
• Padding: Right-padding
• Gradient checkpointing: Enabled

---

Example Usage

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Neeharika20/qwen3-4b-nl2sql",
    max_seq_length=2048,
    load_in_4bit=True,
    device_map="auto",
)

model.eval()

prompt = """### Instruction:
Convert the question into an SQL query.

### Database Schema:
CREATE TABLE students(
    id INT,
    name TEXT,
    age INT
);

### Question:
List the names of all students.

### SQL:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=64,
        temperature=0.2,
        do_sample=False,
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))