---
license: mit
tags:
- text-to-sql
- education
- socratic-learning
- instruction-tuning
- sql
- STEM
- pedagogy
datasets:
- SQL-Instruct
---

# SQL Socratic Models

## Model Description

SQL Socratic Models are a collection of fine-tuned large language models designed for **Socratic SQL instruction in higher education**. Unlike standard Text-to-SQL systems, these models are trained to **guide learners through reasoning steps without producing final SQL solutions**, supporting conceptual understanding and active learning in STEM contexts.

Supported architectures:
- Phi-3
- Qwen2.5
- Gemma2

---

## Intended Use

These models are designed for:

- Teaching SQL concepts in higher education  
- Supporting STEM learners through guided reasoning  
- Providing step-by-step Socratic hints for SQL problems  
- Assisting debugging and conceptual clarification  

### Important Constraint
The models are intentionally trained to:
- ✅ Provide reasoning steps and conceptual hints  
- ❌ Avoid generating complete SQL solutions  

This ensures alignment with pedagogical goals such as scaffolding and learner engagement.

---

## Training Data: SQL-Instruct Corpus

We construct **SQL-Instruct**, a domain-specific Socratic instruction corpus, by mining high-quality interactions from Stack Overflow. This platform captures real-world misconceptions, debugging challenges, and conceptual gaps encountered by learners and practitioners.

### Data Collection

To ensure high-quality instructional signals, we filter SQL-tagged questions based on community impact. The resulting dataset has:

- **1.27 billion total views**  
- **128,535 average views per question**  

For each selected entry, we extract:
- Problem descriptions  
- User-submitted SQL attempts  
- Executable SQL from accepted solutions  

This yields **9,916 unique questions**.

---

### Socratic Augmentation

Each example is transformed into a Socratic instructional format using GPT-4o, which generates:

- Guided reasoning steps  
- Conceptual hints  
- Question decomposition  

This ensures the dataset emphasizes **instructional scaffolding rather than answer generation**.

---

### Dataset Composition

- **Intermediate questions:** 8,604  
- **Advanced questions:** 629  
- **Debugging tasks:** 531  

The dataset emphasizes challenging reasoning scenarios, particularly:

- JOIN operations  
- Aggregations and grouping  
- Query optimization  

We further ensure reliability by selecting entries with a **median Stack Overflow score of 27**.

---

## Training Procedure

### Phase 2: Fine-Tuning

We apply **Full Fine-Tuning (FFT)** on small, open-source LLMs under pedagogical constraints designed to:

- Encourage conceptual scaffolding  
- Promote step-by-step reasoning  
- Discourage direct SQL answer generation  

---

## Evaluation

### Phase 3 Metrics

Models are evaluated using:

- **BERTScore** → semantic alignment with expected reasoning  
- **ROUGE-L** → detection of answer leakage (i.e., unintended full SQL generation)  

---

## Key Contributions

- Socratic SQL instruction tuning for higher education  
- SQL-Instruct dataset derived from real-world misconceptions  
- Multi-model fine-tuning across Phi-3, Qwen2.5, and Gemma2  
- Evaluation framework balancing reasoning quality and answer leakage  
- Ablation study identifying factors enabling:
  - Misconception-based feedback  
  - Iterative guidance  
  - Instructor-like reasoning behavior  

---

## Limitations

- Models may still occasionally generate partial SQL fragments  
- Evaluation focuses on semantic similarity rather than full pedagogical outcomes  
- Dataset is derived from Stack Overflow and may reflect community biases  

---

## Ethical Considerations

These models are designed to support learning, not replace it. By avoiding full solution generation, they aim to:

- Encourage critical thinking  
- Reduce over-reliance on AI-generated answers  
- Support equitable access to SQL learning resources  

---

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("sriram882004/SQL-Socratic-Models/phi3")
tokenizer = AutoTokenizer.from_pretrained("sriram882004/SQL-Socratic-Models/phi3")