metadata
license: apache-2.0
tags:
- socratic-tutoring
- ai-education
- pedagogy
- dpo
- sft
Aligning Open Language Models for Socratic AI Pedagogy
Official repository for the EAAI paper: Aligning Open Language Models for Socratic AI Pedagogy via Preference Optimization.
📦 Project Artifacts
- 🧠 Socratic Muse-30B Model: NextGenInstitute/socraticMuse30B
- 🦙 Socratic Llama-8B Model: NextGenInstitute/socraticLlama8B
- 📚 1,680 Preference Dataset: NextGenInstitute/socraticDataset1680
📊 Benchmark Summary (150 Held-Out AI Debugging Scenarios)
| Model Baseline | Direct Leakage % (↓) | Pedagogical Utility (1-5) (↑) | Conceptual Accuracy % (↑) |
|---|---|---|---|
| Gemini 3.5 Flash | 0.0% | 4.79 / 5.0 | 98.7% |
| GPT-5.4-mini | 0.0% | 4.67 / 5.0 | 98.7% |
| Socratic Muse-30B (SFT+DPO) | 0.0% | 4.75 / 5.0 | 90.0% |
| Socratic Llama-8B (SFT+DPO) | 0.0% | 3.54 / 5.0 | 76.0% |
| Base Llama-3.1-8B-Instruct | 1.3% | 2.55 / 5.0 | 20.0% |
| Qwen2.5-Coder-7B-Instruct | 6.0% | 2.37 / 5.0 | 20.0% |