File size: 1,326 Bytes
732cc9c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | ---
license: apache-2.0
tags:
- socratic-tutoring
- ai-education
- pedagogy
- dpo
- sft
---
# Aligning Open Language Models for Socratic AI Pedagogy
Official repository for the EAAI paper: **Aligning Open Language Models for Socratic AI Pedagogy via Preference Optimization**.
## 📦 Project Artifacts
* 🧠 **Socratic Muse-30B Model:** [NextGenInstitute/socraticMuse30B](https://huggingface.co/NextGenInstitute/socraticMuse30B)
* 🦙 **Socratic Llama-8B Model:** [NextGenInstitute/socraticLlama8B](https://huggingface.co/NextGenInstitute/socraticLlama8B)
* 📚 **1,680 Preference Dataset:** [NextGenInstitute/socraticDataset1680](https://huggingface.co/datasets/NextGenInstitute/socraticDataset1680)
---
## 📊 Benchmark Summary (150 Held-Out AI Debugging Scenarios)
| Model Baseline | Direct Leakage % (↓) | Pedagogical Utility (1-5) (↑) | Conceptual Accuracy % (↑) |
| :--- | :---: | :---: | :---: |
| **Gemini 3.5 Flash** | **0.0%** | **4.79 / 5.0** | **98.7%** |
| **GPT-5.4-mini** | **0.0%** | **4.67 / 5.0** | **98.7%** |
| **Socratic Muse-30B (SFT+DPO)** | **0.0%** | **4.75 / 5.0** | **90.0%** |
| **Socratic Llama-8B (SFT+DPO)** | **0.0%** | **3.54 / 5.0** | **76.0%** |
| Base Llama-3.1-8B-Instruct | 1.3% | 2.55 / 5.0 | 20.0% |
| Qwen2.5-Coder-7B-Instruct | 6.0% | 2.37 / 5.0 | 20.0% |
|