| license: apache-2.0 | |
| tags: | |
| - socratic-tutoring | |
| - ai-education | |
| - pedagogy | |
| - dpo | |
| - sft | |
| # Aligning Open Language Models for Socratic AI Pedagogy | |
| Official repository for the EAAI paper: **Aligning Open Language Models for Socratic AI Pedagogy via Preference Optimization**. | |
| ## 📦 Project Artifacts | |
| * 🧠 **Socratic Muse-30B Model:** [NextGenInstitute/socraticMuse30B](https://huggingface.co/NextGenInstitute/socraticMuse30B) | |
| * 🦙 **Socratic Llama-8B Model:** [NextGenInstitute/socraticLlama8B](https://huggingface.co/NextGenInstitute/socraticLlama8B) | |
| * 📚 **1,680 Preference Dataset:** [NextGenInstitute/socraticDataset1680](https://huggingface.co/datasets/NextGenInstitute/socraticDataset1680) | |
| --- | |
| ## 📊 Benchmark Summary (150 Held-Out AI Debugging Scenarios) | |
| | Model Baseline | Direct Leakage % (↓) | Pedagogical Utility (1-5) (↑) | Conceptual Accuracy % (↑) | | |
| | :--- | :---: | :---: | :---: | | |
| | **Gemini 3.5 Flash** | **0.0%** | **4.79 / 5.0** | **98.7%** | | |
| | **GPT-5.4-mini** | **0.0%** | **4.67 / 5.0** | **98.7%** | | |
| | **Socratic Muse-30B (SFT+DPO)** | **0.0%** | **4.75 / 5.0** | **90.0%** | | |
| | **Socratic Llama-8B (SFT+DPO)** | **0.0%** | **3.54 / 5.0** | **76.0%** | | |
| | Base Llama-3.1-8B-Instruct | 1.3% | 2.55 / 5.0 | 20.0% | | |
| | Qwen2.5-Coder-7B-Instruct | 6.0% | 2.37 / 5.0 | 20.0% | | |