Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- socratic-tutoring
|
| 5 |
+
- ai-education
|
| 6 |
+
- pedagogy
|
| 7 |
+
- dpo
|
| 8 |
+
- sft
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Aligning Open Language Models for Socratic AI Pedagogy
|
| 12 |
+
|
| 13 |
+
Official repository for the EAAI paper: **Aligning Open Language Models for Socratic AI Pedagogy via Preference Optimization**.
|
| 14 |
+
|
| 15 |
+
## 📦 Project Artifacts
|
| 16 |
+
|
| 17 |
+
* 🧠**Socratic Muse-30B Model:** [NextGenInstitute/socraticMuse30B](https://huggingface.co/NextGenInstitute/socraticMuse30B)
|
| 18 |
+
* 🦙 **Socratic Llama-8B Model:** [NextGenInstitute/socraticLlama8B](https://huggingface.co/NextGenInstitute/socraticLlama8B)
|
| 19 |
+
* 📚 **1,680 Preference Dataset:** [NextGenInstitute/socraticDataset1680](https://huggingface.co/datasets/NextGenInstitute/socraticDataset1680)
|
| 20 |
+
|
| 21 |
+
---
|
| 22 |
+
|
| 23 |
+
## 📊 Benchmark Summary (150 Held-Out AI Debugging Scenarios)
|
| 24 |
+
|
| 25 |
+
| Model Baseline | Direct Leakage % (↓) | Pedagogical Utility (1-5) (↑) | Conceptual Accuracy % (↑) |
|
| 26 |
+
| :--- | :---: | :---: | :---: |
|
| 27 |
+
| **Gemini 3.5 Flash** | **0.0%** | **4.79 / 5.0** | **98.7%** |
|
| 28 |
+
| **GPT-5.4-mini** | **0.0%** | **4.67 / 5.0** | **98.7%** |
|
| 29 |
+
| **Socratic Muse-30B (SFT+DPO)** | **0.0%** | **4.75 / 5.0** | **90.0%** |
|
| 30 |
+
| **Socratic Llama-8B (SFT+DPO)** | **0.0%** | **3.54 / 5.0** | **76.0%** |
|
| 31 |
+
| Base Llama-3.1-8B-Instruct | 1.3% | 2.55 / 5.0 | 20.0% |
|
| 32 |
+
| Qwen2.5-Coder-7B-Instruct | 6.0% | 2.37 / 5.0 | 20.0% |
|