| --- |
| license: apache-2.0 |
| language: en |
| base_model: microsoft/deberta-v3-base |
| pipeline_tag: text-classification |
| tags: |
| - education |
| - tutoring |
| - misconception-detection |
| - k-12 |
| - math |
| --- |
| |
| # qlm-map-classifier |
|
|
| **A DeBERTa-v3 misconception-detection classifier for K-12 mathematics tutoring, in production at Quantum Learning Machines since April 2026.** It classifies student utterances and worked steps against a taxonomy of documented math misconceptions, and serves as an *observation model* feeding a Bayesian student-state layer — not as a standalone judge of student thinking. We publish it, with its honest numbers, because the field's progress depends on knowing what this class of model can and cannot do. |
|
|
| ## What it does in production |
|
|
| Given a student turn (utterance, answer, or worked step) in a math tutoring context, the model emits misconception-class probabilities over 34 production classes drawn from QLM's 481-construct misconception ontology (423 constructs public under CC-BY-4.0 via our export API). Its outputs are **not** used to label students directly. They enter a persistent Bayesian posterior alongside triage signals that distinguish misconception-driven errors from slips and disengagement; instructional decisions condition on the accumulated state, not on any single classifier call. |
|
|
| ## Evaluation — read the limitations first |
|
|
| Evaluated on an internal MathTutorBench-derived error-detection probe (unseen, GSM8K-based). All figures below are from versioned result files. |
|
|
| | Variant | Detection recall | Mistake-location accuracy | Notes | |
| |---|---|---|---| |
| | **MAP-34 (this model, deployed)** | **25.1%** | **19.0%** | conservative by design; low over-flagging | |
| | Combined-39 (research retrain, not deployed) | 86.2% | 16.3% | 8,207 mixed examples (EEDI public + expert-pattern-derived synthetic); **49.5% solution-correctness accuracy — severe over-flagging** | |
| | EEDI-only retrain (not deployed) | 72.1% | 15.7% | same over-flagging pattern | |
| | Bare LLM baseline (no classifier) | 78.3% | 14.7% (F1 micro) | flags nearly everything | |
|
|
| **The pattern that matters:** every high-recall variant achieves its recall by over-flagging — calling correct or merely-slipped work "misconception." The training corpora available to this task (misconception-labeled datasets, expert-authored trigger patterns) contain, by construction, **no negative space**: no slips, no disengagement, no correct-but-unconventional work. A classifier trained only on what misconceptions look like cannot learn what they *don't* look like. This is a data-ecology finding about the field, not an implementation artifact — and it is why the deployed configuration is conservative and why the architecture treats the classifier as one observation source among several. |
|
|
| ## The architecture lesson (ablation) |
|
|
| In controlled ablation, injecting this classifier's outputs directly into a tutor model's per-turn prompt **reduced** tutoring quality (solution-correctness evaluation: 60% → 55%). Classifier signal helps when integrated as evidence into persistent student state; it hurts when bolted onto generation. Build state layers, not prompt decorations. |
|
|
| ## Training data |
|
|
| Production K-12 tutoring interaction data (private; de-identified), labeled against the misconception ontology. We do not publish interaction counts or corpus composition. The ontology's public tier — 423 constructs with descriptions, diagnostic questions, and research citations — is available CC-BY-4.0 via the QLM export API. |
|
|
| ## Intended use |
|
|
| - Research on misconception detection, error-type triage, and tutoring-system evaluation |
| - As an observation component inside stateful student-modeling systems |
| - Reproducing and extending the negative-space findings above |
|
|
| ## Out-of-scope use |
|
|
| - Grading, placement, or any consequential decision about an individual student |
| - Standalone "misconception detector" products without state-level integration and human oversight |
| - Non-mathematics domains; languages other than English (untested) |
|
|
| ## Bias, risks, and limitations |
|
|
| Detection behavior has not been separately validated across demographic subgroups; measurement-invariance testing on authentic classroom data is planned, funded work. Low recall means many real misconceptions pass undetected; downstream systems must not treat absence of a flag as evidence of understanding. Trained on production interactions from QLM platforms; distribution shift to other tutoring contexts is untested. |
|
|
| ## Citation |
|
|
| @software{qlm_map_classifier_2026, |
| author = {Quantum Learning Machines}, |
| title = {qlm-map-classifier: misconception detection as an observation model}, |
| year = {2026}, |
| url = {https://huggingface.co/QuantumLearningMachines/qlm-map-classifier}, |
| license = {Apache-2.0} |
| } |
| |
| ## Contact: hello@quantumlearningmachines.com · quantumlearningmachines.com |
