qlm-map-classifier

A DeBERTa-v3 misconception-detection classifier for K-12 mathematics tutoring, in production at Quantum Learning Machines since April 2026. It classifies student utterances and worked steps against a taxonomy of documented math misconceptions, and serves as an observation model feeding a Bayesian student-state layer — not as a standalone judge of student thinking. We publish it, with its honest numbers, because the field's progress depends on knowing what this class of model can and cannot do.

What it does in production

Given a student turn (utterance, answer, or worked step) in a math tutoring context, the model emits misconception-class probabilities over 34 production classes drawn from QLM's 481-construct misconception ontology (423 constructs public under CC-BY-4.0 via our export API). Its outputs are not used to label students directly. They enter a persistent Bayesian posterior alongside triage signals that distinguish misconception-driven errors from slips and disengagement; instructional decisions condition on the accumulated state, not on any single classifier call.

Evaluation — read the limitations first

Evaluated on an internal MathTutorBench-derived error-detection probe (unseen, GSM8K-based). All figures below are from versioned result files.

Variant	Detection recall	Mistake-location accuracy	Notes
MAP-34 (this model, deployed)	25.1%	19.0%	conservative by design; low over-flagging
Combined-39 (research retrain, not deployed)	86.2%	16.3%	8,207 mixed examples (EEDI public + expert-pattern-derived synthetic); 49.5% solution-correctness accuracy — severe over-flagging
EEDI-only retrain (not deployed)	72.1%	15.7%	same over-flagging pattern
Bare LLM baseline (no classifier)	78.3%	14.7% (F1 micro)	flags nearly everything

The pattern that matters: every high-recall variant achieves its recall by over-flagging — calling correct or merely-slipped work "misconception." The training corpora available to this task (misconception-labeled datasets, expert-authored trigger patterns) contain, by construction, no negative space: no slips, no disengagement, no correct-but-unconventional work. A classifier trained only on what misconceptions look like cannot learn what they don't look like. This is a data-ecology finding about the field, not an implementation artifact — and it is why the deployed configuration is conservative and why the architecture treats the classifier as one observation source among several.

The architecture lesson (ablation)

In controlled ablation, injecting this classifier's outputs directly into a tutor model's per-turn prompt reduced tutoring quality (solution-correctness evaluation: 60% → 55%). Classifier signal helps when integrated as evidence into persistent student state; it hurts when bolted onto generation. Build state layers, not prompt decorations.

Training data

Production K-12 tutoring interaction data (private; de-identified), labeled against the misconception ontology. We do not publish interaction counts or corpus composition. The ontology's public tier — 423 constructs with descriptions, diagnostic questions, and research citations — is available CC-BY-4.0 via the QLM export API.

Intended use

• Research on misconception detection, error-type triage, and tutoring-system evaluation
• As an observation component inside stateful student-modeling systems
• Reproducing and extending the negative-space findings above

Out-of-scope use

• Grading, placement, or any consequential decision about an individual student
• Standalone "misconception detector" products without state-level integration and human oversight
• Non-mathematics domains; languages other than English (untested)

Bias, risks, and limitations

Detection behavior has not been separately validated across demographic subgroups; measurement-invariance testing on authentic classroom data is planned, funded work. Low recall means many real misconceptions pass undetected; downstream systems must not treat absence of a flag as evidence of understanding. Trained on production interactions from QLM platforms; distribution shift to other tutoring contexts is untested.

Citation

@software{qlm_map_classifier_2026, author = {Quantum Learning Machines}, title = {qlm-map-classifier: misconception detection as an observation model}, year = {2026}, url = {https://huggingface.co/QuantumLearningMachines/qlm-map-classifier}, license = {Apache-2.0} }

Contact: hello@quantumlearningmachines.com · quantumlearningmachines.com