stanford-nlpxed/exclusionary_isolation_model

Model Card for Exclusionary Consequence -- Out-of-Class Isolation Model

Model Details

• Developed by: Mei Tan, EduNLP Lab @ Stanford University Graduate School of Education
• Release Date: 2025-11-17
• Paper: Tan, Mei, and Dorottya Demszky. (2025). Do As I Say: What Teachers’ Language Reveals About Classroom Management Practices. (EdWorkingPaper: 23-844). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/9yj6-jn52

Model Description

This model is a RoBERTa-base classifier fine-tuned to predict binary labels from teacher utterances in classroom transcripts. It was trained on 5720 annotated teacher utterances from elementary math classroom transcripts from the NCTE dataset [1]. It is intended for research on teachers' classroom discourse.

The model classifies whether a teacher utterance is an instance of out-of-class isolation language. Out-of-class isolation is defined as a subset of material sanctions that involve removing the student from the classroom space entirely, such assending them to the hallway or to another room with a non-instructional adult. This is a subset of exclusionary consequences. Examples include: “Student D, and Student E, please go outside my door”; “Student C, you can go out in the hall”; “If this continues, you’re going to be asked to leave”

Note: Another form of isolation, in-class isolation, is also a subset of exclusionary consequence. Because this model performed less well than others (F1 ~0.75), we do not publish it. However, an estimate of this category can be found when an utterance is classified as involving an exclusionary consequence but not out-of-class isolation nor calling home.

Intended Uses

Not intended for evaluation of teaching quality. What is appropriate in a given classroom is highly contextual and relational in a way that this model does not capture.

Data Formatting

The expected input is a formatted sequence of a single teacher utterance. Example: "Student D, put that away."

Generalizability

The training data for this model come from ~200 observations sampled from the original NCTE study [2], which represents 1652 includes observations of 317 fourth- and fifth-grade mathematics classrooms across 53 schools in New England that were primarily serving low-income students of color. The utterances in this dataset are roughly sentence-length and human-transcribed.

Applying this model to new datasets generalizing to other contexts should involve validation: annotate a sample from the new data context to assess model generalizability.

[1] Demszky, D., & Hill, H. (2023). The NCTE Transcripts: A Dataset of Elementary Math Classroom Transcripts. In 18th Workshop on Innovative Use of NLP for Building Educational Applications.

[2] Kane, Thomas, Hill, Heather, and Staiger, Douglas. National Center for Teacher Effectiveness Main Study. Inter-university Consortium for Political and Social Research [distributor], 2022-06-16. https://doi.org/10.3886/ICPSR36095.v4