README.md

---
library_name: transformers
tags: []
---

# Tutor Talk Moves Classifier (ModernBERT)

## Model Description

This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) that classifies tutor utterances based on teacher talk moves from [talkmoves.com](https://www.talkmoves.com/).
The model identifies whether a tutor's message represents one of the following three categories, or none of them:

- Classroom management
- Pressing for accuracy or reasoning
- Restating or revoicing

## Training Data

This model is trained on text of tutoring sessions from three different tutoring providers, annotated by two raters.

- **Inter-Rater Reliability with Krippendorff's Alpha**: 0.83
- **Total Training Examples**: 1,849
- **Class Distribution**:
| Class | Examples | Percentage |
|-------|----------|------------|
| None | 987 | 53.4% |
| Classroom management | 284 | 15.4% |
| Pressing for accuracy or reasoning | 474 | 25.6% |
| Restating or revoicing | 104 | 5.6% |

## Data Format

The is trained on utterances with the following format: `[PRETEXT] {3 previous messages} [TEXT] {target message}`,
where `[PRETEXT]` and `[TEXT]` are special tokens.
Names are anonymized, message text is lowercased, and leading and trailing whitespace is removed.

**Example**:
```
[PRETEXT] tutor: hello there [student]
tutor: what is the answer to this problem?
student: the answer is 6 [TEXT] tutor: why do you say the answer is 6?
```

## Performance

Test set results (264 examples):

| Class | Precision | Recall | F1-Score | Support |
|-------|-----------|--------|----------|---------|
| 0 - None | 0.9158 | 0.9062 | 0.9110 | 192 |
| 1 - Classroom Management | 0.6818 | 0.6818 | 0.6818 | 22 |
| 2 - Pressing for Accuracy or Reasoning | 0.7826 | 0.8571 | 0.8182 | 42 |
| 3 - Restating or Revoicing | 0.5000 | 0.3750 | 0.4286 | 8 |
| **Macro Average** | **0.7201** | **0.7051** | **0.7099** | **264** |
| **Weighted Average** | **0.8625** | **0.8636** | **0.8625** | **264** |