README.md

metadata

base_model: roberta-base
library_name: transformers
license: mit
pipeline_tag: text-classification
metrics:
  - accuracy
tags:
  - generated_from_trainer
model-index:
  - name: roberta-sentence-classifier
    results: []

roberta-sentence-classifier

This model is a fine-tuned version of roberta-base presented in the paper Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction.

It serves as the sentence-level cognitive episode tagger in the Epi2Diff (Episode to Difficulty) framework. It maps Large Reasoning Model (LRM) reasoning traces into cognitively grounded episode sequences to support interpretable modeling of human item difficulty.

• Repository: c-steve-wang/Epi2Diff
• Paper: Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction

Model description

The model classifies sentence-level reasoning units into 8 problem-solving cognitive episode states:

• Read
• Analyze
• Plan
• Implement
• Explore
• Verify
• Monitor
• Answer

These classified sequences are then used by the Epi2Diff framework to extract compact episode-dynamic process features for downstream item difficulty prediction.

Intended uses & limitations

You can use this model to segment and tag raw reasoning traces into functional problem-solving states to evaluate reasoning behaviors, perform interpretability studies, or support downstream educational measurement tasks.

Training and evaluation data

The model was fine-tuned on annotated reasoning trace sentences derived from datasets such as SAT Math, SAT Reading & Writing, Cambridge, and USMLE.

It achieves the following results on the evaluation set:

• Loss: 0.6266
• Accuracy: 0.7990
• Macro F1: 0.7614
• Micro F1: 0.7990
• Qwk: 0.6588

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Macro F1	Micro F1	Qwk
0.6267	1.0	27540	0.6108	0.7818	0.7364	0.7818	0.6352
0.5539	2.0	55080	0.5939	0.7911	0.7498	0.7911	0.6428
0.475	3.0	82620	0.6021	0.7977	0.7592	0.7977	0.6599
0.4204	4.0	110160	0.6266	0.7990	0.7614	0.7990	0.6588

Framework versions

• Transformers 4.57.3
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.1

Citation

If you use this model, please cite:

@misc{wang2026epi2diff,
  title = {Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction},
  author = {Wang, Chenguang and Li, Ming and Zeng, Xinyue and Li, Zhuochun and Jiao, Hong and Zhou, Tianyi and Zhou, Dawei},
  year = {2026}
}