data/artifacts/models/difficulty_model/model_card.md

# Model Card: Difficulty Model

## Model Details

- **Model Name:** difficulty_model
- **Model Version:** difficulty_model_v2_baseline_001
- **Algorithm:** RandomForestRegressor
- **Framework:** scikit-learn
- **Trained At:** 2026-05-21T05:59:09.943332+00:00
- **Seed:** 42

## Intended Use

Estimate question difficulty as a continuous score in [0, 1] based on
question features (bloom_score, grade, subject, question_type). Used in
the difficulty estimation endpoint to predict how hard a question is for
a given grade level.

## Training Data

- **Source:** training_lo_tagging.csv + questions.csv (for question_type)
- **Split Counts:** train=3912, validation=1033, test=875
- **Features:** bloom_score (numeric), grade (numeric), subject (OrdinalEncoded), question_type (OrdinalEncoded)
- **Target:** difficulty_score (continuous [0, 1])

## Metrics

### Validation Set
- MAE: 0.3475
- R-squared: 0.5003
- Per-bucket MAE: {'easy': 0.3058, 'medium': 0.2934, 'hard': 0.6563}

### Test Set
- MAE: 0.3519
- R-squared: 0.4685
- Per-bucket MAE: {'easy': 0.325, 'medium': 0.2885, 'hard': 0.6797}

## Known Limitations

- Trained on synthetic data only — performance on real questions is unknown.
- difficulty_score distribution may not reflect real-world difficulty.
- OrdinalEncoder assumes an ordering that may not be meaningful for subject/question_type.
- Per-bucket MAE depends on the quality of the difficulty string labels.
- Limited feature set (4 features); text-based features could improve performance.

## Fallback Behavior

When the model is not loaded or confidence is below threshold, the system
falls back to a rule-based difficulty estimation using bloom_score and
grade-level heuristics.