data/artifacts/models/mastery_model/model_card.md

Model Card: Mastery Model

Model Details

• Model Name: mastery_model
• Model Version: mastery_model_v2_baseline_001
• Algorithm: RandomForestClassifier
• Framework: scikit-learn
• Trained At: 2026-05-21T05:59:11.764140+00:00
• Seed: 42

Intended Use

Predict per-student per-LO mastery label (weak, developing, proficient, mastered) based on behavioral and performance features. Used in the mastery prediction endpoint to classify student mastery level for a given learning outcome.

Training Data

• Source: training_mastery_prediction.csv (synthetic dataset v2)
• Split Counts: train=24515, validation=5218, test=5187
• Features: attempt_count, accuracy, average_marks_ratio, average_time_seconds, hint_usage_rate, attendance_percentage, assignment_completion_rate, average_login_per_week, inactive_days_last_14 (all numeric)
• Target: mastery_label (integer 0-3, mapped to weak/developing/proficient/mastered)

Metrics

Validation Set

• Macro F1: 0.8649
• Weighted F1: 0.8952

Test Set

• Macro F1: 0.8754
• Weighted F1: 0.9029

Per-Class Performance (Test Set)

Class	Precision	Recall	F1	Support
weak	0.9667	0.9717	0.9692	2120
developing	0.9155	0.9089	0.9122	1394
proficient	0.7814	0.7555	0.7683	814
mastered	0.8395	0.865	0.8521	859

Known Limitations

• Trained on synthetic data only — performance on real student data is unknown.
• 4-class mastery labels derived from synthetic mastery_score thresholds.
• All features are numeric; no text or contextual features used.
• Class distribution may not reflect real-world mastery patterns.
• No encoding needed since all features are already numeric.

Fallback Behavior

When the model is not loaded or confidence is below the threshold (0.55), the system falls back to rule-based mastery estimation using mastery_score thresholds: <0.4 weak, <0.6 developing, <0.8 proficient, else mastered.