data/artifacts/models/recommender/model_card.md

Model Card: Recommender

Model Details

• Model Name: recommender
• Model Version: recommender_v2_baseline_001
• Algorithm: GradientBoostingClassifier (two models: clicked, is_completed)
• Framework: scikit-learn
• Trained At: 2026-05-21T05:59:13.148420+00:00
• Seed: 42

Intended Use

Predict whether a student will click on a recommendation and whether they will complete the recommended content. Used in the recommendation engine to rank content by predicted engagement. Two separate binary classifiers are trained: one for clicked and one for is_completed.

Training Data

• Source: training_recommendation_outcomes.csv (synthetic dataset v2)
• Split Counts: train=5671, validation=1214, test=1215
• Features: priority (OrdinalEncoded), ai_confidence (numeric), recommendation_type (OrdinalEncoded), grade (numeric), subject (OrdinalEncoded)
• Targets: clicked (binary), is_completed (binary)

Metrics

Validation Set

• ROC-AUC (clicked): 0.5439
• ROC-AUC (is_completed): 0.5303
• Lift@10 (clicked): 1.1392
• Lift@10 (is_completed): 1.1056

Test Set

• ROC-AUC (clicked): 0.5486
• ROC-AUC (is_completed): 0.5424
• Lift@10 (clicked): 1.0471
• Lift@10 (is_completed): 1.0366

Known Limitations

• Trained on synthetic data only — performance on real recommendation data is unknown.
• Two separate GBC models — no joint optimization of clicked + is_completed.
• OrdinalEncoder assumes an ordering for priority/recommendation_type/subject.
• Lift@10 depends on the distribution of positive labels in the dataset.
• No user-level features (e.g., engagement history) included in baseline.
• Limited feature set (5 features); adding student history could improve performance.

Fallback Behavior

When the model is not loaded or confidence is below threshold, the system falls back to knowledge-graph weakest-prerequisite + content_catalog filtered by LO + difficulty, ranked by estimated_mastery_gain.