README.md

---
language:
- en
license: mit
tags:
- xgboost
- sentence-transformers
- text-classification
- tabular
metrics:
- accuracy
- f1
pipeline_tag: text-classification
---

# Difficulty Classifier Model (XGBoost + Sentence Transformers)

This repository contains an XGBoost classification model designed to categorize question difficulty into three levels: **Easy**, **Medium**, and **Hard**.

## Architecture & Pipeline

1. **Text Enrichment:** Inputs are structured by combining the core `question_content`, any multiple choice `options` (formatted as a pipe-separated string), and associated `passage_text` context.
2. **Vector Space Embeddings:** The enriched question string is converted into a 384-dimensional dense vector using the `sentence-transformers/all-MiniLM-L6-v2` embedding engine.
3. **Tabular Classifier:** An XGBoost Classifier (`XGBClassifier`) predicts the difficulty class from the embedding vector.

## Model Files

* `difficulty_xgb_local_model.joblib`: The trained XGBoost model binary.
* `difficulty_label_encoder.joblib`: Scikit-Learn LabelEncoder mapping classes to integers (`['Easy', 'Hard', 'Medium']`).

## How to Use

You can load and use these assets directly using `joblib` and `sentence-transformers`:

```python
import joblib
import json
import pandas as pd
import numpy as np
from sentence_transformers import SentenceTransformer

# 1. Load the model, label encoder and embedder
model = joblib.load("difficulty_xgb_local_model.joblib")
label_encoder = joblib.load("difficulty_label_encoder.joblib")
embedder = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

# 2. Define single item payload
question = "The number of species having non-pyramidal shape is..."
options = ["CO3 2-", "SO3", "NO3-"]

# 3. Format inputs (match training pipeline schema)
options_text = " | ".join(options)
enriched_text = f"Question: {question} \n Options: {options_text}"

# 4. Generate embeddings and run inference
embedding = embedder.encode([enriched_text])
prediction_idx = model.predict(embedding)[0]
predicted_label = label_encoder.classes_[prediction_idx]

print("Predicted Difficulty:", predicted_label)
```