README.md

metadata

title: Smart MCQ Solver
emoji: 🧠
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 5.9.1
app_file: app.py
pinned: false
license: mit
short_description: DeBERTa-v3-large ranking five candidate MCQ answers

Smart MCQ Solver

Five-option multiple-choice question answering over science and philosophy, scored by MAP@3. Fine-tuned microsoft/deberta-v3-large used as an AutoModelForMultipleChoice cross-encoder.

Metric	Value
3-fold grouped CV MAP@3	0.7567
Random MAP@3 baseline	0.3667
Leakage-free project estimate	0.6817

Set the MODEL_ID Space variable to your model repo, e.g. your-username/smart-mcq-deberta-v3-large.

How it works

Each of the five options is paired with the question to form five (question, option) sequences of shape (5, L). The encoder scores each pair independently; a linear head reduces each to one logit; the five logits are reshaped to (1, 5) and softmaxed, so the options compete in a single distribution. The top three, in order, are the MAP@3 submission.

Limitations

Trained on 2,000 rows covering 252 unique questions. Closed-book — no retrieval, no citations, and no abstention mechanism, so it ranks five options confidently regardless of whether it knows the topic.