README.md

metadata

title: Smart MCQ Solver
emoji: 🧠
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 6.22.0
app_file: app.py
pinned: false
license: mit
suggested_hardware: cpu-basic
short_description: Rank the top three answers to a five-option MCQ (MAP@3)

Smart MCQ Solver

Live demo for the Deep Learning & Generative AI project (Diploma level, BS in Data Science and Applications, IIT Madras).

Devang Khirsaria · 23f1001439 · Term T2-2026

What this does

You give it a multiple-choice question and five options labelled A to E. It scores all five and returns the top three in ranked order, which is the submission format of the Kaggle Smart MCQ Solver Challenge. The competition metric is MAP@3, so a correct answer at rank 1 scores 1.000, at rank 2 scores 0.500 and at rank 3 scores 0.333.

What model is running

This Space runs cross-encoder/nli-deberta-v3-small zero-shot — Model 2 from the project. The question is used as the NLI premise, each option as a hypothesis, and the options are ranked by entailment probability. The entailment class index is read from model.config.label2id rather than hard-coded, because label ordering is not consistent across NLI checkpoints.

There is no fine-tuning and no trained weight file behind this demo. The project's best Kaggle score, 0.736 MAP@3, came from an eight-member fine-tuned DeBERTa-v3 multiple-choice LoRA ensemble, which is not what is deployed here. On the project's own validation split this zero-shot model scored 0.528 MAP@3, so its answers are noticeably weaker than the leaderboard number.

Notes

• The model is downloaded on the first prediction, so the first run takes roughly a minute. Later runs take a couple of seconds on the free CPU tier.
• Two worked examples are pre-loaded via the buttons under the form.
• Empty fields are reported back as a message rather than raising an error.

Source

Full project code: https://github.com/DevangK18/23f1001439-dl-genai-project-26-t2