Spaces:
Sleeping
Sleeping
| title: Smart MCQ Solver | |
| emoji: 🧠 | |
| colorFrom: blue | |
| colorTo: indigo | |
| sdk: gradio | |
| sdk_version: 6.22.0 | |
| app_file: app.py | |
| pinned: false | |
| license: mit | |
| suggested_hardware: cpu-basic | |
| short_description: Rank the top three answers to a five-option MCQ (MAP@3) | |
| # Smart MCQ Solver | |
| Live demo for the Deep Learning & Generative AI project (Diploma level, BS in Data | |
| Science and Applications, IIT Madras). | |
| **Devang Khirsaria · 23f1001439 · Term T2-2026** | |
| ## What this does | |
| You give it a multiple-choice question and five options labelled A to E. It scores all | |
| five and returns the **top three in ranked order**, which is the submission format of the | |
| Kaggle *Smart MCQ Solver Challenge*. The competition metric is MAP@3, so a correct answer | |
| at rank 1 scores 1.000, at rank 2 scores 0.500 and at rank 3 scores 0.333. | |
| ## What model is running | |
| This Space runs **`cross-encoder/nli-deberta-v3-small` zero-shot** — Model 2 from the | |
| project. The question is used as the NLI premise, each option as a hypothesis, and the | |
| options are ranked by entailment probability. The entailment class index is read from | |
| `model.config.label2id` rather than hard-coded, because label ordering is not consistent | |
| across NLI checkpoints. | |
| There is **no fine-tuning and no trained weight file** behind this demo. The project's | |
| best Kaggle score, 0.736 MAP@3, came from an eight-member fine-tuned DeBERTa-v3 | |
| multiple-choice LoRA ensemble, which is not what is deployed here. On the project's own | |
| validation split this zero-shot model scored 0.528 MAP@3, so its answers are noticeably | |
| weaker than the leaderboard number. | |
| ## Notes | |
| - The model is downloaded on the first prediction, so the first run takes roughly a | |
| minute. Later runs take a couple of seconds on the free CPU tier. | |
| - Two worked examples are pre-loaded via the buttons under the form. | |
| - Empty fields are reported back as a message rather than raising an error. | |
| ## Source | |
| Full project code: `https://github.com/DevangK18/23f1001439-dl-genai-project-26-t2` | |