additional files/HOW_TO_RUN.md

How to Run: Tamil MCQ Generator

This guide explains the directory structure of the project and guides you through running the MCQ generator locally to test the model.

📂 Directory Structure

Quiz_MCQ_model/
├── data/                             # Raw corpus and text data files
├── models/                           # Trained machine learning models (.bin fasttext model, Hugging Face models)
├── nlp/                              # Core NLP tools
│   ├── ner_tagger.py                 # Hugging Face Named Entity Recognition
│   └── pos_tagger.py                 # Stanza POS Tagging
├── preprocessing/                    # Text sanitization pipeline
│   ├── cleaner.py                    # Cleans texts of unwanted characters
│   ├── data_loader.py                # Loads zips and txt corpus files
│   ├── process_pipeline.py           # Orchestrates the validation and tokenization
│   └── tokenizer.py                  # Sentence and word tokenizers
├── question_generation/              # The MCQ Logic
│   ├── candidate_selector.py         # Selects the best word to be the answer
│   ├── domain_vocab.py               # Pre-defined vocabularies by subject
│   ├── option_generator.py           # Uses FastText to generate wrong answers (distractors)
│   ├── question_builder.py           # Masks the answer to build the question
│   └── test_model.py                 # The MAIN script to run the pipeline
├── prepare_corpus.py                 # Script to prepare the 'data/' folder
├── tamil_mcq_clustering.py           # Script to cluster words offline (HDBSCAN)
├── train_fasttext.py                 # Script to train your own FastText embeddings
├── requirements.txt                  # Python dependencies
├── README.md                         # Project overview
└── HOW_TO_RUN.md                     # This file

---

💻 Running the Model

Step 1: Install Dependencies

Open a terminal in the root directory and install the required Python packages:

pip install -r requirements.txt

Step 2: Preparing the Offline Data (Optional)

Note: If models/tamil_fasttext.bin and the cluster JSON files are already built and present, you can skip this step.

If you need to train the model from scratch:

1. Run python prepare_corpus.py to compile the corpus.
2. Run python train_fasttext.py to create the word embeddings.
3. Run python tamil_mcq_clustering.py to generate the word clusters.

Step 3: Checking the Model (The Main Entry Point)

To observe the MCQ Generation happening live on a sample Tamil text passage, you run the test_model.py script located in the question_generation folder.

Run the following command from the root directory:

python question_generation/test_model.py

What to Expect:

When you run test_model.py, the code will pass a sample paragraph (e.g., about the Solar System) through the entire pipeline. The terminal output will display:

• The original sentence.
• The formatted MCQ sentence with the blank "_____".
• 4 generated options (including the correct answer and 3 smart distractors).
• The correct answer extracted below.