| # MNLP_M3_mcqa_model_optimized | |
| This model is a fine-tuned version of Qwen/Qwen3-0.6B-Base for Multiple Choice Question Answering (MCQA). | |
| ## Training Details | |
| - **Base model**: Qwen/Qwen3-0.6B-Base | |
| - **Task**: Multiple Choice Question Answering | |
| - **Training dataset**: aymanbakiri/MNLP_M2_mcqa_dataset | |
| - **Training approach**: Multi-stage training with curriculum learning | |
| - **LoRA config**: r=128, alpha=256 | |
| - **Training stages**: | |
| - Stage 1: 4 epochs, LR=8e-5 | |
| - Stage 2: 6 epochs, LR=3e-5 | |
| - Stage 3: 2 epochs, LR=1e-5 | |
| ## Usage | |
| ```python | |
| from transformers import AutoTokenizer, AutoModelForCausalLM | |
| from peft import PeftModel | |
| model = AutoModelForCausalLM.from_pretrained("MNLP_M3_mcqa_model_optimized") | |
| tokenizer = AutoTokenizer.from_pretrained("MNLP_M3_mcqa_model_optimized") | |
| # Example inference | |
| question = "What is the capital of France?" | |
| choices = ["London", "Berlin", "Paris", "Madrid"] | |
| prompt = f"Question: {question}\nA) {choices[0]}\nB) {choices[1]}\nC) {choices[2]}\nD) {choices[3]}\nAnswer:" | |
| inputs = tokenizer(prompt, return_tensors="pt") | |
| outputs = model.generate(**inputs, max_new_tokens=5, temperature=0.1) | |
| answer = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True).strip() | |
| ``` | |