# MNLP_M3_mcqa_model_optimized

This model is a fine-tuned version of Qwen/Qwen3-0.6B-Base for Multiple Choice Question Answering (MCQA).

## Training Details

- **Base model**: Qwen/Qwen3-0.6B-Base
- **Task**: Multiple Choice Question Answering
- **Training dataset**: aymanbakiri/MNLP_M2_mcqa_dataset
- **Training approach**: Multi-stage training with curriculum learning
- **LoRA config**: r=128, alpha=256
- **Training stages**:
  - Stage 1: 4 epochs, LR=8e-5
  - Stage 2: 6 epochs, LR=3e-5  
  - Stage 3: 2 epochs, LR=1e-5

## Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained("MNLP_M3_mcqa_model_optimized")
tokenizer = AutoTokenizer.from_pretrained("MNLP_M3_mcqa_model_optimized")

# Example inference
question = "What is the capital of France?"
choices = ["London", "Berlin", "Paris", "Madrid"]

prompt = f"Question: {question}\nA) {choices[0]}\nB) {choices[1]}\nC) {choices[2]}\nD) {choices[3]}\nAnswer:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=5, temperature=0.1)
answer = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True).strip()
```