README.md

# MCQ Generation Model

This model is fine-tuned on the RACE dataset for generating multiple-choice questions. It is based on Mistral-Nemo-Base-2407 and uses unsloth optimizations.

## Model Details
- Base Model: unsloth/Mistral-Nemo-Base-2407
- Task: Multiple Choice Question Generation
- Training Data: RACE dataset
- Optimization: unsloth LoRA fine-tuning

## Usage
```python
from transformers import AutoTokenizer
from peft import AutoPeftModelForCausalLM

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("kenzykhaled/Question_generator_Mistral")

# Load model
model = AutoPeftModelForCausalLM.from_pretrained(
    "kenzykhaled/Question_generator_Mistral",
    device_map="auto",
    load_in_4bit=True
)

# Prepare your input
text = """
Generate a multiple-choice question (MCQ) based on the passage, provide options, and indicate the correct option.

Passage: [Your passage here]
"""

# Generate MCQ
inputs = tokenizer(text, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=128)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)
```

## Training Details
- LoRA rank: 16
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Training dataset: RACE (all)
- Training framework: unsloth + transformers