README.md

---
library_name: peft
license: apache-2.0
base_model: Qwen/Qwen2.5-1.5B-Instruct
tags:
- base_model:adapter:Qwen/Qwen2.5-1.5B-Instruct
- llama-factory
- lora
- transformers
- question-generation
- education
- secondary-school
pipeline_tag: text-generation
model-index:
- name: question_generation_1.5B_model_v2
  results: []
---

# Question Generation 1.5B Model v2

A fine-tuned language model specifically designed to generate high-quality English comprehension and assessment questions for secondary school students. This model is optimized to create questions aligned with standard educational curricula and learning objectives.

## Model Description

This model is a LoRA (Low-Rank Adaptation) fine-tuned version of [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct). It has been trained specifically on educational question generation tasks to produce contextually relevant, pedagogically sound questions suitable for secondary school assessment.

### Key Features

- **Lightweight and Efficient**: 1.5B parameters with LoRA adaptation for fast inference
- **Education-Focused**: Trained on curated educational content
- **Curriculum-Aligned**: Questions follow standard secondary school curricula and learning outcomes
- **Question Variety**: Capable of generating multiple question types (multiple choice, short answer, essay prompts, etc.)
- **Context-Aware**: Generates questions based on provided text passages or topics

## Intended Use

This model is intended for:
- **Educational Content Creation**: Generating practice questions and assessments for secondary school students
- **Curriculum Support**: Creating supplementary learning materials aligned with educational standards
- **Assessment Design**: Assisting educators in developing comprehension questions and quiz content
- **Language Learning**: Generating English language proficiency assessment questions

### Limitations

- Designed for English language question generation
- Best performance on secondary school level content (ages 14-18)
- May require post-processing or human review for use in high-stakes assessments
- Performance may vary with non-English text inputs

## Training Data

The model was fine-tuned on a curated dataset of secondary school English curriculum materials and assessment question templates. Training data includes various question types aligned with standard educational frameworks.

## Training Procedure

### Hyperparameters

| Parameter | Value |
|-----------|-------|
| Learning Rate | 0.0005 |
| Training Batch Size | 8 (gradient accumulation) |
| Epochs | 10 |
| Optimizer | AdamW (fused) |
| LR Scheduler | Cosine with 0.1 warmup ratio |
| Seed | 42 |
| Training Precision | Native AMP (Mixed Precision) |

### Training Performance

The model achieved strong convergence with decreasing training loss across epochs:

| Epoch | Step | Training Loss |
|-------|------|---------------|
| 1.1   | 100  | 0.6345        |
| 2.3   | 200  | 0.4720        |
| 3.4   | 300  | 0.3499        |
| 4.5   | 400  | 0.2457        |
| 5.7   | 500  | 0.1229        |
| 6.8   | 600  | 0.0728        |
| 8.0   | 700  | 0.0398        |
| 9.1   | 800  | 0.0213        |

The model demonstrates consistent improvement in question generation quality as training progresses, with training loss decreasing from 0.63 to 0.02.

## Framework Versions

- PEFT: 0.17.1
- Transformers: 4.57.1
- PyTorch: 2.9.0+cu126
- Datasets: 4.0.0
- Tokenizers: 0.22.1

## Usage

```python
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model_id = "tokhey/question_generation_1.5B_model_v2"
model = AutoPeftModelForCausalLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Generate questions from a passage
prompt = "Generate 3 comprehension questions about: [your text passage]"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=512)
print(tokenizer.decode(outputs[0]))
```

## Recommendations for Use

- Test the model on sample content before deploying in production
- Review generated questions for accuracy and appropriateness
- Use as an assistive tool to reduce educator workload, not as a sole assessment creation method
- Provide context and learning materials with generated questions for optimal student engagement

## License

Apache License 2.0

---

*This model card was automatically generated and updated. For questions or contributions, please reach out to the model developers.*