README.md

---
library_name: transformers
tags: []
---
# Model Card for `zay25/MNLP_M3_quantized_model`

This model is a quantized version of a multiple-choice question answering (MCQA) model fine-tuned on STEM datasets. It uses Activation-aware Weight Quantization (AWQ) to reduce model size and VRAM usage while preserving strong performance. The model is well-suited for memory- and latency-constrained environments.

---

## Model Details

- **Developed by**: Zeineb Mellouli (EPFL, CS-552 Project)
- **Base model**: `hssawhney/Best-Performing-Model` (Qwen3-0.6B-Base)
- **Quantization**: AWQ (4-bit weights, 16-bit activations)
- **Architecture**: Transformer-based Causal Language Model
- **Language**: English
- **License**: Apache 2.0

---

## Uses

### Direct Use

This model is intended for multiple-choice question answering (MCQA) tasks, particularly in science, math, and engineering education datasets. It is optimized for inference on GPUs with limited VRAM (e.g., A10, T4, or laptop GPUs).

### Out-of-Scope Use

- Not intended for open-ended or dialog generation
- Not suitable for high-stakes decision-making or critical applications without human oversight


## Training Details

- **Quantization method**: Post-training quantization using [AWQ (Activation-aware Weight Quantization)](https://github.com/mit-han-lab/awq) via the `awq` library
- **Base model**: `hssawhney/Best-Performing-Model`, fine-tuned on MCQA-style reasoning tasks
- **Quantization configuration**:
  - 4-bit weights (`w_bit = 4`)
  - Group size: 64
  - Per-channel zero point: enabled
- **Calibration dataset**: 512 samples from `hssawhney/Reasoning-Dataset`

---

## How to Use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("zay25/MNLP_M3_quantized_model", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("zay25/MNLP_M3_quantized_model")