|
|
--- |
|
|
tags: |
|
|
- model_hub_mixin |
|
|
--- |
|
|
|
|
|
# Model Card for `abdou-u/MNLP_M3_w4a8_quantized_mcqa_model` |
|
|
|
|
|
## Summary |
|
|
|
|
|
This model is a W4A8 (4-bit weights, 8-bit activations) quantized version of the `mgatti/MNLP_M3_mcqa_model`, obtained using [Optimum-Quanto](https://huggingface.co/docs/optimum/main/en/quanto/index). It has been pushed to the Hugging Face Hub using the `PyTorchModelHubMixin` interface. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
|
|
|
- **Name**: MNLP_M3_w4a8_quantized_mcqa_model |
|
|
- **Source model**: `mgatti/MNLP_M3_mcqa_model` |
|
|
- **Quantization**: Optimum-Quanto W4A8 (qint4 weights, qint8 activations) |
|
|
- **Usage**: Efficient inference for multiple-choice question answering (MCQA) tasks |
|
|
- **Developer**: Ahmed Abdelmalek, EPFL CS-552 2025 Project M3 |
|
|
- **License**: MIT |
|
|
- **Language(s)**: English |
|
|
- **Hardware target**: Consumer and cloud GPUs with low memory footprint |
|
|
|
|
|
### Model Sources |
|
|
|
|
|
- **Repository**: *Private GitHub (Training script not public)* |
|
|
- **Paper**: Not published |
|
|
- **Docs**: This README |
|
|
|
|
|
## Use Cases |
|
|
|
|
|
### Direct Use |
|
|
|
|
|
This model is optimized for fast inference in MCQA tasks under constrained VRAM settings. |
|
|
|
|
|
### Intended Users |
|
|
|
|
|
Researchers and engineers looking to deploy a small, high-performance MCQA model. |
|
|
|
|
|
## Limitations |
|
|
|
|
|
This model is quantized and may have a slight performance drop compared to full-precision models. It is not suitable for generation or tasks beyond MCQA. |
|
|
|
|
|
## Getting Started |
|
|
|
|
|
```python |
|
|
from transformers import AutoTokenizer |
|
|
from optimum.quanto.models import QuantizedModelForCausalLM |
|
|
|
|
|
model = QuantizedModelForCausalLM.from_pretrained("abdou-u/MNLP_M3_w4a8_quantized_mcqa_model") |
|
|
tokenizer = AutoTokenizer.from_pretrained("abdou-u/MNLP_M3_w4a8_quantized_mcqa_model") |
|
|
``` |
|
|
|
|
|
## Technical Specifications |
|
|
|
|
|
- **Quantization library**: Optimum-Quanto |
|
|
- **Weights**: 4-bit (qint4) |
|
|
- **Activations**: 8-bit (qint8) |
|
|
- **Format**: Hugging Face Transformers-compatible |
|
|
|
|
|
## Environmental Impact |
|
|
|
|
|
- **Hardware**: A100 80GB (used during validation) |
|
|
- **Quantization**: 1 pass, full model (approx. 3 mins) |
|
|
- **Carbon Emissions**: Negligible for quantization |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you use this model, please cite: |
|
|
|
|
|
``` |
|
|
@misc{abdelmalek2025mnlp, |
|
|
title={MNLP M3 Quantized MCQA Model (W4A8)}, |
|
|
author={Ahmed Abdelmalek}, |
|
|
year={2025}, |
|
|
howpublished={\url{https://huggingface.co/abdou-u/MNLP_M3_w4a8_quantized_mcqa_model}}, |
|
|
note={CS-552 Project M3} |
|
|
} |
|
|
``` |
|
|
|
|
|
## Contact |
|
|
|
|
|
Ahmed Abdelmalek - ahmed.abdelmalek@epfl.ch |
