---
license: apache-2.0
tags:
  - onnx
  - int8
  - quantized
  - sentence-similarity
  - embeddings
  - justembed
base_model: sentence-transformers/all-mpnet-base-v2
library_name: onnxruntime
pipeline_tag: feature-extraction
---

# MPNet INT8 — ONNX Quantized

ONNX INT8 quantized version of [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) for efficient general-purpose sentence embeddings.

## Model Details

| Property | Value |
|----------|-------|
| Base Model | [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) |
| Format | ONNX |
| Quantization | INT8 (dynamic quantization) |
| Embedding Dimension | 768 |
| Quantized by | [JustEmbed](https://pypi.org/project/justembed/) |

## What is this?

This is a quantized ONNX export of all-mpnet-base-v2, one of the best general-purpose sentence embedding models from the sentence-transformers library. It maps sentences and paragraphs to a 768-dimensional dense vector space. The INT8 quantization reduces model size and improves inference speed while maintaining high accuracy.

## Use Cases

- Semantic text search
- Sentence similarity
- Clustering and topic modeling
- Paraphrase detection
- General-purpose text embeddings

## Files

- `model_quantized.onnx` — INT8 quantized ONNX model
- `tokenizer.json` — Fast tokenizer
- `vocab.txt` — Vocabulary file
- `config.json` — Model configuration

## Usage with JustEmbed

```python
from justembed import Embedder

embedder = Embedder("mpnet-int8")
vectors = embedder.embed(["This is a sentence", "This is another sentence"])
```

## Usage with ONNX Runtime

```python
import onnxruntime as ort
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(".")
session = ort.InferenceSession("model_quantized.onnx")

inputs = tokenizer("This is a sentence", return_tensors="np")
outputs = session.run(None, dict(inputs))
```

## Quantization Details

- Method: Dynamic INT8 quantization via ONNX Runtime
- Source: Original PyTorch weights converted to ONNX, then quantized
- Speed: ~2-3x faster inference than FP32
- Size: ~4x smaller than FP32

## License

This model is a derivative work of [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2).

The original model is licensed under **Apache License 2.0**. This quantized version is distributed under the same license. See the [LICENSE](LICENSE) file for the full text.

## Citation

```bibtex
@inproceedings{song2020mpnet,
  title={MPNet: Masked and Permuted Pre-training for Language Understanding},
  author={Song, Kaitao and Tan, Xu and Qin, Tao and Lu, Jianfeng and Liu, Tie-Yan},
  booktitle={NeurIPS},
  year={2020}
}
```

## Acknowledgments

- Original model by [UKP Lab / sentence-transformers](https://www.sbert.net/)
- Quantization and packaging by [JustEmbed](https://pypi.org/project/justembed/)