# Load model directly
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("dcferreira/detoxify-optimized")
model = AutoModelForSequenceClassification.from_pretrained("dcferreira/detoxify-optimized")Quick Links
This repo has an optimized version of Detoxify, which needs less disk space and less memory at the cost of just a little bit of accuracy.
This is an experiment for me to learn how to use 🤗 Optimum.
Usage
Loading the model requires the 🤗 Optimum library installed.
from optimum.onnxruntime import ORTModelForSequenceClassification
from optimum.pipelines import pipeline as opt_pipeline
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("dcferreira/detoxify-optimized")
model = ORTModelForSequenceClassification.from_pretrained("dcferreira/detoxify-optimized")
pipe = opt_pipeline(
model=model,
task="text-classification",
function_to_apply="sigmoid",
accelerator="ort",
tokenizer=tokenizer,
top_k=None, # return scores for all the labels, model was trained as multilabel
)
print(pipe(['example text','exemple de texte','texto de ejemplo','testo di esempio','texto de exemplo','örnek metin','пример текста']))
Performance
The table below compares some statistics on running the original model, vs the original model with the onnxruntime, vs optimizing the model with onnxruntime.
| model | Accuracy (%) | Samples p/ second (CPU) | Samples p/ second (GPU) | GPU VRAM | Disk Space |
|---|---|---|---|---|---|
| original | 92.1083 | 16 | 250 | 3GB | 1.1GB |
| ort | 92.1067 | 19 | 340 | 4GB | 1.1GB |
| optimized (O4) | 92.1031 | 14 | 650 | 2GB | 540MB |
For details on how these numbers were reached, check out evaluate.py in this repo.
- Downloads last month
- 8
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="dcferreira/detoxify-optimized")