Create README.md

README.md

@@ -0,0 +1,54 @@
+---
+tags:
+- optimum
+datasets:
+- banking77
+metrics:
+- accuracy
+model-index:
+- name: quantized-distilbert-banking77
+  results:
+  - task:
+      name: Text Classification
+      type: text-classification
+    dataset:
+      name: banking77
+      type: banking77
+    metrics:
+    - name: Accuracy
+      type: accuracy
+      value: 0.9224
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# Quantized-distilbert-banking77
+
+This model is a statically quantized version of [optimum/distilbert-base-uncased-finetuned-banking77](https://huggingface.co/optimum/distilbert-base-uncased-finetuned-banking77) on the `banking77` dataset.
+It achieves the following results on the evaluation set:
+
+- Vanilla model: 92.5%
+- Quantized model: 92.24%
+=> The quantized model achieves 99.72% accuracy of the fp32 model
+
+Latency
+Payload sequence length: 128
+Instance type: AWS c6i.xlarge
+Vanilla model: P95 latency (ms) - 86.7772593483096; Average latency (ms) - 62.55 +\- 8.66;
+Quantized model: P95 latency (ms) - 27.027633551188046; Average latency (ms) - 26.17 +\- 0.66;
+Improvement through quantization: 2.39x
+
+## How to use
+
+```python
+from optimum.onnxruntime import ORTModelForSequenceClassification
+from transformers import pipeline, AutoTokenizer
+
+model = ORTModelForSequenceClassification.from_pretrained("philschmid/quantized-distilbert-banking77")
+tokenizer = AutoTokenizer.from_pretrained("philschmid/quantized-distilbert-banking77")
+
+remote_clx = pipeline("text-classification",model=model, tokenizer=tokenizer)
+
+remote_clx("What is the exchange rate like on this app?")
+```