Merge branch 'main' of https://huggingface.co/zeroshot/sst2-distilbert-sparse into main

README.md

@@ -1,6 +1,9 @@
 ---
 license: apache-2.0
 ---
+
+### DISTILBERT RUNNING ON [DEEPSPARSE](https://github.com/neuralmagic/deepsparse) ENGINE GOES BRHMMMMMMMM. 🚀🚀🚀
+
 This model is 👇
 
         ███████╗ ██████╗   █████╗  ██████╗  ███████╗ ███████╗
@@ -11,6 +14,45 @@ This model is 👇
         ╚══════╝ ╚═╝      ╚═╝  ╚═╝ ╚═╝  ╚═ ╝╚══════╝ ╚══════╝
                                                                                                          
 ![Alt Text](https://media.giphy.com/media/4Hmjz2sqdtASJ2gFMH/giphy.gif)
+
+
+### LOOKS LIKE THIS 👇
+
+![Imgur](https://imgur.com/gWfX811.jpg)
+
+### Inference endpoints, outside of outliers (4ms) is avg. latency on 2 vCPUs:
+
+![Imgur](https://i.imgur.com/qceSdjZ.png)
+
+
+### Handler for access to inference endpoints
+
+```python
+class EndpointHandler:
+
+    def __init__(self, path=""):
+
+        self.pipeline = Pipeline.create(task="text-classification", model_path=path)
+
+    def __call__(self, data: Dict[str, Any]) -> Dict[str, str]:
+        """
+        Args:
+            data (:obj:): prediction input text
+        """
+        inputs = data.pop("inputs", data)
+
+        start = perf_counter()
+        prediction = self.pipeline(inputs)
+        end = perf_counter()
+        delta = end - start
+
+        return {
+            "labels":prediction.labels, 
+            "scores": prediction.scores,
+            "latency (secs.)": str(delta)
+        }
+```
+      
 ̷͈̍
 ̵̳͒R̶̙̓i̸̟͘c̴̻̆k̸̑͜ÿ̷̳́
 ̸̪̚