Update README.md

README.md

@@ -12,14 +12,18 @@ widget:
 - text: "El autor se perfila, a los 50 años de su muerte, como uno de los grandes de su siglo"
   candidate_labels: "cultura, sociedad, economia, salud, deportes"
 ---
-# ZERO-SHOT SELECTRA: A zero-shot classifier based on SELECTRA
+# Zero-shot SELECTRA: A zero-shot classifier based on SELECTRA
+
+*Zero-shot SELECTRA* is a [SELECTRA model](https://huggingface.co/Recognai/selectra_small) fine-tuned on the Spanish portion of the [XNLI dataset](https://huggingface.co/datasets/xnli). You can use it with Hugging Face's [Zero-shot pipeline](https://huggingface.co/transformers/master/main_classes/pipelines.html#transformers.ZeroShotClassificationPipeline) to make [zero-shot classifications](https://joeddav.github.io/blog/2020/05/29/ZSL.html).
+
+In comparison to our previous zero-shot classifier [based on BETO](https://huggingface.co/Recognai/bert-base-spanish-wwm-cased-xnli), zero-shot SELECTRA is **much more lightweight**. As shown in the *Metrics* section, the *small* version (5 times fewer parameters) performs slightly worse, while the *medium* version (3 times fewer parameters) **outperforms** the BETO based zero-shot classifier.
 
 ## Usage
 
 ```python
 from transformers import pipeline
 classifier = pipeline("zero-shot-classification", 
-                       model="Recognai/zeroshot_selectra_small")
+                       model="Recognai/zeroshot_selectra_medium")
 
 classifier(
     "El autor se perfila, a los 50 años de su muerte, como uno de los grandes de su siglo",
@@ -28,6 +32,26 @@ classifier(
 )
 ```
 
+## Metrics
+
+| Model | Params | XNLI (acc) | \*MLSUM (acc) |
+| --- | --- | --- | --- |
+| zs BETO | 110M | 0.799 | 0.530 |
+| zs SELECTRA medium | 41M | **0.807** | **0.589** |
+| zs SELECTRA small | **22M** | 0.795 | 0.446 |
+
+\*evaluated with zero-shot learning (ZSL)
+
+- **XNLI**: The stated accuracy refers to the test portion of the [XNLI dataset](https://huggingface.co/datasets/xnli), after finetuning the model on the training portion.
+- **MLSUM**: For this accuracy we take the test set of the [MLSUM dataset](https://huggingface.co/datasets/mlsum) and classify the summaries of 5 selected labels. For details, check out our [evaluation notebook](https://github.com/recognai/selectra/blob/main/zero-shot_classifier/evaluation.ipynb)
+
 ## Training
 
-For the training notebook, please check out our [repository](https://github.com/recognai/selectra/tree/main/zero-shot_classifier).
+Check out our [training notebook](https://github.com/recognai/selectra/blob/main/zero-shot_classifier/training.ipynb) for all the details.
+
+## Authors
+
+- David Fidalgo ([GitHub](https://github.com/dcfidalgo))
+- Daniel Vila ([GitHub](https://github.com/dvsrepo))
+- Francisco Aranda ([GitHub](https://github.com/frascuchon))
+- Javier Lopez ([GitHub](https://github.com/javispp))