Update README.md

add a template for readme

README.md

@@ -1,3 +1,111 @@
 ---
+language: en
 license: openrail
+metrics:
+- accuracy
+- precision
+- recall
+- f1
+pipeline_tag: text-classification
 ---
+
+# DistilBERT base model (uncased)
+
+This model is a specialised version of the [BERT base model](https://huggingface.co/ultraleow/cloud4bert). The code for the training process will be uploaded
+[here](https://huggingface.co/ultraleow/cloud4bert/). This model is uncased: it does not make a difference between english and English.
+
+## Model description
+
+Cloud4bert is a transformers model, smaller and faster than BERT, which was pretrained on the same corpus in a
+self-supervised fashion, using the BERT base model as a teacher. This means it was pretrained on the raw texts only,
+with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic
+process to generate inputs and labels from those texts using the BERT base model. More precisely, it was pretrained
+with three objectives:
+
+- Distillation loss: the model was trained to return the same probabilities as the BERT base model.
+- Masked language modeling (MLM): this is part of the original training loss of the BERT base model. When taking a
+  sentence, the model randomly masks 15% of the words in the input then run the entire masked sentence through the
+  model and has to predict the masked words. This is different from traditional recurrent neural networks (RNNs) that
+  usually see the words one after the other, or from autoregressive models like GPT which internally mask the future
+  tokens. It allows the model to learn a bidirectional representation of the sentence.
+- Cosine embedding loss: the model was also trained to generate hidden states as close as possible as the BERT base
+  model.
+
+This way, the model learns the same inner representation of the English language than its teacher model, while being
+faster for inference or downstream tasks.
+
+## Intended uses & limitations
+
+will be added soon
+
+### How to use
+
+You can use this model directly with a pipeline for masked language modeling:
+
+```python
+>>> from transformers import pipeline
+>>> sentiment_analzyor = pipeline('text-classification', model='ultraleow/cloud4bert')
+>>> sentiment_analzyor("Sorry, I don't understand - are you saying you don't have the `paypal` section defined? You need to, otherwise, it's an 'unknown element' in the web.config.")
+[{'label': 'LABEL_0', 'score': 0.6916515231132507}]
+#LABEL_0 = negative
+#LABEL_1 = neutral
+#LABEL_2 = positive
+```
+
+Here is how to use this model to get the features of a given text in PyTorch:
+
+```python
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+model = AutoModelForSequenceClassification.from_pretrained("ultraleow/cloud4bert")
+tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
+text = "Replace me by any text you'd like."
+encoded_input = tokenizer(text, return_tensors='pt')
+output = model(**encoded_input)
+```
+
+and in TensorFlow:
+
+```python
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+model = AutoModelForSequenceClassification.from_pretrained("ultraleow/cloud4bert")
+tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
+text = "Replace me by any text you'd like."
+encoded_input = tokenizer(text, return_tensors='tf')
+output = model(encoded_input)
+```
+
+
+
+## Training data
+
+will be added soon
+
+## Training procedure
+
+will be added soon
+
+### Preprocessing
+
+will be added soon
+
+
+### Pretraining
+
+will be added soon
+
+
+## Evaluation results
+
+When fine-tuned on downstream tasks, this model achieves the following results:
+
+Glue test results:
+
+| Task | Recall(Weighted) | Precision(Weighted)  | f1(Weighted) | ACC |
+|:----:|:----:|:----:|:----:|:-----:|
+|      | 94.03% | 94.06% | 94.02% | 94.03%  | 
+
+
+
+<a href="https://huggingface.co/exbert/?model=distilbert-base-uncased">
+	<img width="300px" src="https://cdn-media.huggingface.co/exbert/button.png">
+</a>