Update README.md

README.md

@@ -38,7 +38,7 @@ A sequence of word embeddings is therefore processed sequentially by each transf
 
 The *conventional* T5 architectures are summarized in the following table:
 
-| Model | nl | ff | dm | kv | nh | #Params|
+| Model | nl (el/dl) | ff | dm | kv | nh | #Params|
 | ----| ---- | ---- | ---- | ---- | ---- | ----|
 | Tiny | 4/4 | 1024 | 256 | 32 | 4 | 16M|
 | Mini | 4/4 | 1536 | 384 | 32 | 8 | 31M|
@@ -52,29 +52,57 @@ with the following definitions:
 
 | Abbreviation | Definition |
 | ----| ---- |
-| NL | Number of transformer blocks (depth) | 
-| EL | Number of transformer blocks in the encoder (encoder depth) | 
-| DL | Number of transformer blocks in the decoder (decoder depth) | 
-| DM | Dimension of embedding vector (output vector of transformers block) | 
-| KV | Dimension of key/value projection matrix | 
-| NH | Number of attention heads | 
-| FF | Dimension of intermediate vector within transformer block (size of feed-forward projection matrix) | 
-| SH | Signifies that attention heads are shared | 
-| SKV | Signifies that key-values projection matrices are tied | 
+| nl | Number of transformer blocks (depth) | 
+| dm | Dimension of embedding vector (output vector of transformers block) | 
+| kv | Dimension of key/value projection matrix | 
+| nh | Number of attention heads | 
+| ff | Dimension of intermediate vector within transformer block (size of feed-forward projection matrix) | 
+| el | Number of transformer blocks in the encoder (encoder depth) | 
+| dl | Number of transformer blocks in the decoder (decoder depth) | 
+| sh | Signifies that attention heads are shared | 
+| skv | Signifies that key-values projection matrices are tied | 
+
+If a model checkpoint has no specific, *el* or *dl* than both the number of encoder- and decoder layers correspond no *nl*.
+
+This model checkpoint - **t5-efficient-xl** - is of model type **XL** with **no** variations.
+It has **2852** million parameters and thus requires **11406** MB of memory in full precision (*fp32*) 
+ or  **5703** MB of memory in half precision (*fp16* or *bf16*).
 
 ## Pre-Training
 
 The checkpoint was pretrained on the [Colossal, Cleaned version of Common Crawl (C4)](https://huggingface.co/datasets/c4) for 524288 steps using 
 the span-based masked language modeling (MLM) objective.
 
-## Downstream Performance
+## Fine-Tuning
+
+**Note**: This model is a **pretrained** checkpoint and has to be fine-tuned for practical usage.
+The checkpoint was pretrained in English and is therefore only useful for English NLP tasks.
+You can follow on of the following examples on how to fine-tune the model:
+
+*PyTorch*:
+
+- [Summarization](https://github.com/huggingface/transformers/tree/master/examples/pytorch/summarization)
+- [Question Answering](https://github.com/huggingface/transformers/blob/master/examples/pytorch/question-answering/run_seq2seq_qa.py)
+- [Text Classification](https://github.com/huggingface/transformers/tree/master/examples/pytorch/text-classification) - *Note*: You will have to slightly adapt the training example here to make it work with an encoder-decoder model.
 
-TODO:
+*Tensorflow*:
 
+- [Summarization](https://github.com/huggingface/transformers/tree/master/examples/tensorflow/summarization)
+- [Text Classification](https://github.com/huggingface/transformers/tree/master/examples/tensorflow/text-classification) - *Note*: You will have to slightly adapt the training example here to make it work with an encoder-decoder model.
 
+*JAX/Flax*:
+
+- [Summarization](https://github.com/huggingface/transformers/tree/master/examples/flax/summarization)
+- [Text Classification](https://github.com/huggingface/transformers/tree/master/examples/flax/text-classification) - *Note*: You will have to slightly adapt the training example here to make it work with an encoder-decoder model.
+
+## Downstream Performance
 
+TODO: Add table of full downstream performances if possible.
 
+## More information
 
-Pretraining Dataset: [C4](https://huggingface.co/datasets/c4)
+We strongly recommend the reader to go carefully through the original paper **[Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers](https://arxiv.org/abs/2109.10686)** to get a more nuanced understanding of this model checkpoint.
+As explained in the following [issue](https://github.com/google-research/google-research/issues/986#issuecomment-1035051145), checkpoints including the *sh* or *skv* 
+model architecture variations have *not* been ported to Transformers as they are probably of limited practical usage and are lacking a more detailed description.