Update README.md

README.md

@@ -5,11 +5,13 @@ datasets:
 - billsum
 ---
 
-## Introduction
+# Longformer Encoder-Decoder (LED) fine-tuned on Billsum
+This model is a fine-tuned version of led-large-16384 on the billsum dataset.
 
-[Allenai's Longformer Encoder-Decoder (LED)](https://github.com/allenai/longformer#longformer) finetuned on BillSum dataset.
+As described in Longformer: The Long-Document Transformer by Iz Beltagy, Matthew E. Peters, Arman Cohan, led-base-16384 was initialized from bart-base since both models share the exact same architecture. To be able to process 16K tokens, bart-base's position embedding matrix was simply copied 16 times.
 
 
+# Use In Transformers
 ```
 from transformers import AutoTokenizer, AutoModelForSeq2SeqLM