Update README.md

README.md

@@ -113,4 +113,18 @@ widget:
     medical parole hearing or medical parole release is scheduled for an inmate receiving
     medical parole consideration, regardless of whether the inmate is sentenced either
     determinately or indeterminately.'
----
+---
+
+# Longformer Encoder-Decoder (LED) fine-tuned on Billsum
+This model is a fine-tuned version of led-base-16384 on the billsum dataset.
+
+As described in Longformer: The Long-Document Transformer by Iz Beltagy, Matthew E. Peters, Arman Cohan, led-base-16384 was initialized from bart-base since both models share the exact same architecture. To be able to process 16K tokens, bart-base's position embedding matrix was simply copied 16 times.
+
+
+# Use In Transformers
+```
+from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
+
+tokenizer = AutoTokenizer.from_pretrained("Artifact-AI/led_base_16384_billsum_summarization")
+
+model = AutoModelForSeq2SeqLM.from_pretrained("Artifact-AI/led_base_16384_billsum_summarization")