Update README.md

README.md

@@ -19,6 +19,9 @@ metrics:
 
 # Multi-purpose Summarizer (Fine-tuned 3B google/flan-t5-xl on several Summarization datasets)
 
+ <a href="https://colab.research.google.com/gist/pszemraj/5dc89199a631a9c6cfd7e386011452a0/demo-flan-t5-large-grammar-synthesis.ipynb">
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
+</a>
 
 A fine-tuned version of [google/flan-t5-xl](https://huggingface.co/google/flan-t5-xl) on various summarization datasets (xsum, wikihow, cnn_dailymail/3.0.0, samsum, scitldr/AIC, billsum, TLDR)
 
@@ -31,7 +34,47 @@ Goal: a model that can be used for a general-purpose summarizer for academic and
 
 ---
 
-# Usage - Basic
+## Usage 
+
+Check the colab notebook. **The model expects a prompt prepended to the source document to indicate the type of summary**, examples of prompts used to train the model here:
+```
+prompts = {
+    "article": "Produce an article summary of the following news article:",
+    "one_sentence": "Given the following news article, summarize the article in one sentence:",
+    "conversation": "Briefly summarize in third person the following conversation:",
+    "scitldr": "Given the following scientific article, provide a TL;DR summary:",
+    "bill": "Summarize the following proposed legislation (bill):",
+    "outlines": "Produce an article summary including outlines of each paragraph of the following article:",
+}
+```
+After `pip install transformers` run the following code:
+
+```python
+from transformers import pipeline
+
+summarizer = pipeline("summarization", "jordiclive/flan-t5-3b-summarizer", torch_dtype=torch.bfloat16)
+
+raw_document = 'You must be 18 years old to live or work in New York State...'
+prompt = "Produce an article summary of the following news article:"
+results = summarizer(
+        f"{prompt} {raw_document}",
+        num_beams=5,
+        min_length=5,
+        no_repeat_ngram_size=3,
+        skip_special_tokens=True,
+        truncation=True,
+        max_length=512,
+    )
+```
+
+**For Batch Inference:** see [this discussion thread](https://huggingface.co/pszemraj/flan-t5-large-grammar-synthesis/discussions/1) for details, but essentially the dataset consists of several sentences at a time, and so I'd recommend running inference **in the same fashion:** batches of 64-96 tokens ish (or, 2-3 sentences split with regex) 
+
+- it is also helpful to **first** check whether or not a given sentence needs grammar correction before using the text2text model. You can do this with BERT-type models fine-tuned on CoLA like `textattack/roberta-base-CoLA`
+- I made a notebook demonstrating batch inference [here](https://colab.research.google.com/gist/pszemraj/6e961b08970f98479511bb1e17cdb4f0/batch-grammar-check-correct-demo.ipynb)
+
+
+
+---
 
 ## Training procedure