Update README.md

README.md

@@ -1,3 +1,97 @@
 ---
+datasets:
+- tatsu-lab/alpaca
+language:
+- en
+pipeline_tag: text2text-generation
+library_name: transformers
 license: other
 ---
+
+
+# Model Details
+
+- **Model name:** Flan-UL2-Alpaca-LORA
+- **Model type:** - Text2Text Generation
+- **Parent Model:** [google/flan-ul2](https://huggingface.co/google/flan-ul2)
+- **Training dataset:** [Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) 
+- **Language:** English
+- **Framework:** PyTorch
+- **Model version:** 1.0
+
+
+We take the instruction-tuned Flan models (trained on Academic datasets) and perform style transfer using the Alpaca dataset.
+
+We fine-tuned the google/flan-ul2 model on the Alpaca datset using [PEFT-LORA](https://huggingface.co/docs/diffusers/main/en/training/lora). 
+
+# License
+- Parent model ([google/flan-ul2](https://huggingface.co/google/flan-ul2)): Apache 2.0
+- Dataset ([Alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca)) : cc-by-4.0
+- Text-Davinci-3 (Used to generate Alpaca): [OpenAI License](https://openai.com/policies/terms-of-use)
+
+
+
+# How to Use
+
+```
+import torch
+from transformers import pipeline
+
+# Chose the model inference precision
+dtype = torch.float16 # options are torch.bfloat16, torch.float32
+
+model = pipeline(model="VMware/flan-ul2-alpaca-lora",device_map = 'auto',torch_dtype=dtype )
+
+prompt = "YOUR PROMPT HERE"
+
+output = model(prompt_template.format(instruction= prompt), max_length=2048, do_sample=True)
+
+
+```
+
+
+Using Alpaca prompt template might generate better outputs for certain prompts as the model was trained using the template.
+
+```
+# Chose the model inference precision
+import torch
+from transformers import pipeline
+
+dtype = torch.float16 # options are torch.bfloat16, torch.float32
+model = pipeline(model="VMware/flan-ul2-alpaca-lora",device_map = 'auto',torch_dtype=dtype )
+
+prompt_template = "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response:"
+
+prompt = "YOUR PROMPT HERE"
+
+output = model(prompt_template.format(instruction= prompt), max_length=2048, do_sample=True)
+
+
+
+```
+
+# Training Details
+
+The model was trained on 3xV100 GPUs using PEFT-LORA and Deepspeed
+
+* Hyperparameters:
+* learning_rate = 3e-4
+* per_device_batch_size = 2
+* gradient_accumulation_steps = 21 
+*  epochs = 3
+
+
+```
+
+
+
+# Limitations and Bias
+
+The model is based on a large and diverse dataset, but it may still have limitations and biases in certain areas. Some limitations include:
+
+- Language: The model is designed to work with English text only and may not perform as well in other languages.
+
+
+In addition, the model may have some bias in terms of the data it was trained on. The dataset includes questions from a variety of sources, but it may not be representative of all populations or perspectives. As a result, the model may perform better or worse for certain types of questions or on certain types of texts.
+
+# Contribution