Update README.md

README.md

@@ -2,11 +2,13 @@
 language:
 - en
 library_name: transformers
+datasets:
+- GAIR/lima
 ---
 
 # model_42_70b
 
-A Llama2-70b model trained on Orca Style datasets.
+A Llama2-70b model fine tuned on lima datasets, which has < ~1k conversations.
 
 **I am actively seeking sponsorship and partnership opportunities. If you're interested, please connect with me at www.linkedin.com/in/pankajam.**
 
@@ -31,11 +33,9 @@ Here are the results on metrics used by [HuggingFaceH4 Open LLM Leaderboard](htt
 Here is the prompt format
 
 ```
-### System:
-You are an AI assistant that follows instruction extremely well. Help as much as you can.
 
 ### User:
-Tell me about Orcas.
+Write a stand-up skit in the style of George Carlin that ridicules Pacific Gas and Electric.
 
 ### Assistant:
 
@@ -55,11 +55,10 @@ model = AutoModelForCausalLM.from_pretrained(
   low_cpu_mem_usage=True,
   device_map="auto"
 )
-system_prompt = "### System:\nYou are an AI assistant that follows instruction extremely well. Help as much as you can.\n\n"
 
 #generate text steps
-instruction = "Tell me about Orcas."
-prompt = f"{system_prompt}### User: {instruction}\n\n### Assistant:\n"
+instruction = "Write a stand-up skit in the style of George Carlin that ridicules Pacific Gas and Electric."
+prompt = f"### User: {instruction}\n\n### Assistant:\n"
 inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
 output = model.generate(**inputs, do_sample=True, top_p=0.95, top_k=0, max_new_tokens=4096)
 
@@ -85,7 +84,7 @@ Please kindly cite using the following BibTeX:
 ```
 @misc{model_42_70b,
   author = {Pankaj Mathur},
-  title = {model_42_70b: An Orca Style Llama2-70b model},
+  title = {model_42_70b: A LIMA style Llama2-70b model},
   year = {2023},
   publisher = {HuggingFace},
   journal = {HuggingFace repository},
@@ -94,11 +93,12 @@ Please kindly cite using the following BibTeX:
 ```
 
 ```
-@misc{mukherjee2023orca,
-      title={Orca: Progressive Learning from Complex Explanation Traces of GPT-4}, 
-      author={Subhabrata Mukherjee and Arindam Mitra and Ganesh Jawahar and Sahaj Agarwal and Hamid Palangi and Ahmed Awadallah},
+@misc{ChuntingZhou,
+      title={LIMA: Less Is More for Alignment}, 
+      author={Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu,
+      Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, Omer Levy},
       year={2023},
-      eprint={2306.02707},
+      eprint={2305.11206},
       archivePrefix={arXiv},
       primaryClass={cs.CL}
 }