Update README.md

README.md

@@ -1,18 +1,36 @@
 ---
-{}
+license: apache-2.0
+language:
+- en
+library_name: transformers
 ---
+
+
+# Genstruct 7B
+
+Genstruct 7B is an instruction-generation model, inspired by [Ada-Instruct](https://arxiv.org/abs/2310.04484).
+
+Previous methods largely rely on in-context approaches to generate instructions, while Ada-Instruct trained a custom instruction-generation model.
+
+Inspired by this, we took this approach further by grounding the generations in user-provided context passages.
+Further, the model is trained to generate questions involving complex scenarios that require detailed reasoning, allowing for models trained on the generated data to reason step-by-step.
+
+An example notebook is provided [here](https://gist.github.com/euclaise/bb7113b9596666cbf939484156375f29), which details how to load and sample from the model.
+
+Alternatively, here's a minimal example:
 ```
-pre_text = "The following is an interaction between a user and an AI assistant that is related to the above text."                                                                                                                                                                                                                                                                                                                  
-def ds_map_fn(row):                                                                                                                                                                                             
-    input =  f"[[[Title]]] {row['title'].strip()}\n[[[Content]]] {row['context'].strip()}\n\n" + pre_text + "\n\n[[[User]]] "                                                                                     
-    output = f"{row['question'].strip()}\n[[[Assistant]]] {row['answer'].strip()}"                                                                                                                                
-                                                                                                                                                                                                                  
-    input = tokenizer.encode(input, add_special_tokens=False)                                                                                                                                                     
-    output = tokenizer.encode(output, add_special_tokens=False)                                                                                                                                                   
-                                                                                                                                                                                                                  
-    input_ids = input + output + [tokenizer.eos_token_id]                                                                                                                                                         
-    labels = [-100]*len(input) + output + [tokenizer.eos_token_id]                                                                                                                                                
-                                                                                                                                                                                                                  
-    return {'input_ids': input_ids, 'labels': labels}                                                                                                                                                             
-ds = ds.map(ds_map_fn, remove_columns=ds.column_names)
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+MODEL_NAME = 'NousResearch/Genstruct-7B'
+
+model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map='cuda', load_in_8bit=True)
+tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
+
+msg =[{
+    'title': 'p-value',
+    'content': "The p-value is used in the context of null hypothesis testing in order to quantify the statistical significance of a result, the result being the observed value of the chosen statistic T {\displaystyle T}.[note 2] The lower the p-value is, the lower the probability of getting that result if the null hypothesis were true. A result is said to be statistically significant if it allows us to reject the null hypothesis. All other things being equal, smaller p-values are taken as stronger evidence against the null hypothesis."
+}]
+inputs = tokenizer.apply_chat_template(msg, return_tensors='pt').cuda()
+
+print(tokenizer.decode(model.generate(inputs, max_new_tokens=512)[0]).split(tokenizer.eos_token)[0])
 ```