Update README.md

README.md

@@ -1,11 +1,17 @@
 ---
 base_model: google/gemma-2b
 library_name: peft
+license: apache-2.0
+datasets:
+- AnonySub628/physics-scienceqa
+language:
+- en
+pipeline_tag: question-answering
 ---
 
 # Model Card for Model ID
 
-<!-- Provide a quick summary of what the model is/does. -->
+A Gemma-2b finetuned LoRA trained on science Q&A
 
 
 
@@ -17,13 +23,13 @@ library_name: peft
 
 
 
-- **Developed by:** [More Information Needed]
+- **Developed by:** Venkat
 - **Funded by [optional]:** [More Information Needed]
 - **Shared by [optional]:** [More Information Needed]
 - **Model type:** [More Information Needed]
 - **Language(s) (NLP):** [More Information Needed]
 - **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+- **Finetuned from model [optional]:** Gemma-2b
 
 ### Model Sources [optional]
 
@@ -69,8 +75,53 @@ Users (both direct and downstream) should be made aware of the risks, biases and
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
-
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
+from peft import PeftModel
+from typing import Optional
+import time
+import os
+
+def generate_prompt(input_text: str, instruction: Optional[str] = None) -> str:
+    text = f"### Question: {input_text}\n\n### Answer: "
+    if instruction:
+        text = f"### Instruction: {instruction}\n\n{text}"
+    return text
+
+huggingface_token = os.environ.get('HUGGINGFACE_TOKEN')
+
+base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2b", token=huggingface_token)
+tokenizer = AutoTokenizer.from_pretrained("google/gemma-2b", token=huggingface_token)
+
+lora_model = PeftModel.from_pretrained(base_model, "vdpappu/lora_scienceqa")
+merged_model = lora_model.merge_and_unload()
+
+eos_token = '<eos>'
+eos_token_id = tokenizer.encode(eos_token, add_special_tokens=False)[-1]
+
+generation_config = GenerationConfig(
+       eos_token_id=tokenizer.eos_token_id,
+       min_length=5,
+       max_length=200,
+       do_sample=True,
+       temperature=0.7,
+       top_p=0.9,
+       top_k=50,
+       repetition_penalty=1.5,
+       no_repeat_ngram_size=3,
+       early_stopping=True
+   )
+
+question = "Which is the smoothest? Choose from: concrete sidewalk, sandpaper, paper."
+prompt = generate_prompt(input_text=question)
+
+with torch.no_grad():
+    inputs = tokenizer(prompt, return_tensors="pt")
+    output = merged_model.generate(**inputs, generation_config=generation_config)
+    response = tokenizer.decode(output[0], skip_special_tokens=True)
+
+print(f"Inference time: {end-start:.2f} seconds")
+print(response)
 [More Information Needed]
 
 ## Training Details