Update README.md

README.md

@@ -25,7 +25,9 @@ widget:
 
 <!-- Provide a quick summary of what the model is/does. -->
 
-A fine-tuned version of Phi-2 for the NL2SQL usecase on `b-mc2/sql-create-context` dataset. __*This contains just the adapters!! You need to load the Phi2 model and add these adapters*__
+A fine-tuned version of Phi-2 for the NL2SQL usecase on `b-mc2/sql-create-context` dataset. 
+
+__*This contains just the adapters!! You need to load the Phi2 model and add these adapters*__
 
 ## Model Details
 
@@ -53,35 +55,6 @@ This model has been finetuned with `b-mc2/sql-create-context` on `microsoft/phi-
 
 Model is supposed to be used for the cases where you have a natural language question, database schema which is relevant the question to retrieve a SQL query which answers the question. The context should be below 2048 tokens. The output will be generated in postgresql.
 
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-```python
-# SAME TEMPLATE AS DEFOG MODEL
-prompt = f"""### Task
-Generate a SQL query to answer the following question:
-`{data_point['question']}`
-
-### Database Schema
-The query will run on a database with the following schema:
-{data_point['context']}
-
-### Answer
-Given the database schema, here is the SQL query that answers `{data_point['question']}`:
-```sql"""
-```
-
-```python
-# USING ON CPU MACHINE
-from llama_cpp import Llama
-
-phi2 = Llama(model_path=f"{path_to_model}/phi2_sqlcoder_f16.gguf")
-
-response = phi2(prompt=prompt, max_tokens = 200, temperature = 0.2, stop = ['```'])
-
-print(response['choices'][0]['text'].strip())
-```
 
 ### Downstream Use
 
@@ -89,26 +62,27 @@ print(response['choices'][0]['text'].strip())
 
 ```python
 # USING ON GPU MACHINE
-import torch
+from peft import PeftModel
 from transformers import AutoModelForCausalLM, AutoTokenizer
-# from peft import PeftModel, PeftConfig
 
-model_name = "pavankumarbalijepalli/phi2-sqlcoder"
+model_name = "microsoft/phi-2"
 
 model = AutoModelForCausalLM.from_pretrained(
     model_name,
     trust_remote_code=True,
-    device_map="auto"
+    device_map="auto",
+    torch_dtype=torch.float16
 )
 
-prompt = ""
+peftmodel = PeftModel.from_pretrained(model, "pavankumarbalijepalli/phi2-nl2sql-lora")
+peftmodel = peftmodel.merge_and_unload()
 
 tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
 tokenizer.pad_token = tokenizer.eos_token
 inputs = tokenizer(prompt, return_tensors="pt", padding=True, truncation=True)
 inputs.to('cuda')
 
-outputs = model.generate(**inputs, max_length=1000)
+outputs = peftmodel.generate(**inputs, max_length=1000)
 text = tokenizer.batch_decode(outputs,skip_special_tokens=True)[0]
 print(text)
 ```