support-pvelocity

/

Code-Llama-2-13B-instruct-text2sql-GPTQ

like 0

I tried the below code and am facing a 'NoneType' object is not subscriptable error:

from transformers import AutoTokenizer
from auto_gptq import AutoGPTQForCausalLM

model_name = 'support-pvelocity/Code-Llama-2-13B-instruct-text2sql-GPTQ'

model = AutoGPTQForCausalLM.from_quantized(model_name, use_safetensors=True, device_map='auto')
tokenizer = AutoTokenizer.from_pretrained(model_name)

table = "CREATE TABLE sales (...); CREATE TABLE product_suppliers (...);"

question = 'Find the salesperson who made the most sales.'

prompt = f"[INST] Write SQLite query to answer the following question given the database schema. Please wrap your code answer using ```: Schema: {table} Question: {question} [/INST] Here is the SQLite query to answer to the question: {question}: ``` "

tokens = tokenizer(prompt, return_tensors="pt").to('cuda:0')
input_ids = tokens.input_ids

generated_ids = model.generate(input_ids=input_ids, max_length=4048, pad_token_id=tokenizer.eos_token_id)
output = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
output = output.split('```')[2]
print(output)

Below is the complete error:

TypeError                                 Traceback (most recent call last)
Cell In[1], line 18
     15 tokens = tokenizer(prompt, return_tensors="pt").to('cuda:0')
     16 input_ids = tokens.input_ids
---> 18 generated_ids = model.generate(input_ids=input_ids, max_length=4048, pad_token_id=tokenizer.eos_token_id)
     19 output = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
     20 output = output.split('```')[2]

File ~/.local/lib/python3.10/site-packages/auto_gptq/modeling/_base.py:447, in BaseGPTQForCausalLM.generate(self, **kwargs)
    ...
TypeError: 'NoneType' object is not subscriptable

Can anyone help me understand why I am getting this error and how to fix it?