metadata
library_name: peft
base_model: meta-llama/Llama-2-7b-chat-hf
Model Card for Model ID
Model Details
Model Description
- Developed by: [More Information Needed]
- Funded by [optional]: [More Information Needed]
- Shared by [optional]: [More Information Needed]
- Model type: [More Information Needed]
- Language(s) (NLP): [More Information Needed]
- License: [More Information Needed]
- Finetuned from model [optional]: [More Information Needed]
Infrence Function
for branded or generic
def generate1(keyword):
prompt = f"""[INST] Annotate the keyword into branded or generic.[/INST]
[KW] {keyword} [/KW]
response ###"""
print("Prompt:")
print(prompt)
encoding = tokenizer(prompt, return_tensors="pt").to("cuda:0")
output = model.generate(input_ids=encoding.input_ids,
attention_mask=encoding.attention_mask,
max_new_tokens=200,
do_sample=True,
temperature=0.9,
eos_token_id=tokenizer.eos_token_id,
top_p=0.9,
repetition_penalty=1.2)
print()
# Subtract the length of input_ids from output to get only the model's response
output_text = tokenizer.decode(output[0, len(encoding.input_ids[0]):], skip_special_tokens=False)
output_text = re.sub('\n+', '\n', output_text) # remove excessive newline characters
print("Generated Assistant Response:")
print(output_text)
return output_text
for brand name
def generate2(lista,keyword):
prompt = f"""[INST] Extract the brand of the keyword from the given list if present.[/INST]
[KW] {keyword} [/KW]
[LIST] {lista} [/LIST]
response ###"""
print("Prompt:")
print(prompt)
encoding = tokenizer(prompt, return_tensors="pt").to("cuda:0")
output = model.generate(input_ids=encoding.input_ids,
attention_mask=encoding.attention_mask,
max_new_tokens=200,
do_sample=True,
temperature=0.9,
eos_token_id=tokenizer.eos_token_id,
top_p=0.9,
repetition_penalty=1.2)
print()
# Subtract the length of input_ids from output to get only the model's response
output_text = tokenizer.decode(output[0, len(encoding.input_ids[0]):], skip_special_tokens=False)
output_text = re.sub('\n+', '\n', output_text) # remove excessive newline characters
print("Generated Assistant Response:")
return output_text