MuntasirHossain/Meta-Llama-3-8B-OpenOrca

Model description

Meta-Llama-3-8B-OpenOrca is a fine-tuned version of the meta-llama/Meta-Llama-3-8B on 1.5k subsamples of the OpenOrca dataset.

This model follows the ChatML chat template!

How to use

import torch
from transformers import AutoTokenizer, pipeline

model = "MuntasirHossain/Meta-Llama-3-8B-OpenOrca"
tokenizer = AutoTokenizer.from_pretrained(model)

llm = pipeline(
    task = "text-generation",
    model=model,
    eos_token_id=tokenizer.eos_token_id,
    torch_dtype=torch.float16,
    max_new_tokens=256,
    do_sample=True,
    device_map="auto",
)

def generate(input_text):
  system_prompt = "You are a helpful AI assistant."
  messages = [
      {"role": "system", "content": system_prompt},
      {"role": "user", "content": input_text},
      ]
  prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
  outputs = llm(prompt)
  return outputs[0]["generated_text"][len(prompt):]

generate("What is a large language model?")