dmitry315/llm-course-hw2-dpo

Model Card for Model ID

LLM trained with DPO for VK NLP course.

Model Details

Model Description

The model is LLM HuggingFaceTB/SmolLM-135M-Instruct.

Model trained with DPO procedure.

Model Sources

• Pretrained Model: HuggingFaceTB/SmolLM-135M-Instruct
• Train Data: HumanLLMs/Human-Like-DPO-Dataset

Getting start

device = torch.device("cuda")

tokenizer = AutoTokenizer.from_pretrained("dmitry315/llm-course-hw2-dpo")
check_model = AutoModelForCausalLM.from_pretrained("dmitry315/llm-course-hw2-dpo")
check_model = check_model.to(device)
check_model = check_model.eval()

messages = [{"role": "user", "content": "What's your morning routine like?"}]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

generated_ids = check_model.generate(model_inputs.input_ids, max_new_tokens=256, do_sample=False)
response = tokenizer.decode(generated_ids, skip_special_tokens=True)[0]
print(response)
# > My morning routine typically involves starting with a gentle stretch to calm my mind, then moving to my morning exercise routine to power through my day.