Qwen2-1.5B-DPO-Finetuned

This is a DPO fine-tuned version of Qwen2-1.5B.

Model Description

• Architecture: Transformer-based language model
• Parameters: 1.5 billion
• Fine-tuning: DPO (Direct Preference Optimization)
• Base Model: Qwen/Qwen2-1.5B

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "SpringDai/Qwen2-1.5B-DPO-Finetuned"

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Inference
inputs = tokenizer("What is machine learning?", return_tensors="pt")
outputs = model.generate(**inputs, max_length=100, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))