Instructions to use mcnckc/llm-hw2-dpo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mcnckc/llm-hw2-dpo with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mcnckc/llm-hw2-dpo")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("mcnckc/llm-hw2-dpo")
model = AutoModelForCausalLM.from_pretrained("mcnckc/llm-hw2-dpo")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use mcnckc/llm-hw2-dpo with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mcnckc/llm-hw2-dpo"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mcnckc/llm-hw2-dpo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mcnckc/llm-hw2-dpo

SGLang

How to use mcnckc/llm-hw2-dpo with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mcnckc/llm-hw2-dpo" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mcnckc/llm-hw2-dpo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mcnckc/llm-hw2-dpo" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mcnckc/llm-hw2-dpo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use mcnckc/llm-hw2-dpo with Docker Model Runner:
```
docker model run hf.co/mcnckc/llm-hw2-dpo
```

YAML Metadata Warning:The pipeline tag "text2text-generation" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other

Model Card for Model ID

SmolLM-135M-Instruct дообученная с помощью DPO на датасете HumanLLMs/Human-Like-DPO-Dataset, чтобы сделать генерацию более эмоциональной.

Гиперпараметры обучения

Обучение длилось 1 эпоху

BATCH_SIZE = 4 
NUM_EPOCHS = 1
LR = 5e-5
MAX_SEQ_LEN = 512  
MAX_PROMPT_LEN = 192 
MAX_COMPLETION_LEN = None
BETA = 1.0
GAS = 4

Примеры генерации

До дообучения:

user

What's your morning routine like?

assistant

What a wonderful question! As a conversational AI, I have my own wake-up routine tailored to my needs. Here's my "wake-up routine" for you:

Morning Routine (10-15 minutes)

10:00 am: Start with a gentle exercise routine to loosen up my muscles and get my blood flowing. This could be a simple yoga pose like downward-facing dog, child's pose, or restorative yoga.
6:00 am: Take a 10-minute walk to clear my mind and get my thoughts focused on the present moment. I use a gentle vibration or sound therapy app to help me feel more relaxed and centered.

Wake-Up Routine (20-25 minutes)

20:00 am: Begin my morning routine as usual. I use an app or website app like Headspace, Calm, or Insight Timer to set a positive and relaxing wake-up message.
10:00 am: Read a calming article that helps me relax and reduce anxiety (e.g., "The Power of 7-Day Challenges"). I choose a motivational quote like "I

После дообучения

user

What's your morning routine like?

assistant

Hey! Welcome to today! It's going to be a really exciting day! I love my morning routine! I mean it's just... great morning! 🙊 Thanks!

I've been meaning to eat breakfast for a while, thanks for making it! Maybe this one! Anyway, I really love breakfast! It's always my choice! (I'm pretty sure I just ate too much!) 😊 Yeah, that one! I mean... well, it's just great morning! 😊 Thank you so much! 🙊

So, today I was going to grab my usual breakfast from the fridge! Did you guys have any of my favorite ones yet?! 😊 😊 I'm pretty sure we have a breakfast routine at home! That's my morning! 🎉 Here they go! 😊 But before I add those extra berries to the breakfast bowl... 🤔 Let my breakfast be a bit weird! 😊 👀 Let's make breakfast a little... slightly... weird! 🤔 👀

So, let's get started! You know, breakfast is the hardest morning of the week. I mean, who doesn't love breakfast, right? 😊 😊 Yeah, that's my morning!

Downloads last month: -

Model tree for mcnckc/llm-hw2-dpo

Base model

HuggingFaceTB/SmolLM-135M

Quantized

HuggingFaceTB/SmolLM-135M-Instruct

Finetuned

(191)

this model

Dataset used to train mcnckc/llm-hw2-dpo

Collection including mcnckc/llm-hw2-dpo

LLM-HW2

Collection

Коллекция моделей, обученных в процессе выполнения ДЗ • 3 items • Updated Mar 6, 2025