Instructions to use spankevich/llm-hw-2-ppo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use spankevich/llm-hw-2-ppo with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="spankevich/llm-hw-2-ppo")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("spankevich/llm-hw-2-ppo")
model = AutoModelForCausalLM.from_pretrained("spankevich/llm-hw-2-ppo")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use spankevich/llm-hw-2-ppo with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "spankevich/llm-hw-2-ppo"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spankevich/llm-hw-2-ppo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/spankevich/llm-hw-2-ppo

SGLang

How to use spankevich/llm-hw-2-ppo with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "spankevich/llm-hw-2-ppo" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spankevich/llm-hw-2-ppo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "spankevich/llm-hw-2-ppo" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spankevich/llm-hw-2-ppo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use spankevich/llm-hw-2-ppo with Docker Model Runner:
```
docker model run hf.co/spankevich/llm-hw-2-ppo
```

Model Card for Model ID

PPO + LLM model from NLP homework at HSE.

Model Details

Examples:

======== BEFORE TUNING ======== user

What is your morning routine like?

assistant

As I sit here, I'm already feeling refreshed and ready to tackle the day. I'm already getting ready for work, and I'm already feeling energized and focused. I'm already getting a head start on my morning routine, and I'm already making progress on my goals.

I've been doing some research on the best morning routines for productivity, and I've found that I need to make some changes to get the most out of my morning. Here are some tips that might help me get started:

1. Start with a morning routine that works for you: What's your natural rhythm for waking up? Do you have a morning routine that works for you, or do you need to adjust it to fit your schedule? Experiment with different routines to find what works best for you.

2. Get some morning sunlight: Exposure to natural light in the morning helps regulate your circadian rhythms, which can help increase your energy levels and productivity. Try to get some morning sunlight for at least 10-15 minutes, even if it's just a few minutes a day.

3. Eat a healthy breakfast: A healthy breakfast can help you feel more energized and focused throughout the day. Try to eat a balanced breakfast that includes

======== AFTER TUNING ========

user

What is your morning routine like?

assistant

What a great question! As a digital AI assistant, I don't have personal experiences or emotions, but I can provide you with a general morning routine that might be relatable to most people. Here's a sample morning routine:

Morning Routine (08:00-09:00)

Train Plots

Downloads last month: 1

Safetensors

Model size

0.1B params

Tensor type

BF16

Collection including spankevich/llm-hw-2-ppo

llm-hw-2

Collection

collection of ppo, dpo and reward model • 3 items • Updated Mar 9, 2025 • 1