Instructions to use LumiOpen/Poro-34B-chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LumiOpen/Poro-34B-chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LumiOpen/Poro-34B-chat")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LumiOpen/Poro-34B-chat")
model = AutoModelForCausalLM.from_pretrained("LumiOpen/Poro-34B-chat")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use LumiOpen/Poro-34B-chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LumiOpen/Poro-34B-chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LumiOpen/Poro-34B-chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LumiOpen/Poro-34B-chat

SGLang

How to use LumiOpen/Poro-34B-chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LumiOpen/Poro-34B-chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LumiOpen/Poro-34B-chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LumiOpen/Poro-34B-chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LumiOpen/Poro-34B-chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LumiOpen/Poro-34B-chat with Docker Model Runner:
```
docker model run hf.co/LumiOpen/Poro-34B-chat
```

Poro-34B-chat

Commit History

Add model card (#8)

ee6c811
verified

jonabur

nielsr HF Staff commited on Jun 12, 2025

Update README.md

2afed85
verified

Elaine commited on Jun 18, 2024

Update README.md

12be9a9
verified

Elaine commited on Jun 18, 2024

Update README.md

a341ce1
verified

Elaine commited on Jun 18, 2024

Update README.md

2767d58
verified

Elaine commited on Jun 18, 2024

update chat version

5da6497

jonabur commited on Jun 17, 2024

Update README.md

c9757cd
verified

Elaine commited on May 27, 2024

Update README.md

8b4fabc
verified

jonabur commited on May 24, 2024

update README

a7708da

jonabur commited on Apr 11, 2024

Update.

4cd3a61

jonabur commited on Apr 11, 2024

initial commit

e1590f5

jonabur commited on Apr 4, 2024

initial commit

dea64f7
verified

jonabur commited on Apr 4, 2024

Commit History

Add model card (#8) ee6c811 verified

Update README.md 2afed85 verified

Update README.md 12be9a9 verified

Update README.md a341ce1 verified

Update README.md 2767d58 verified

update chat version 5da6497

Update README.md c9757cd verified

Update README.md 8b4fabc verified

update README a7708da

Update. 4cd3a61

initial commit e1590f5

initial commit dea64f7 verified

Add model card (#8)

ee6c811
verified

Update README.md

2afed85
verified

Update README.md

12be9a9
verified

Update README.md

a341ce1
verified

Update README.md

2767d58
verified

update chat version

5da6497

Update README.md

c9757cd
verified

Update README.md

8b4fabc
verified

update README

a7708da

Update.

4cd3a61

initial commit

e1590f5

initial commit

dea64f7
verified