Instructions to use Qwen/Qwen2.5-Coder-1.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Qwen/Qwen2.5-Coder-1.5B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Qwen/Qwen2.5-Coder-1.5B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-1.5B")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-1.5B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Qwen/Qwen2.5-Coder-1.5B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Qwen/Qwen2.5-Coder-1.5B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen2.5-Coder-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Qwen/Qwen2.5-Coder-1.5B

SGLang

How to use Qwen/Qwen2.5-Coder-1.5B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Qwen/Qwen2.5-Coder-1.5B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen2.5-Coder-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Qwen/Qwen2.5-Coder-1.5B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen2.5-Coder-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Qwen/Qwen2.5-Coder-1.5B with Docker Model Runner:
```
docker model run hf.co/Qwen/Qwen2.5-Coder-1.5B
```

Qwen2.5-Coder-1.5B

Commit History

update tokenizer_config.json,config.json,generation_config.json

df3ce67

JustinLin610 commited on Nov 18, 2024

update README.md

6df6cbb

feihu.hf commited on Nov 12, 2024

Update README.md

acb224f
verified

cyente commited on Nov 11, 2024

Update README.md

51df6bb
verified

cyente commited on Nov 9, 2024

Update README.md

dba2098
verified

cyente commited on Nov 8, 2024

Update README.md

99644c1
verified

cyente commited on Nov 8, 2024

Update README.md

ad88ed4
verified

cyente commited on Sep 25, 2024

Merge branch 'main' of hf.co:Qwen/Qwen2.5-Coder-1.5B

ad47162

feihu.hf commited on Sep 20, 2024

update README.md

2a7876a

feihu.hf commited on Sep 20, 2024

Upload ./tokenizer_config.json with huggingface_hub

102a541
verified

clonefy commited on Sep 20, 2024

update tokenizer_config.json

d3586cf

feihu.hf commited on Sep 20, 2024

Update README.md (#3)

5978c88
verified

hzhwcmhf

cyente commited on Sep 20, 2024

Update README.md (#2)

ccb64f0
verified

huybery

cyente commited on Sep 19, 2024

Upload ./tokenizer_config.json with huggingface_hub

8fb5dca
verified

clonefy commited on Sep 19, 2024

update README & config.json

835b517

feihu.hf commited on Sep 18, 2024

update README & config.json

6a7051e

feihu.hf commited on Sep 18, 2024

update README & LICENSE

19a986b

feihu.hf commited on Sep 18, 2024

Upload LICENSE

d85b5e3
verified

hzhwcmhf commited on Sep 18, 2024

Create README.md (#1)

7a895e8
verified

hzhwcmhf

cyente commited on Sep 18, 2024

Upload folder using huggingface_hub

94fddf4
verified

clonefy commited on Sep 18, 2024

initial commit

e4919b8
verified

clonefy commited on Sep 18, 2024

Commit History

update tokenizer_config.json,config.json,generation_config.json df3ce67

update README.md 6df6cbb

Update README.md acb224f verified

Update README.md 51df6bb verified

Update README.md dba2098 verified

Update README.md 99644c1 verified

Update README.md ad88ed4 verified

Merge branch 'main' of hf.co:Qwen/Qwen2.5-Coder-1.5B ad47162

update README.md 2a7876a

Upload ./tokenizer_config.json with huggingface_hub 102a541 verified

update tokenizer_config.json d3586cf

Update README.md (#3) 5978c88 verified

Update README.md (#2) ccb64f0 verified

Upload ./tokenizer_config.json with huggingface_hub 8fb5dca verified

update README & config.json 835b517

update README & config.json 6a7051e

update README & LICENSE 19a986b

Upload LICENSE d85b5e3 verified

Create README.md (#1) 7a895e8 verified

Upload folder using huggingface_hub 94fddf4 verified

initial commit e4919b8 verified

update tokenizer_config.json,config.json,generation_config.json

df3ce67

update README.md

6df6cbb

Update README.md

acb224f
verified

Update README.md

51df6bb
verified

Update README.md

dba2098
verified

Update README.md

99644c1
verified

Update README.md

ad88ed4
verified

Merge branch 'main' of hf.co:Qwen/Qwen2.5-Coder-1.5B

ad47162

update README.md

2a7876a

Upload ./tokenizer_config.json with huggingface_hub

102a541
verified

update tokenizer_config.json

d3586cf

Update README.md (#3)

5978c88
verified

Update README.md (#2)

ccb64f0
verified

Upload ./tokenizer_config.json with huggingface_hub

8fb5dca
verified

update README & config.json

835b517

update README & config.json

6a7051e

update README & LICENSE

19a986b

Upload LICENSE

d85b5e3
verified

Create README.md (#1)

7a895e8
verified

Upload folder using huggingface_hub

94fddf4
verified

initial commit

e4919b8
verified