Instructions to use microsoft/Phi-3.5-mini-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use microsoft/Phi-3.5-mini-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="microsoft/Phi-3.5-mini-instruct", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3.5-mini-instruct", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3.5-mini-instruct", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use microsoft/Phi-3.5-mini-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "microsoft/Phi-3.5-mini-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Phi-3.5-mini-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/microsoft/Phi-3.5-mini-instruct

SGLang

How to use microsoft/Phi-3.5-mini-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "microsoft/Phi-3.5-mini-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Phi-3.5-mini-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "microsoft/Phi-3.5-mini-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Phi-3.5-mini-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use microsoft/Phi-3.5-mini-instruct with Docker Model Runner:
```
docker model run hf.co/microsoft/Phi-3.5-mini-instruct
```

Phi-3.5-mini-instruct

Commit History

Update EvalEval source links to flat datastore

00e2edb
verified

EvalEvalBot commited on 5 days ago

Add EvalEval community eval results

209bc0a
verified

EvalEvalBot commited on 28 days ago

Add EvalEval community eval results

c2ada13
verified

EvalEvalBot commited on about 1 month ago

Add EvalEval community eval results

2549fee
verified

EvalEvalBot commited on about 1 month ago

Update README.md (#43)

2fe1924
verified

nguyenbh

bsnelling commited on Dec 10, 2025

Upload data_summary_card.md (#42)

a74e37c
verified

nguyenbh

bsnelling commited on Dec 10, 2025

Update readme

3145e03
verified

nguyenbh commited on Mar 2, 2025

Add Onnx model url

af0dfb8
verified

nguyenbh commited on Sep 18, 2024

Update README.md

ccf028f
verified

nguyenbh commited on Aug 29, 2024

Update README.md. Added Korean benchmarks. (#18)

bdf5de1
verified

nguyenbh

daekeun-ml commited on Aug 29, 2024

Update README.md

cd6881a
verified

nguyenbh commited on Aug 22, 2024

Update README.md

8ddd775
verified

nguyenbh commited on Aug 22, 2024

Update README.md

6496300
verified

nguyenbh commited on Aug 20, 2024

Update README.md

5adb202
verified

gargamit commited on Aug 20, 2024

upload model card

d509c56

garg-amit commited on Aug 20, 2024

upload weights

02b3eb6

garg-amit commited on Aug 17, 2024

Upload initial files

7a1ac0d

garg-amit commited on Aug 17, 2024

initial commit

f27a2f3
verified

gargamit commited on Aug 16, 2024

Commit History

Update EvalEval source links to flat datastore 00e2edb verified

Add EvalEval community eval results 209bc0a verified

Add EvalEval community eval results c2ada13 verified

Add EvalEval community eval results 2549fee verified

Update README.md (#43) 2fe1924 verified

Upload data_summary_card.md (#42) a74e37c verified

Update readme 3145e03 verified

Add Onnx model url af0dfb8 verified

Update README.md ccf028f verified

Update README.md. Added Korean benchmarks. (#18) bdf5de1 verified

Update README.md cd6881a verified

Update README.md 8ddd775 verified

Update README.md 6496300 verified

Update README.md 5adb202 verified

upload model card d509c56

upload weights 02b3eb6

Upload initial files 7a1ac0d

initial commit f27a2f3 verified

Update EvalEval source links to flat datastore

00e2edb
verified

Add EvalEval community eval results

209bc0a
verified

Add EvalEval community eval results

c2ada13
verified

Add EvalEval community eval results

2549fee
verified

Update README.md (#43)

2fe1924
verified

Upload data_summary_card.md (#42)

a74e37c
verified

Update readme

3145e03
verified

Add Onnx model url

af0dfb8
verified

Update README.md

ccf028f
verified

Update README.md. Added Korean benchmarks. (#18)

bdf5de1
verified

Update README.md

cd6881a
verified

Update README.md

8ddd775
verified

Update README.md

6496300
verified

Update README.md

5adb202
verified

upload model card

d509c56

upload weights

02b3eb6

Upload initial files

7a1ac0d

initial commit

f27a2f3
verified