microsoft
/

Phi-3-mini-4k-instruct

Text Generation

text-generation-inference

Model card Files Files and versions

Instructions to use microsoft/Phi-3-mini-4k-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use microsoft/Phi-3-mini-4k-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="microsoft/Phi-3-mini-4k-instruct", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use microsoft/Phi-3-mini-4k-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "microsoft/Phi-3-mini-4k-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Phi-3-mini-4k-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/microsoft/Phi-3-mini-4k-instruct

How to use microsoft/Phi-3-mini-4k-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "microsoft/Phi-3-mini-4k-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Phi-3-mini-4k-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "microsoft/Phi-3-mini-4k-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/Phi-3-mini-4k-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use microsoft/Phi-3-mini-4k-instruct with Docker Model Runner:
```
docker model run hf.co/microsoft/Phi-3-mini-4k-instruct
```

Phi-3-mini-4k-instruct

Commit History

Update EvalEval source links to flat datastore

26c4fa6
verified

EvalEvalBot commited on 13 days ago

Add EvalEval community eval results

d765acc
verified

EvalEvalBot commited on Jun 7

Add EvalEval community eval results

8f0e609
verified

EvalEvalBot commited on Jun 5

Add EvalEval community eval results

865822e
verified

EvalEvalBot commited on Jun 5

Update README.md (#109)

f39ac1d
verified

bsnelling commited on Dec 10, 2025

Upload data_summary_card.md (#108)

87e6c1e
verified

bsnelling commited on Dec 10, 2025

Update README.md (#96)

0a67737
verified

Levaillant commited on Sep 20, 2024

Update README.md

5a516f8
verified

nguyenbh commited on Aug 20, 2024

Minor updates on README

c1358f8
verified

nguyenbh commited on Jul 1, 2024

Update model (#77)

4f818b1
verified

gargamit commited on Jul 1, 2024

Add attention_bias to make TGI work (#64)

ff07dc0
verified

philschmid commited on Jun 11, 2024

Add link to Phi-3 vision ONNX models

d269012
verified

kvaishnavi commited on May 31, 2024

Update sample_finetune.py (#65)

5fa3419
verified

damajercakms commited on May 22, 2024

Update README

19f7775
verified

nguyenbh commited on May 21, 2024

Update README.md

d6b5c7b
verified

gugarosa commited on May 20, 2024

Fix typo (#58)

8f5f3a0
verified

ftnext commited on May 16, 2024

fix(README): Adds temperature=0.7 to prevent Inference API from going off the chart.

920b6cf
verified

gugarosa commited on May 3, 2024

fix(tokenizer): Also updates tokenizer.json to prevent any mismatch.

653ee82
verified

gugarosa commited on May 2, 2024

fix(tokenizer_config): Adjusts `rstrip` of special tokens. (#53)

4eea1a7
verified

gugarosa commited on May 2, 2024

Update README.md

2921602
verified

gugarosa commited on Apr 29, 2024

fix(root): Replaces system by user to improve generation experience.

300945e

gugarosa commited on Apr 29, 2024

Update docstrings

291e9e3
verified

nguyenbh commited on Apr 28, 2024

Update docstring in sample_finetune.py

13565ec
verified

nguyenbh commited on Apr 28, 2024

Update README.md

cc80d6c
verified

gugarosa commited on Apr 26, 2024

Update sample_finetune.py (#42)

6a07688
verified

wwwaj commited on Apr 26, 2024

chore(readme): Adds a proper chat example to widget.

4b6011c
verified

gugarosa commited on Apr 26, 2024

fix(readme): Adds information about placeholder tokens.

3c0c9df

gugarosa commited on Apr 25, 2024

chore(root): Initial files upload.

65be4e0

gugarosa commited on Apr 22, 2024

initial commit

c91d28f
verified

gugarosa commited on Apr 22, 2024