Instructions to use openbmb/MiniCPM-2B-sft-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openbmb/MiniCPM-2B-sft-bf16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="openbmb/MiniCPM-2B-sft-bf16", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM-2B-sft-bf16", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use openbmb/MiniCPM-2B-sft-bf16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openbmb/MiniCPM-2B-sft-bf16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM-2B-sft-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/openbmb/MiniCPM-2B-sft-bf16

SGLang

How to use openbmb/MiniCPM-2B-sft-bf16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openbmb/MiniCPM-2B-sft-bf16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM-2B-sft-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openbmb/MiniCPM-2B-sft-bf16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM-2B-sft-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use openbmb/MiniCPM-2B-sft-bf16 with Docker Model Runner:
```
docker model run hf.co/openbmb/MiniCPM-2B-sft-bf16
```

MiniCPM-2B-sft-bf16

Commit History

Update tokenizer_config.json

4ec1634
verified

neoz commited on Sep 7, 2024

Upload config.json

79fbb1d
verified

hyx21 commited on Apr 7, 2024

Update config.json

4e4aa66
verified

ganqu commited on Apr 3, 2024

Upload 2 files

fe1d740
verified

hyx21 commited on Feb 3, 2024

Update README.md

9784ec7
verified

hyx21 commited on Feb 1, 2024

Update README.md

6122bb4
verified

hyx21 commited on Feb 1, 2024

Update README.md

9b606ef
verified

hyx21 commited on Feb 1, 2024

Update README.md

e642f17
verified

hyx21 commited on Feb 1, 2024

Update README.md

300c50d
verified

hyx21 commited on Feb 1, 2024

Upload README.md

adeb523
verified

hyx21 commited on Feb 1, 2024

Upload README.md

ac57e9f
verified

hyx21 commited on Feb 1, 2024

Upload generation_config.json

4f6ea04
verified

hyx21 commited on Feb 1, 2024

Upload modeling_minicpm.py

4ab96b0
verified

hyx21 commited on Jan 31, 2024

Add: ckpt

1b5dc80

Yuxiang commited on Jan 30, 2024

Upload 9 files

fab7c2d
verified

hyx21 commited on Jan 30, 2024

initial commit

c482e5e
verified

hyx21 commited on Jan 29, 2024

Commit History

Update tokenizer_config.json 4ec1634 verified

Upload config.json 79fbb1d verified

Update config.json 4e4aa66 verified

Upload 2 files fe1d740 verified

Update README.md 9784ec7 verified

Update README.md 6122bb4 verified

Update README.md 9b606ef verified

Update README.md e642f17 verified

Update README.md 300c50d verified

Upload README.md adeb523 verified

Upload README.md ac57e9f verified

Upload generation_config.json 4f6ea04 verified

Upload modeling_minicpm.py 4ab96b0 verified

Add: ckpt 1b5dc80

Upload 9 files fab7c2d verified

initial commit c482e5e verified

Update tokenizer_config.json

4ec1634
verified

Upload config.json

79fbb1d
verified

Update config.json

4e4aa66
verified

Upload 2 files

fe1d740
verified

Update README.md

9784ec7
verified

Update README.md

6122bb4
verified

Update README.md

9b606ef
verified

Update README.md

e642f17
verified

Update README.md

300c50d
verified

Upload README.md

adeb523
verified

Upload README.md

ac57e9f
verified

Upload generation_config.json

4f6ea04
verified

Upload modeling_minicpm.py

4ab96b0
verified

Add: ckpt

1b5dc80

Upload 9 files

fab7c2d
verified

initial commit

c482e5e
verified