Instructions to use openbmb/BitCPM-CANN-1B-unquantized with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openbmb/BitCPM-CANN-1B-unquantized with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="openbmb/BitCPM-CANN-1B-unquantized", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("openbmb/BitCPM-CANN-1B-unquantized", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("openbmb/BitCPM-CANN-1B-unquantized", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use openbmb/BitCPM-CANN-1B-unquantized with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openbmb/BitCPM-CANN-1B-unquantized"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/BitCPM-CANN-1B-unquantized",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/openbmb/BitCPM-CANN-1B-unquantized

SGLang

How to use openbmb/BitCPM-CANN-1B-unquantized with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openbmb/BitCPM-CANN-1B-unquantized" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/BitCPM-CANN-1B-unquantized",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openbmb/BitCPM-CANN-1B-unquantized" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/BitCPM-CANN-1B-unquantized",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use openbmb/BitCPM-CANN-1B-unquantized with Docker Model Runner:
```
docker model run hf.co/openbmb/BitCPM-CANN-1B-unquantized
```

BitCPM-CANN-1B-unquantized / example /README.md

guanwenyu1995

Add example/ folder with training scripts

d7bc7ce verified 3 months ago

preview code

Raw

History Blame

4.03 kB

BitCPM4 Continue Pretrain Example

This project provides scripts for continue pretraining BitCPM4-CANN-1B-unquantized.

Environment Setup

Docker Image

Use the following Huawei NPU image:

swr.cn-south-1.myhuaweicloud.com/ascendhub/mindspeed-llm:openeuler22.03-mindspeed-llm-2.3.0-a3-arm

Other Huawei NPU images may also work but have not been fully tested.

Install Dependencies

After entering the container, install the Python dependencies:

pip install -r requirements.txt

Dependency list:

Package	Version
transformers	4.46.3
tokenizers	0.20.3
accelerate	1.1.1
deepspeed	0.16.2
datasets	3.1.0
safetensors	0.4.5
pyarrow	17.0.0
tensorboard	2.18.0

Dataset

The test dataset used is C4-Pro, stored in parquet format after downloading.

Usage

Modify the path configuration in run.sh:

MODEL_PATH="/path/to/BitCPM4-CANN-1B-unquantized/"
DATA_PATH="/path/to/c4-pro/data/your_file.parquet"

Then start training:

bash run.sh

By default, the script trains for 500 steps using 8 devices, DeepSpeed ZeRO-2, and bf16 precision.

Training Results Reference

Below is the loss curve for the first 100 steps (learning rate warmup covers the first 50 steps):

Step	Loss	Learning Rate	Epoch
2	2.7920	1.60e-06	0.01
4	2.8012	3.20e-06	0.02
6	2.7984	4.80e-06	0.03
8	2.7839	6.40e-06	0.04
10	2.8084	8.00e-06	0.05
12	2.8064	9.60e-06	0.06
14	2.7994	1.12e-05	0.07
16	2.7463	1.28e-05	0.08
18	2.7580	1.44e-05	0.09
20	2.8007	1.60e-05	0.10
22	2.8916	1.76e-05	0.12
24	2.8144	1.92e-05	0.13
26	2.7723	2.08e-05	0.14
28	2.7556	2.24e-05	0.15
30	2.7414	2.40e-05	0.16
32	2.7469	2.56e-05	0.17
34	2.7428	2.72e-05	0.18
36	2.7392	2.88e-05	0.19
38	2.7132	3.04e-05	0.20
40	2.7008	3.20e-05	0.21
42	2.7547	3.36e-05	0.22
44	2.7151	3.52e-05	0.23
46	2.7119	3.68e-05	0.24
48	2.7029	3.84e-05	0.25
50	2.6803	4.00e-05	0.26
52	2.6980	4.00e-05	0.27
54	2.6923	4.00e-05	0.28
56	2.7068	4.00e-05	0.29
58	2.6965	4.00e-05	0.30
60	2.7179	3.99e-05	0.31
62	2.7119	3.99e-05	0.32
64	2.7178	3.99e-05	0.33
66	2.7069	3.99e-05	0.35
68	2.6870	3.98e-05	0.36
70	2.6775	3.98e-05	0.37
72	2.7038	3.98e-05	0.38
74	2.6924	3.97e-05	0.39
76	2.7061	3.97e-05	0.40
78	2.6929	3.96e-05	0.41
80	2.6787	3.96e-05	0.42
82	2.6749	3.95e-05	0.43
84	2.6909	3.94e-05	0.44
86	2.6893	3.94e-05	0.45
88	2.6788	3.93e-05	0.46
90	2.6831	3.92e-05	0.47
92	2.7039	3.91e-05	0.48
94	2.6619	3.91e-05	0.49
96	2.6903	3.90e-05	0.50
98	2.6993	3.89e-05	0.51
100	2.6891	3.88e-05	0.52
102	2.6739	3.87e-05	0.53

Note: BitCPM has its own training dataset and data mixture. It is expected that the loss continues to decrease when continue pretraining on open-source datasets.

As shown in the table, the loss gradually decreases from ~2.79 to ~2.67, indicating a stable training process and that the model is learning normally.

File Description

File	Description
`train.py`	Training script based on HuggingFace Trainer + DeepSpeed
`run.sh`	Launch script with training hyperparameter configuration
`train_sft.py`	Supervised fine-tuning script based on HuggingFace Trainer + DeepSpeed
`run_sft.sh`	Launch script for SFT with hyperparameter configuration
`ds_config.json`	DeepSpeed ZeRO-3 configuration (with CPU offload)
`ds_config_z2.json`	DeepSpeed ZeRO-2 configuration (used by default)
`requirements.txt`	Python dependency list