Instructions to use openbmb/cpm-bee-10b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openbmb/cpm-bee-10b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="openbmb/cpm-bee-10b", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("openbmb/cpm-bee-10b", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use openbmb/cpm-bee-10b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "openbmb/cpm-bee-10b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/cpm-bee-10b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/openbmb/cpm-bee-10b
- SGLang
How to use openbmb/cpm-bee-10b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "openbmb/cpm-bee-10b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/cpm-bee-10b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "openbmb/cpm-bee-10b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/cpm-bee-10b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use openbmb/cpm-bee-10b with Docker Model Runner:
docker model run hf.co/openbmb/cpm-bee-10b
'<' has been replaced with '<<', but the tokenizer still reported an error
The prompt is as follows, with an error displaying <iostream. h> as invalid:
result = model.generate({'input': """问题:在有n(n>1000)个元素的升序数组A中查找关键字x。查找算法的伪代码如下所示。
k=0;
while(k<<n且A[k]<<x)k=k+3;
if(k<<n且A[k]==x)查找成功;
else if(k-1<<n且A[k-1]==x)查找成功;
else if(k-2<<n且A[k-2]==x)查找成功;
else 查找失败;
本算法与折半查找算法相比,有可能具有更少比较次数的情形是____
A. 当 x 不在数组中
B. 当 x 接近数组开头处
C. 当 x 接近数组结尾处
D. 当 x 位于数组中间位置
答案:B
问题:#include<<iostream.h>
#define SQR(x) x*x
void main()
{
int a=10,k=2,m=1;
a/=SQR(k+m);cout<<<<a;
}
执行上面的C++程序后,a的值是____。
A. 10
B. 2
C. 9
D. 0
答案:""", '': ''}, tokenizer)
ValueError: Invalid token <iostream.h>
Hi, thanks for your feedback. This has been fixed, you can have a check:)