Instructions to use OpenMEDLab/PULSE-20bv5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenMEDLab/PULSE-20bv5 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenMEDLab/PULSE-20bv5")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OpenMEDLab/PULSE-20bv5")
model = AutoModelForCausalLM.from_pretrained("OpenMEDLab/PULSE-20bv5")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use OpenMEDLab/PULSE-20bv5 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenMEDLab/PULSE-20bv5"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenMEDLab/PULSE-20bv5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OpenMEDLab/PULSE-20bv5

SGLang

How to use OpenMEDLab/PULSE-20bv5 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenMEDLab/PULSE-20bv5" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenMEDLab/PULSE-20bv5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenMEDLab/PULSE-20bv5" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenMEDLab/PULSE-20bv5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OpenMEDLab/PULSE-20bv5 with Docker Model Runner:
```
docker model run hf.co/OpenMEDLab/PULSE-20bv5
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

PULSE

开源模型

PULSE-20bv5

模型介绍

大规模训练：PULSE模型在internlm-20b模型的基础上，使用约4,000,000个医学领域和通用领域的SFT数据进行进一步微调。
全面的医学自然语言处理任务：PULSE支持医学领域的各种自然语言处理任务，包括健康教育、医师考试问题、报告解读、医疗记录结构化以及模拟诊断和治疗。

局限性

由于模型参数量较小和自回归生成范式，尽管模型提供了有关疾病诊断和治疗的推理结果，但这些结果不能代替线下职业医生的建议和治疗方案。所有回答仅供参考，不应作为诊断或治疗的依据。我们强烈建议用户在需要诊断或治疗疾病时，寻求专业医生的帮助和建议。

Elo评测

Model Name	AVG Rank	MedQA-USMLE	MedQA-Mainland	PromptCBLUE	WebMedQA	CheckupQA	MedicineQA	DialogSumm	MedTriage (F1)
GPT-4	1.25	1129	1117	1110	1116	1096	1098	1109	0.65
PULSE-Pro	1.75	1089	1092	1088	1119	1105	1083	1096	0.63
ChatGPT	4.00	1086	1057	1064	1053	1020	1029	1080	0.43
PULSE-20b	4.12	1042	1024	1039	1059	1049	1069	1076	0.40
Baichuan2	4.50	1024	1041	1065	1044	1062	1035	1069	0.33
ChatGLM3	5.62	1038	1062	997	1012	1003	1024	1021	0.06
HuatuoGPT2	7.62	955	993	985	963	983	1003	980	0.01
QiZhenGPT	8.38	955	959	945	989	1039	932	921	0.00
BenTsao	8.75	961	921	936	910	927	986	920	0.02
BianQue2	10.12	913	928	919	988	974	900	908	0.00
MING	10.75	902	909	924	867	862	960	918	0.01
DoctorGLM	11.12	906	896	930	879	880	880	905	0.00

注： PULSE-20b=PULSE-20bv5

推理

下载安装

下载本仓库内容至本地/远程服务器

git clone https://github.com/openmedlab/PULSE
cd PULSE

创建conda环境安装依赖

conda env create -f llm.yml
conda activate llm

其中torch和transformers版本不建议低于推荐版本。

使用示例

网页Demo

Gradio

python web_demo_gradio.py

命令行Demo

您可以运行仓库中的cli_demo.py来启动一个简单的命令行Demo：

python cli_demo.py

致谢

上海人工智能实验室
上海交通大学-清源研究院
华东理工大学-自然语言处理与大数据挖掘实验室

开源协议

本项目所含代码采用Apache 2.0协议，模型权重采用GNU AGPL 3.0协议。如使用本项目所含模型及其修改版本提供服务产生误导性或有害性言论，造成不良影响，由服务提供方负责，与本项目无关。

Downloads last month: 26

Safetensors

Model size

20B params

Tensor type

BF16

Model tree for OpenMEDLab/PULSE-20bv5

Quantizations

1 model

OpenMEDLab
/

PULSE-20bv5