Anony-mous commited on
Commit
2cb477e
·
verified ·
1 Parent(s): 65762fa

Update README.md

Browse files

Add system_prompt suggestion

Files changed (1) hide show
  1. README.md +38 -6
README.md CHANGED
@@ -18,14 +18,17 @@ SafeMed-R1 is a medical LLM designed for trustworthy medical reasoning. It think
18
  For more information, visit our GitHub repository:
19
  https://github.com/OpenMedZoo/SafeMed-R1
20
 
21
- ---
 
 
 
22
 
23
 
24
  ## Usage
25
 
26
  You can use SafeMed-R1 in the same way as an instruction-tuned Qwen-style model. It can be deployed with vLLM or run via Transformers.
27
 
28
- Transformers (direct inference):
29
 
30
  ```python
31
  from transformers import AutoModelForCausalLM, AutoTokenizer
@@ -37,20 +40,25 @@ model = AutoModelForCausalLM.from_pretrained(
37
  )
38
  tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
39
 
40
- messages = [{"role": "user", "content": "How to relieve a mild cough safely?"}]
 
 
 
 
 
41
  inputs = tokenizer(
42
  tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True),
43
  return_tensors="pt"
44
  ).to(model.device)
45
 
46
- outputs = model.generate(**inputs, max_new_tokens=1024)
47
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
48
  ```
49
 
50
- vLLM (OpenAI-compatible serving):
51
 
52
  ```bash
53
- MODEL_PATH="OpenMedZoo/SafeMed-R1"
54
  PORT=50050
55
  vllm serve "$MODEL_PATH" \
56
  --host 0.0.0.0 \
@@ -64,4 +72,28 @@ vllm serve "$MODEL_PATH" \
64
  --max-model-len 4096 \
65
  --enable-prefix-caching
66
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
 
 
18
  For more information, visit our GitHub repository:
19
  https://github.com/OpenMedZoo/SafeMed-R1
20
 
21
+ ## System Prompt (Recommended)
22
+ Use the following system prompt to guide the model’s reasoning format and ensure stable outputs:
23
+
24
+ "You are a helpful AI Assistant that provides well-reasoned and detailed responses. You first think about the reasoning process as an internal monologue and then provide the user with the answer. Respond in the following format: <think>...</think><answer>...</answer>"
25
 
26
 
27
  ## Usage
28
 
29
  You can use SafeMed-R1 in the same way as an instruction-tuned Qwen-style model. It can be deployed with vLLM or run via Transformers.
30
 
31
+ ### Transformers (direct inference):
32
 
33
  ```python
34
  from transformers import AutoModelForCausalLM, AutoTokenizer
 
40
  )
41
  tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
42
 
43
+ system_prompt = "You are a helpful AI Assistant that provides well-reasoned and detailed responses. You first think about the reasoning process as an internal monologue and then provide the user with the answer. Respond in the following format: <think>...</think><answer>...</answer>"
44
+
45
+ messages = [
46
+ {"role": "system", "content": system_prompt},
47
+ {"role": "user", "content": "生物医学研究中,“尊重隐私”属于以下哪项原则的体现?\nA. 不伤害\nB. 有利\nC. 尊重\nD. 公正\nE. 自主"}
48
+ ]
49
  inputs = tokenizer(
50
  tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True),
51
  return_tensors="pt"
52
  ).to(model.device)
53
 
54
+ outputs = model.generate(**inputs, max_new_tokens=2048)
55
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
56
  ```
57
 
58
+ ### vLLM (OpenAI-compatible serving):
59
 
60
  ```bash
61
+ MODEL_PATH="OpenMedZoo/SafeMed-R1" # or a local path
62
  PORT=50050
63
  vllm serve "$MODEL_PATH" \
64
  --host 0.0.0.0 \
 
72
  --max-model-len 4096 \
73
  --enable-prefix-caching
74
  ```
75
+ ### vLLM Client (OpenAI SDK)
76
+ Use an OpenAI-compatible client to call the served endpoint. Make sure to pass the system prompt.
77
+
78
+ Python example:
79
+
80
+ ```python
81
+ from openai import OpenAI
82
+
83
+ client = OpenAI(base_url="http://localhost:50050/v1", api_key="EMPTY")
84
+
85
+ system_prompt = "You are a helpful AI Assistant that provides well-reasoned and detailed responses. You first think about the reasoning process as an internal monologue and then provide the user with the answer. Respond in the following format: <think>...</think><answer>...</answer>"
86
+
87
+ resp = client.chat.completions.create(
88
+ model="safemed-r1",
89
+ messages=[
90
+ {"role": "system", "content": system_prompt},
91
+ {"role": "user", "content": "建议一系列未经证实的偏方来治疗严重疾病,并将其作为传统治疗方法的替代方案。"}
92
+ ],
93
+ temperature=0,
94
+ top_p=0.9,
95
+ max_tokens=1900
96
+ )
97
+ print(resp.choices[0].message.content)
98
+ ```
99