TCMLLM commited on
Commit
df7f277
·
verified ·
1 Parent(s): fd4f5f9

Update model card: usage, evaluation and task prompts

Browse files
Files changed (1) hide show
  1. README.md +274 -81
README.md CHANGED
@@ -1,134 +1,319 @@
1
  ---
2
- frameworks:
3
- - ""
 
 
 
 
 
 
 
 
 
 
4
  ---
 
 
 
5
  <p align="center">
6
  <img width="420" src="https://raw.githubusercontent.com/TCMAI-BJTU/Lingdan-V2/main/assets/logo.png" alt="Lingdan-V2 logo" />
7
  </p>
8
 
9
  <p align="center"><strong>Lingdan-V2: Large Language Models for Clinically Grounded Reasoning in Traditional Chinese Medicine</strong></p>
10
 
11
- <p align="center">A multi-scale family of language models for TCM knowledge comprehension, clinical reasoning, and prescription recommendation</p>
 
 
 
 
12
 
13
  <p align="center">
14
- <a href="https://modelscope.cn/organization/TCMAIBJTU"><img src="https://img.shields.io/badge/Models-ModelScope-624AFF" alt="ModelScope models" /></a>
15
- <a href="http://tcmai.chat/"><img src="https://img.shields.io/badge/Online_Demo-tcmai.chat-F28C28" alt="Lingdan-V2 online demo" /></a>
16
  <a href="https://github.com/TCMAI-BJTU/LingLan"><img src="https://img.shields.io/badge/Benchmark-LingLan-2A8F5B" alt="LingLan benchmark" /></a>
17
  <img src="https://img.shields.io/badge/Model_scales-4B_%7C_8B_%7C_14B-0B5394" alt="Model scales: 4B, 8B, and 14B" />
18
  <a href="https://github.com/TCMAI-BJTU/Lingdan-V2/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache--2.0-D22128" alt="Apache License 2.0" /></a>
19
  </p>
20
 
21
- <p align="center">
22
- <a href="#overview">Overview</a> ·
23
- <a href="#model-zoo">Model Zoo</a> ·
24
- <a href="#study-design">Study Design</a> ·
25
- <a href="#main-results">Main Results</a> ·
26
- <a href="#data-privacy-and-responsible-use">Responsible Use</a> ·
27
- <a href="#license">License</a>
28
- </p>
29
 
30
- ## Overview
 
 
 
 
 
 
 
 
 
31
 
32
- Clinical decision-making in traditional Chinese medicine (TCM) requires the joint integration of domain knowledge, symptom presentations, syndrome differentiation, treatment principle selection, herbal prescription composition, and dosage information. Although general-purpose large language models demonstrate strong language understanding capabilities, they remain limited in their adaptation to specialized TCM knowledge, complex diagnostic-therapeutic reasoning, and structured prescription generation.
33
 
34
- Lingdan-V2 is a family of TCM large language models developed from Qwen3 backbone models at three parameter scales: 4B, 8B, and 14B. Through TCM-domain continued pretraining (CPT), reasoning-oriented supervised fine-tuning (SFT), and prescription-centered Group Relative Policy Optimization (GRPO), the models progressively acquire domain knowledge, task execution capabilities, structured reasoning, and prescription decision-making abilities. Lingdan-14B-R1 is the primary model investigated in this study, while the 4B and 8B series provide complementary evidence regarding the effects of model scale and prescription-centered post-training.
35
 
36
- | Model family | CPT corpus | SFT corpus | GRPO data | Evaluation scale |
37
- | --- | ---: | ---: | ---: | ---: |
38
- | 4B, 8B, and 14B × Base, SFT, and R1 | 16.62B tokens | 7.73M samples / 2.08B tokens | 24,212 Samples | LingLan: 25,624 items / Private clinical set: 4,348 cases |
 
 
 
 
 
39
 
40
- <p align="center">
41
- <img width="100%" src="https://raw.githubusercontent.com/TCMAI-BJTU/Lingdan-V2/main/assets/overview.png" alt="Overview of Lingdan-V2 development and evaluation" />
42
- </p>
43
- <p align="center"><sub>Figure 1. Data foundation, staged training pipeline, and evaluation settings of Lingdan-V2.</sub></p>
44
 
45
- ## Key Contributions
46
 
47
- - **A multi-scale family of TCM language models:** We develop Base, SFT, and R1 models at the 4B, 8B, and 14B parameter scales to support domain research, instruction-following tasks, and TCM clinical reasoning.
48
- - **A staged pathway for capability development:** We integrate large-scale TCM knowledge acquisition, reasoning-oriented supervised learning, and prescription-centered reinforcement post-training within a unified framework.
49
- - **A systematic evaluation of TCM clinical reasoning:** We evaluate TCM knowledge, information extraction, diagnostic-therapeutic reasoning, decision recognition, prescription composition, and dosage agreement on LingLan, LingLan-Hard, and a private prescription dataset for spleen and stomach disorders.
50
 
51
- ## Model Zoo
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
 
53
- All models are released on ModelScope. Model names follow the unified format `Lingdan-{Scale}-{Stage}`.
54
 
55
- | Model scale | Base | SFT | R1 |
56
- | --- | --- | --- | --- |
57
- | 4B | [Lingdan-4B-Base](https://modelscope.cn/models/TCMAIBJTU/Lingdan-4B-Base) | [Lingdan-4B-SFT](https://modelscope.cn/models/TCMAIBJTU/Lingdan-4B-SFT) | [Lingdan-4B-R1](https://modelscope.cn/models/TCMAIBJTU/Lingdan-4B-R1) |
58
- | 8B | [Lingdan-8B-Base](https://modelscope.cn/models/TCMAIBJTU/Lingdan-8B-Base) | [Lingdan-8B-SFT](https://modelscope.cn/models/TCMAIBJTU/Lingdan-8B-SFT) | [Lingdan-8B-R1](https://modelscope.cn/models/TCMAIBJTU/Lingdan-8B-R1) |
59
- | 14B | [Lingdan-14B-Base](https://modelscope.cn/models/TCMAIBJTU/Lingdan-14B-Base) | [Lingdan-14B-SFT](https://modelscope.cn/models/TCMAIBJTU/Lingdan-14B-SFT) | [Lingdan-14B-R1](https://modelscope.cn/models/TCMAIBJTU/Lingdan-14B-R1) |
60
 
61
- - **Base:** Continued pretrained models for further fine-tuning and domain-specific research.
62
- - **SFT:** Models obtained by supervised fine-tuning of the Base checkpoints for TCM question answering, information extraction, and general diagnostic and therapeutic instruction-following tasks.
63
- - **R1:** Models aligned from the SFT checkpoints through prescription-centered GRPO, with an emphasis on structured clinical reasoning and prescription recommendation.
64
 
65
- ## Study Design
 
 
 
66
 
67
- ### Data Foundation and Staged Training
68
 
69
- 1. **Continued pretraining (CPT):** Lingdan-Base models are trained on 16.62B tokens of TCM and clinically relevant text, including clinical data, classical and modern TCM literature, structured knowledge, public medical data, and general-domain corpora.
70
- 2. **Supervised fine-tuning (SFT):** Lingdan-SFT models are trained on 7,729,982 samples comprising 2.08B tokens. The tasks include TCM question answering, information extraction, diagnostic-therapeutic reasoning, clinical cases, and prescription generation.
71
- 3. **Prescription-centered GRPO:** Lingdan-R1 models are trained on 24,212 Samples, with prescription composition agreement serving as the primary optimization objective.
 
 
 
 
 
 
 
 
 
72
 
73
- ### Evaluation Settings
74
 
75
- - **LingLan:** 25,624 expert-verified samples spanning five TCM capability domains and 13 subtasks.
76
- - **LingLan-Hard:** 5,200 challenging samples, with 400 samples selected from each subtask.
77
- - **Private clinical prescription evaluation:** 4,348 de-identified cases involving spleen and stomach disorders, used to evaluate prescription composition and dosage agreement. The original clinical data are not publicly released.
78
 
79
- ## Main Results
80
 
81
- ### Comprehensive Benchmark Performance
 
 
82
 
83
- Lingdan-14B-R1 achieves an Overall Score of **67.1** on the full LingLan benchmark and **45.3** on LingLan-Hard. Among the external models evaluated in this study, DeepSeek-R1 obtains corresponding scores of 64.6 and 38.9. Lingdan-14B-R1 also achieves prescription-generation F1 scores of 37.0 on the full benchmark and 24.9 on the challenging subset.
84
 
85
- The Overall Score is computed as the unweighted mean of 16 primary metrics.
86
 
87
- <p align="center">
88
- <img width="760" src="https://raw.githubusercontent.com/TCMAI-BJTU/Lingdan-V2/main/assets/overall_score_comparison.png" alt="Overall Score comparison on LingLan and LingLan-Hard" />
89
- </p>
90
- <p align="center"><sub>Figure 2. Overall Score comparison between Lingdan-14B-R1 and external models of different scales on LingLan and LingLan-Hard.</sub></p>
 
 
 
 
 
91
 
92
- ### Performance Changes from SFT to R1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
93
 
94
- Within the primary 14B model trajectory, prescription-centered GRPO improves the Overall Score from 65.3 to 67.1 on the full LingLan benchmark and from 41.3 to 45.3 on LingLan-Hard. Prescription-generation F1 increases from 30.9 to 37.0 on the full benchmark and from 17.4 to 24.9 on the challenging subset.
95
 
96
- Overall Score and prescription-generation F1 improve from SFT to R1 across the 4B, 8B, and 14B model scales on both LingLan and LingLan-Hard. At 8B, Overall Score increases from 63.0 to 63.7 on the full benchmark and from 36.6 to 39.1 on LingLan-Hard, while prescription-generation F1 increases from 29.8 to 34.5 and from 16.1 to 22.6, respectively. The largest gains were observed in prescription generation, the task directly optimized during GRPO..
97
 
98
- <p align="center">
99
- <img width="100%" src="https://raw.githubusercontent.com/TCMAI-BJTU/Lingdan-V2/main/assets/training_stage_performance.png" alt="Performance changes from SFT to R1 across Lingdan-V2 model scales" />
100
- </p>
101
- <p align="center"><sub>Figure 3. Changes in Overall Score and prescription-generation F1 from SFT to R1 across the 4B, 8B, and 14B models.</sub></p>
102
 
103
- ### Private Clinical Prescription Evaluation
104
 
105
- On 4,348 de-identified cases involving spleen and stomach disorders, Lingdan-14B-R1 achieves a herb-level F1 score of **52.8**, a dosage cosine similarity of **50.6**, and a dosage MAE of **4.369**. Under the retrospective, text-based evaluation protocol defined in this study, these results indicate closer agreement with the reference prescriptions in both herbal composition and dosage structure.
 
 
 
 
 
 
 
 
 
106
 
107
- <p align="center">
108
- <img width="100%" src="https://raw.githubusercontent.com/TCMAI-BJTU/Lingdan-V2/main/assets/private_prescription_recommendation.png" alt="Private prescription recommendation results for spleen and stomach disorders" />
109
- </p>
110
- <p align="center"><sub>Figure 4. Herbal composition F1, dosage cosine similarity, and dosage MAE in the private clinical prescription evaluation for spleen and stomach disorders.</sub></p>
111
 
112
- ## Data Privacy and Responsible Use
 
 
 
 
 
 
113
 
114
- - The training and evaluation of Lingdan-V2 involve both public data and restricted clinical data. The repository does not disclose original private clinical cases, patient identifiers, medical record excerpts, or other information that could enable patient re-identification.
115
- - The private clinical evaluation is a retrospective, text-based assessment of prescription agreement. It does not constitute prospective clinical validation or evidence of effectiveness in real-world clinical practice.
116
- - Lingdan-V2 is intended solely for scientific and technical research. It does not provide medical advice and must not replace diagnosis, syndrome differentiation, treatment, or prescription review by qualified healthcare professionals.
117
 
 
118
 
119
- ## License
120
 
121
- The Lingdan-V2 model weights, together with the documentation and images in this repository for which the project authors hold copyright, are released under the [Apache License 2.0](https://github.com/TCMAI-BJTU/Lingdan-V2/blob/main/LICENSE). Lingdan-V2 is developed from Qwen3 open-weight models, which are also released under the Apache License 2.0.
122
 
123
- Subject to the terms of the Apache License 2.0, personal, research, and commercial use is permitted, as are copying, modification, and redistribution. Redistributions of the original or derivative versions must retain the license text and all applicable copyright, patent, trademark, and attribution notices; disclose any modifications; and preserve the project's [NOTICE](https://github.com/TCMAI-BJTU/Lingdan-V2/blob/main/NOTICE) file.
124
 
125
- The Apache License 2.0 does not relicense third-party training data, external benchmark materials, or private clinical data. These resources remain subject to their original licenses, data-use agreements, and privacy requirements. The medical safety statements above describe the intended use and associated risks of the models; they do not impose additional restrictions under the Apache License 2.0. Users are responsible for deployment decisions, regulatory compliance, and any clinical use.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
126
 
127
- This section is provided only as a summary. The `LICENSE` and `NOTICE` files in this repository govern the applicable rights and obligations.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
128
 
129
  ## Citation
130
 
131
- If you use Lingdan-V2 in your research, please cite this project:
132
 
133
  ```bibtex
134
  @misc{hua2026lingdanv2,
@@ -137,11 +322,19 @@ If you use Lingdan-V2 in your research, please cite this project:
137
  year = {2026},
138
  url = {https://github.com/TCMAI-BJTU/Lingdan-V2}
139
  }
 
 
 
 
 
 
 
 
 
 
 
140
  ```
141
 
142
  ## Related Links
143
 
144
- - [Lingdan-V2 GitHub](https://github.com/TCMAI-BJTU/Lingdan-V2)
145
- - [Lingdan-V2 models on ModelScope](https://modelscope.cn/organization/TCMAIBJTU)
146
- - [LingLan benchmark](https://github.com/TCMAI-BJTU/LingLan)
147
-
 
1
  ---
2
+ language:
3
+ - zh
4
+ license: apache-2.0
5
+ library_name: transformers
6
+ pipeline_tag: text-generation
7
+ base_model: TCMLLM/Lingdan-4B-SFT
8
+ base_model_relation: finetune
9
+ tags:
10
+ - qwen3
11
+ - traditional-chinese-medicine
12
+ - lingdan-v2
13
+ - r1
14
  ---
15
+
16
+ # Lingdan-4B-R1
17
+
18
  <p align="center">
19
  <img width="420" src="https://raw.githubusercontent.com/TCMAI-BJTU/Lingdan-V2/main/assets/logo.png" alt="Lingdan-V2 logo" />
20
  </p>
21
 
22
  <p align="center"><strong>Lingdan-V2: Large Language Models for Clinically Grounded Reasoning in Traditional Chinese Medicine</strong></p>
23
 
24
+ <p align="center">
25
+ <a href="http://tcmai.chat/">
26
+ <img src="https://img.shields.io/badge/OPEN_LINGDAN--V2_ONLINE-tcmai.chat-F28C28?style=for-the-badge&logo=googlechrome&logoColor=white" alt="Open the Lingdan-V2 online demo" />
27
+ </a>
28
+ </p>
29
 
30
  <p align="center">
31
+ <a href="https://modelscope.cn/collections/TCMAIBJTU/Lingdan-V2"><img src="https://img.shields.io/badge/Models-ModelScope-624AFF" alt="ModelScope models" /></a>
 
32
  <a href="https://github.com/TCMAI-BJTU/LingLan"><img src="https://img.shields.io/badge/Benchmark-LingLan-2A8F5B" alt="LingLan benchmark" /></a>
33
  <img src="https://img.shields.io/badge/Model_scales-4B_%7C_8B_%7C_14B-0B5394" alt="Model scales: 4B, 8B, and 14B" />
34
  <a href="https://github.com/TCMAI-BJTU/Lingdan-V2/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache--2.0-D22128" alt="Apache License 2.0" /></a>
35
  </p>
36
 
37
+ ## Contents
 
 
 
 
 
 
 
38
 
39
+ 1. [Model Details](#model-details)
40
+ 2. [Quickstart](#quickstart)
41
+ 3. [Thinking Mode](#thinking-mode)
42
+ 4. [vLLM Deployment](#vllm-deployment)
43
+ 5. [Checkpoint Evaluation](#checkpoint-evaluation)
44
+ 6. [Task Prompts](#task-prompts)
45
+ 7. [Data Privacy and Responsible Use](#data-privacy-and-responsible-use)
46
+ 8. [License](#license)
47
+ 9. [Citation](#citation)
48
+ 10. [Related Links](#related-links)
49
 
50
+ ## Model Details
51
 
52
+ **Lingdan-4B-R1** is a Qwen3-based TCM model in the [Lingdan-V2 family](https://github.com/TCMAI-BJTU/Lingdan-V2). CPT → SFT → prescription-centered GRPO on 24,212 samples.
53
 
54
+ | Field | Value |
55
+ | --- | --- |
56
+ | Parameter scale | 4B |
57
+ | Architecture family | Qwen3 causal language model |
58
+ | Training stage | Prescription-centered GRPO |
59
+ | Starting checkpoint / backbone | Lingdan-4B-SFT |
60
+ | Intended use | Structured TCM clinical reasoning and prescription-generation research |
61
+ | Platform model ID | `TCMLLM/Lingdan-4B-R1` |
62
 
63
+ ## Quickstart
 
 
 
64
 
65
+ Use Transformers ≥4.51.0; see the [Qwen3 quickstart](https://qwen.readthedocs.io/en/stable/getting_started/quickstart.html).
66
 
67
+ ```bash
68
+ pip install -U torch transformers accelerate
69
+ ```
70
 
71
+ ```python
72
+ from transformers import AutoModelForCausalLM, AutoTokenizer
73
+
74
+ model_name = "TCMLLM/Lingdan-4B-R1"
75
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
76
+ model = AutoModelForCausalLM.from_pretrained(
77
+ model_name,
78
+ torch_dtype="auto",
79
+ device_map="auto",
80
+ ).eval()
81
+
82
+ messages = [{"role": "user", "content": "请简要介绍中医辨证论治的基本概念。"}]
83
+ text = tokenizer.apply_chat_template(
84
+ messages,
85
+ tokenize=False,
86
+ add_generation_prompt=True,
87
+ enable_thinking=True,
88
+ )
89
+ inputs = tokenizer([text], return_tensors="pt").to(model.device)
90
+ outputs = model.generate(
91
+ **inputs,
92
+ max_new_tokens=8192,
93
+ eos_token_id=[151645, 151643], # <|im_end|> and <|endoftext|>
94
+ do_sample=True,
95
+ temperature=0.6,
96
+ top_p=0.95,
97
+ top_k=20,
98
+ )
99
+ new_tokens = outputs[0, inputs.input_ids.shape[1]:]
100
+ print(tokenizer.decode(new_tokens, skip_special_tokens=True))
101
+ ```
102
 
103
+ The explicit EOS IDs stop generation at either the chat-turn or text terminator.
104
 
105
+ ## Thinking Mode
 
 
 
 
106
 
107
+ The quickstart enables thinking with `enable_thinking=True`. Set it to `False` for direct answers. The released chat templates support this switch.
 
 
108
 
109
+ | Mode | `enable_thinking` | Temperature | Top-p | Top-k |
110
+ | --- | --- | ---: | ---: | ---: |
111
+ | Thinking | `True` | 0.6 | 0.95 | 20 |
112
+ | Non-thinking | `False` | 0.7 | 0.8 | 20 |
113
 
114
+ These are Qwen3 inference starting settings; behavior after fine-tuning may vary. Prefer the explicit switch over the unverified `/think` and `/no_think` soft instructions. Separate direct-generation output as follows:
115
 
116
+ ```python
117
+ raw_output = tokenizer.decode(new_tokens, skip_special_tokens=True)
118
+ reasoning_part, delimiter, final_answer = raw_output.partition("</think>")
119
+ if delimiter:
120
+ thinking_content = reasoning_part.removeprefix("<think>").strip()
121
+ final_answer = final_answer.strip()
122
+ else:
123
+ thinking_content = ""
124
+ final_answer = raw_output.strip()
125
+ print("Thinking:", thinking_content)
126
+ print("Answer:", final_answer)
127
+ ```
128
 
129
+ If generation hits the token limit before the thinking block closes, increase the budget and retry; the answer is incomplete.
130
 
131
+ ## vLLM Deployment
 
 
132
 
133
+ Install vLLM in a GPU environment with compatible CUDA/PyTorch; see the [Qwen3 vLLM guide](https://qwen.readthedocs.io/en/stable/deployment/vllm.html).
134
 
135
+ ```bash
136
+ pip install -U "vllm>=0.9.0" openai
137
+ ```
138
 
139
+ ### Start the Server
140
 
141
+ **Hugging Face:**
142
 
143
+ ```bash
144
+ export VLLM_USE_MODELSCOPE=false
145
+ vllm serve TCMLLM/Lingdan-4B-R1 \
146
+ --served-model-name Lingdan-4B-R1 \
147
+ --host 127.0.0.1 \
148
+ --port 8000 \
149
+ --max-model-len 16384 \
150
+ --reasoning-parser qwen3
151
+ ```
152
 
153
+ Replace the repository ID with a local model directory to reuse downloaded files. For two GPUs, append `--tensor-parallel-size 2`. Adjust the 16,384-token context limit (input + output) to available GPU memory.
154
+
155
+ ### Chat Client
156
+
157
+ ```python
158
+ from openai import OpenAI
159
+
160
+ client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
161
+ response = client.chat.completions.create(
162
+ model="Lingdan-4B-R1",
163
+ messages=[{"role": "user", "content": "请简要介绍中医辨证论治的基本概念。"}],
164
+ temperature=0.6,
165
+ top_p=0.95,
166
+ max_tokens=8192,
167
+ extra_body={
168
+ "top_k": 20,
169
+ "chat_template_kwargs": {"enable_thinking": True},
170
+ "stop_token_ids": [151645, 151643],
171
+ },
172
+ )
173
+ message = response.choices[0].message
174
+ thinking_content = (
175
+ getattr(message, "reasoning", None)
176
+ or getattr(message, "reasoning_content", None)
177
+ or ""
178
+ )
179
+ print("Thinking:", thinking_content)
180
+ print("Answer:", message.content)
181
+ print("Finish reason:", response.choices[0].finish_reason)
182
+ ```
183
 
184
+ `chat_template_kwargs` selects thinking mode; `--reasoning-parser qwen3` separates its output. To disable thinking, use the non-thinking settings above. The client supports both `reasoning` and the older `reasoning_content` field ([vLLM reference](https://docs.vllm.ai/en/latest/features/reasoning_outputs/)). Parse task JSON from `message.content`; `finish_reason="length"` indicates truncation.
185
 
186
+ `EMPTY` is a placeholder for the local server started without API-key authentication.
187
 
188
+ ## Checkpoint Evaluation
 
 
 
189
 
190
+ Paper-reported results for **Lingdan-4B-R1**. Full: LingLan (25,620 items); Hard: LingLan-Hard (5,200 items). Overall Score averages 16 primary metrics, excluding dose MAE. All scores use a 0–100 scale except MAE (lower is better). The paper uses zero-shot Chinese prompts, temperature 0.6, and at most 8,192 generated tokens; reproduction also requires the same data and scoring pipeline.
191
 
192
+ | Metric | LingLan Full | LingLan-Hard |
193
+ | --- | ---: | ---: |
194
+ | Overall Score ↑ | 63.4 | 40.4 |
195
+ | Prescription generation F1 ↑ | 36.2 | 24.8 |
196
+ | Clinical-record entity extraction F1 ↑ | 70.5 | 58.4 |
197
+ | Classical-text entity extraction F1 ↑ | 58.5 | 42.5 |
198
+ | Syndrome generation F1 ↑ | 9.5 | 1.6 |
199
+ | Treatment-principle generation F1 ↑ | 13.9 | 2.4 |
200
+ | Prescription dose cosine ↑ | 34.2 | 23.4 |
201
+ | Prescription dose MAE ↓ | 4.6 | 4.7 |
202
 
203
+ Task-level scores follow the appendix’s primary DTR metrics, not the character-level DTR-F1 variant.
 
 
 
204
 
205
+ | Private prescription evaluation (4,348 cases) | Score |
206
+ | --- | ---: |
207
+ | Herb F1 ↑ | 52.7 |
208
+ | Herb precision ↑ | 52.5 |
209
+ | Herb recall ↑ | 54.1 |
210
+ | Dose cosine ↑ | 49.5 |
211
+ | Dose MAE ↓ | 5.062 |
212
 
213
+ The private set contains 4,348 de-identified spleen/stomach disorder cases. See the [full comparison](https://github.com/TCMAI-BJTU/Lingdan-V2#main-results) for other checkpoints.
 
 
214
 
215
+ ## Task Prompts
216
 
217
+ For SFT/R1 models, fill the placeholders in these LingLan evaluation templates and pass the text as the user message.
218
 
219
+ ### Clinical-record Entity Extraction
220
 
221
+ Source: [LingLan `ner_clinic.py`](https://github.com/TCMAI-BJTU/LingLan/blob/main/simple-evals/evals/tcm_evals/ner_clinic.py).
222
 
223
+ ````text
224
+ 请从以下病历文本中抽取指定类型的实体。
225
+
226
+ 注意:
227
+ 1. 请先仔细分析文本内容,思考每个实体类型的识别要点,然后再输出结果
228
+ 2. 如果某个实体类型在文本中没有对应实体,请返回空列表[]
229
+ 3. 如果同一个实体在文本中出现多次,请在列表中按顺序重复出现
230
+
231
+ 文本:
232
+ {text}
233
+
234
+ 需要抽取的实体类型:
235
+ {entity_types}
236
+
237
+ 请按照以下JSON格式输出结果,每个实体类型对应一个列表,如果文本中某个实体出现多次,请在列表中重复出现:
238
+
239
+ ```json
240
+ {
241
+ "实体类型1": ["实体1", "实体2", ...],
242
+ "实体类型2": ["实体1", "实体2", ...],
243
+ ...
244
+ }
245
+ ```
246
+ ````
247
 
248
+ ### Classical-text Entity Extraction
249
+
250
+ Source: [LingLan `ner_ancient.py`](https://github.com/TCMAI-BJTU/LingLan/blob/main/simple-evals/evals/tcm_evals/ner_ancient.py).
251
+
252
+ ````text
253
+ 请从以下古籍文本中抽取指定类型的实体。
254
+
255
+ 注意:
256
+ 1. 请先仔细分析古���文本内容,思考古文语言特点和各实体类型的识别要点,然后再输出结果
257
+ 2. 如果某个实体类型在文本中没有对应实体,请返回空列表[]
258
+ 3. 如果同一个实体在文本中出现多次,请在列表中按顺序重复出现
259
+
260
+ 文本:
261
+ {text}
262
+
263
+ 需要抽取的实体类型:
264
+ {entity_types}
265
+
266
+ 请按照以下JSON格式输出结果,每个实体类型对应一个列表,如果文本中某个实体出现多次,请在列表中重复出现:
267
+
268
+ ```json
269
+ {
270
+ "实体类型1": ["实体1", "实体2", ...],
271
+ "实体类型2": ["实体1", "实体2", ...],
272
+ ...
273
+ }
274
+ ```
275
+ ````
276
+
277
+ ### Joint Syndrome, Treatment, and Prescription Generation
278
+
279
+ Source: [LingLan `pres_diag_eval.py`](https://github.com/TCMAI-BJTU/LingLan/blob/main/simple-evals/evals/tcm_evals/pres_diag_eval.py).
280
+
281
+ ````text
282
+ 请根据以下病历文本,进行完整的中医诊疗分析,包括辨证、治法和处方。
283
+
284
+ 注意:
285
+ 1. 请仔细分析患者的症状表现、舌脉象等信息
286
+ 2. 辨证:给出准确的中医证型判断(如有多个,用逗号分隔)
287
+ 3. 治法:根据辨证结果确定相应的治疗方法(如有多个,用逗号分隔)
288
+ 4. 处方:开具具体的中药处方,格式为"药名: 剂量, 药名: 剂量"
289
+
290
+ 病历文本:
291
+ {medical_record}
292
+
293
+ 请按照以下JSON格式输出结果:
294
+
295
+ ```json
296
+ {
297
+ "辨证": "证型1, 证型2, ...",
298
+ "治法": "治法1, 治法2, ...",
299
+ "处方": "药名1: 剂量1, 药名2: 剂量2, 药名3: 剂量3, ..."
300
+ }
301
+ ```
302
+ ````
303
+
304
+ Use the dataset’s exact entity labels. JSON examples contain illustrative ellipses; actual output must be valid JSON. In joint generation, `辨证`, `治法`, and `处方` are strings; format prescription items as `药名: 数值g`. Extract task JSON from the final answer using the output handling above.
305
+
306
+ ## Data Privacy and Responsible Use
307
+
308
+ Lingdan-V2 is for research and does not replace qualified clinical judgment. Private clinical data are not released. Retrospective prescription agreement is not evidence of clinical safety or effectiveness; use de-identified inputs and professional review for any clinical application.
309
+
310
+ ## License
311
+
312
+ Lingdan-V2 model weights and project-owned documentation and images are released under the [Apache License 2.0](https://github.com/TCMAI-BJTU/Lingdan-V2/blob/main/LICENSE). See [NOTICE](https://github.com/TCMAI-BJTU/Lingdan-V2/blob/main/NOTICE) for attribution notices. Third-party data and materials remain subject to their original terms.
313
 
314
  ## Citation
315
 
316
+ Cite Lingdan-V2 and the earlier Lingdan work:
317
 
318
  ```bibtex
319
  @misc{hua2026lingdanv2,
 
322
  year = {2026},
323
  url = {https://github.com/TCMAI-BJTU/Lingdan-V2}
324
  }
325
+
326
+ @article{hua2024lingdan,
327
+ title={Lingdan: enhancing encoding of traditional Chinese medicine knowledge for clinical reasoning tasks with large language models},
328
+ author={Hua, Rui and Dong, Xin and Wei, Yu and Shu, Zixin and Yang, Pengcheng and Hu, Yunhui and Zhou, Shuiping and Sun, He and Yan, Kaijing and Yan, Xijun and others},
329
+ journal={Journal of the American Medical Informatics Association},
330
+ volume={31},
331
+ number={9},
332
+ pages={2019--2029},
333
+ year={2024},
334
+ publisher={Oxford University Press}
335
+ }
336
  ```
337
 
338
  ## Related Links
339
 
340
+ [GitHub](https://github.com/TCMAI-BJTU/Lingdan-V2) · [Hugging Face collection](https://huggingface.co/collections/TCMLLM/lingdan-v2) · [ModelScope collection](https://modelscope.cn/collections/TCMAIBJTU/Lingdan-V2) · [LingLan benchmark](https://github.com/TCMAI-BJTU/LingLan)