Instructions to use TCMLLM/Lingdan-4B-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TCMLLM/Lingdan-4B-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TCMLLM/Lingdan-4B-Base")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TCMLLM/Lingdan-4B-Base")
model = AutoModelForCausalLM.from_pretrained("TCMLLM/Lingdan-4B-Base", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TCMLLM/Lingdan-4B-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TCMLLM/Lingdan-4B-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TCMLLM/Lingdan-4B-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TCMLLM/Lingdan-4B-Base

SGLang

How to use TCMLLM/Lingdan-4B-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TCMLLM/Lingdan-4B-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TCMLLM/Lingdan-4B-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TCMLLM/Lingdan-4B-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TCMLLM/Lingdan-4B-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use TCMLLM/Lingdan-4B-Base with Docker Model Runner:
```
docker model run hf.co/TCMLLM/Lingdan-4B-Base
```

Lingdan-4B-Base / README.md

TCMLLM

Document cumulative CPT, SFT and RL training stages

d8f30f8 verified 6 days ago

preview code

Raw History Blame Contribute Delete

6.59 kB

	---
	language:
	- zh
	license: apache-2.0
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- qwen3
	- traditional-chinese-medicine
	- lingdan-v2
	- base
	---

	# Lingdan-4B-Base

	<p align="center">
	<img width="420" src="https://raw.githubusercontent.com/TCMAI-BJTU/Lingdan-V2/main/assets/logo.png" alt="Lingdan-V2 logo" />
	</p>

	<p align="center"><strong>Lingdan-V2: Large Language Models for Clinically Grounded Reasoning in Traditional Chinese Medicine</strong></p>

	<p align="center">
	<a href="http://tcmai.chat/">
	<img src="https://img.shields.io/badge/OPEN_LINGDAN--V2_ONLINE-tcmai.chat-F28C28?style=for-the-badge&logo=googlechrome&logoColor=white" alt="Open the Lingdan-V2 online demo" />
	</a>
	</p>

	<p align="center">
	<a href="https://huggingface.co/collections/TCMLLM/lingdan-v2"><img src="https://img.shields.io/badge/Models-Hugging_Face-FFD21E?logo=huggingface&logoColor=black" alt="Hugging Face models" /></a>
	<a href="https://modelscope.cn/collections/TCMAIBJTU/Lingdan-V2"><img src="https://img.shields.io/badge/Models-ModelScope-624AFF" alt="ModelScope models" /></a>
	<a href="https://github.com/TCMAI-BJTU/LingLan"><img src="https://img.shields.io/badge/Benchmark-LingLan-2A8F5B" alt="LingLan benchmark" /></a>
	<img src="https://img.shields.io/badge/Model_scales-4B_%7C_8B_%7C_14B-0B5394" alt="Model scales: 4B, 8B, and 14B" />
	<a href="https://github.com/TCMAI-BJTU/Lingdan-V2/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache--2.0-D22128" alt="Apache License 2.0" /></a>
	</p>

	[Model](#model-details) · [Quickstart](#quickstart) · [vLLM](#vllm-deployment) · [Citation](#citation)

	## Model Details

	Lingdan-4B-Base is a Qwen3-based TCM model in the [Lingdan-V2 family](https://github.com/TCMAI-BJTU/Lingdan-V2).

	\| Field \| Value \|
	\| --- \| --- \|
	\| Parameter scale \| 4B \|
	\| Architecture family \| Qwen3 causal language model \|
	\| Training stage \| Continued pretraining (CPT) \|
	\| Starting checkpoint / backbone \| Qwen3 4B backbone \|
	\| Intended use \| Text completion, domain research, and further fine-tuning \|
	\| Platform model ID \| `TCMLLM/Lingdan-4B-Base` \|

	### Training History

	This checkpoint includes the following training stages, in order:

	\| Stage \| Objective \| Training data \|
	\| --- \| --- \| --- \|
	\| CPT — continued pretraining \| TCM-domain knowledge acquisition \| 16.62B tokens \|

	## Quickstart

	Use Transformers ≥4.51.0; see the [Qwen3 quickstart](https://qwen.readthedocs.io/en/stable/getting_started/quickstart.html).

	```bash
	pip install -U torch transformers accelerate
	```

	```python
	from transformers import AutoModelForCausalLM, AutoTokenizer

	model_name = "TCMLLM/Lingdan-4B-Base"
	tokenizer = AutoTokenizer.from_pretrained(model_name)
	model = AutoModelForCausalLM.from_pretrained(
	model_name,
	torch_dtype="auto",
	device_map="auto",
	).eval()

	# Base checkpoints are intended for completion and further fine-tuning.
	text = "中医辨证论治的基本概念是"
	inputs = tokenizer([text], return_tensors="pt").to(model.device)
	outputs = model.generate(
	**inputs,
	max_new_tokens=1024,
	do_sample=True,
	temperature=0.6,
	top_p=0.95,
	top_k=20,
	)
	new_tokens = outputs[0, inputs.input_ids.shape[1]:]
	print(tokenizer.decode(new_tokens, skip_special_tokens=True))
	```

	## vLLM Deployment

	Install vLLM in a GPU environment with compatible CUDA/PyTorch; see the [Qwen3 vLLM guide](https://qwen.readthedocs.io/en/stable/deployment/vllm.html).

	```bash
	pip install -U "vllm>=0.9.0" openai
	```

	### Start the Server

	Hugging Face:

	```bash
	export VLLM_USE_MODELSCOPE=false
	vllm serve TCMLLM/Lingdan-4B-Base \
	--served-model-name Lingdan-4B-Base \
	--host 127.0.0.1 \
	--port 8000 \
	--max-model-len 16384
	```

	Replace the repository ID with a local model directory to reuse downloaded files. For two GPUs, append `--tensor-parallel-size 2`. Adjust the 16,384-token context limit (input + output) to available GPU memory.

	### Completion Client

	```python
	from openai import OpenAI

	client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
	response = client.completions.create(
	model="Lingdan-4B-Base",
	prompt="中医辨证论治的基本特点是",
	max_tokens=512,
	temperature=0.6,
	top_p=0.95,
	extra_body={"top_k": 20},
	)
	print(response.choices[0].text)
	```

	`EMPTY` is a placeholder for the local server started without API-key authentication.

	## Data Privacy and Responsible Use

	Lingdan-V2 is for research and does not replace qualified clinical judgment. Private clinical data are not released. Retrospective prescription agreement is not evidence of clinical safety or effectiveness; use de-identified inputs and professional review for any clinical application.

	## License

	Lingdan-V2 model weights and project-owned documentation and images are released under the [Apache License 2.0](https://github.com/TCMAI-BJTU/Lingdan-V2/blob/main/LICENSE). See [NOTICE](https://github.com/TCMAI-BJTU/Lingdan-V2/blob/main/NOTICE) for attribution notices. Third-party data and materials remain subject to their original terms.

	## Citation

	Cite Lingdan-V2 and the earlier Lingdan work:

	```bibtex
	@misc{hua2026lingdanv2,
	title = {{Lingdan-V2}: Large language models for clinically grounded reasoning in traditional Chinese medicine},
	author = {Hua, Rui and Wei, Yu and Jin, Ziri and Sun, Zhuo and Chang, Kai and Xia, Jianan and Shu, Zixin and Li, Xiaodong and Jia, Dongmei and Dong, Fei and Zhang, Runshun and Yu, Jian and Xu, Hao and Yu, Haibin and Li, Jiansheng and Liu, Baoyan and Wang, Wenjia and Zhou, Xuezhong},
	year = {2026},
	url = {https://github.com/TCMAI-BJTU/Lingdan-V2}
	}

	@article{hua2024lingdan,
	title={Lingdan: enhancing encoding of traditional Chinese medicine knowledge for clinical reasoning tasks with large language models},
	author={Hua, Rui and Dong, Xin and Wei, Yu and Shu, Zixin and Yang, Pengcheng and Hu, Yunhui and Zhou, Shuiping and Sun, He and Yan, Kaijing and Yan, Xijun and others},
	journal={Journal of the American Medical Informatics Association},
	volume={31},
	number={9},
	pages={2019--2029},
	year={2024},
	publisher={Oxford University Press}
	}
	```

	## Related Links

	[GitHub](https://github.com/TCMAI-BJTU/Lingdan-V2) · [Hugging Face collection](https://huggingface.co/collections/TCMLLM/lingdan-v2) · [ModelScope collection](https://modelscope.cn/collections/TCMAIBJTU/Lingdan-V2) · [LingLan benchmark](https://github.com/TCMAI-BJTU/LingLan)

	For task prompts and thinking-mode chat, see the [SFT/R1 guide](https://github.com/TCMAI-BJTU/Lingdan-V2#task-prompts).