Instructions to use lianghsun/TangYin with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lianghsun/TangYin with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lianghsun/TangYin")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lianghsun/TangYin")
model = AutoModelForCausalLM.from_pretrained("lianghsun/TangYin")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use lianghsun/TangYin with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lianghsun/TangYin"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/TangYin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lianghsun/TangYin

SGLang

How to use lianghsun/TangYin with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lianghsun/TangYin" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/TangYin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lianghsun/TangYin" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/TangYin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lianghsun/TangYin with Docker Model Runner:
```
docker model run hf.co/lianghsun/TangYin
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Model Card for TangYin

TangYin（唐寅） 是以 gemma-3-tw-270m-it 為基底，以中國古典詩詞對話資料（tw-poetry-chat）微調的 270M 級繁體中文詩詞對話模型。模型命名取自明代著名才子畫家「唐寅」，呼應其專長於古典詩詞解析、賞析與創作的設計初衷。

⚠️ 規格重點：本模型為 270M 參數 SLM、純文本單模態，主題聚焦在中國古典詩詞對話。

Model Details

通用模型在古典詩詞領域的表現常流於表面、且常出現張冠李戴的錯誤。TangYin 透過以七言詩等古典詩詞 chat 資料微調，使 270M 級小模型在詩詞翻譯、賞析、典故、用典等任務上具備穩定的表現，適合作為國文教學助理、文學陪聊與在地文化推廣場景的輕量化工具。

本模型是 Google DevFest Taipei 2025 的實作演示之一，用以展示 270M 級小模型透過領域微調可達成的應用效果。

核心特點 (Key Features)

古典詩詞專精：訓練資料聚焦在七言詩等古典詩詞題型，回答結構穩定。
270M 端側可部署：模型小巧，可在筆電 CPU、行動裝置上順暢執行。
DevFest 演示模型：作為「小模型 + 領域微調」設計實踐之公開範例。

Model Description

Developed by: Liang Hsun Huang
Funded by: APMIC
Base model: lianghsun/gemma-3-tw-270m-it
Model type: Gemma3ForCausalLM (Transformers)
Language(s) (NLP): Traditional Chinese（含古文）
License: gemma (Google usage license)
Finetuned from model: lianghsun/gemma-3-tw-270m-it

Model Sources

Repository: lianghsun/TangYin

Citation

@misc{tangyin,
  title        = {TangYin: A Lightweight Classical Chinese Poetry Chat Model},
  author       = {Huang, Liang Hsun},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/lianghsun/TangYin}}
}