Instructions to use twnlp/ChineseErrorCorrector4-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use twnlp/ChineseErrorCorrector4-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="twnlp/ChineseErrorCorrector4-4B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("twnlp/ChineseErrorCorrector4-4B")
model = AutoModelForCausalLM.from_pretrained("twnlp/ChineseErrorCorrector4-4B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use twnlp/ChineseErrorCorrector4-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "twnlp/ChineseErrorCorrector4-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "twnlp/ChineseErrorCorrector4-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/twnlp/ChineseErrorCorrector4-4B

SGLang

How to use twnlp/ChineseErrorCorrector4-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "twnlp/ChineseErrorCorrector4-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "twnlp/ChineseErrorCorrector4-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "twnlp/ChineseErrorCorrector4-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "twnlp/ChineseErrorCorrector4-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use twnlp/ChineseErrorCorrector4-4B with Docker Model Runner:
```
docker model run hf.co/twnlp/ChineseErrorCorrector4-4B
```

Add library_name metadata

by nielsr HF Staff - opened 1 day ago

base: refs/heads/main

←

from: refs/pr/1

Discussion Files changed

+40

-59

nielsr

1 day ago

Hello! I'm Niels, part of the community science team at Hugging Face.

I noticed that this model repository doesn't have the library_name metadata in its README. Since the model card provides code snippets using the transformers library, I've opened this PR to add library_name: transformers to the YAML front matter.

Adding this metadata enables the "Use in Transformers" button and the inference widget on the model page, making it easier for researchers to use your work. I have also ensured the paper is linked in the introduction section for better visibility.

Add library_name metadataee5383f0

twnlp changed pull request status to merged about 24 hours ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment