Instructions to use inclusionAI/Ling-2.5-1T with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use inclusionAI/Ling-2.5-1T with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="inclusionAI/Ling-2.5-1T", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("inclusionAI/Ling-2.5-1T", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use inclusionAI/Ling-2.5-1T with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "inclusionAI/Ling-2.5-1T"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inclusionAI/Ling-2.5-1T",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/inclusionAI/Ling-2.5-1T

SGLang

How to use inclusionAI/Ling-2.5-1T with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "inclusionAI/Ling-2.5-1T" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inclusionAI/Ling-2.5-1T",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "inclusionAI/Ling-2.5-1T" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "inclusionAI/Ling-2.5-1T",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use inclusionAI/Ling-2.5-1T with Docker Model Runner:
```
docker model run hf.co/inclusionAI/Ling-2.5-1T
```

zhanghanxiao commited on Feb 15

Commit

6d0e711

verified ·

1 Parent(s): 0eaf464

Add files using upload-large-folder tool

Browse files

Files changed (21) hide show

model-00016-of-00163.safetensors +3 -0
model-00026-of-00163.safetensors +3 -0
model-00038-of-00163.safetensors +3 -0
model-00055-of-00163.safetensors +3 -0
model-00058-of-00163.safetensors +3 -0
model-00063-of-00163.safetensors +3 -0
model-00067-of-00163.safetensors +3 -0
model-00070-of-00163.safetensors +3 -0
model-00082-of-00163.safetensors +3 -0
model-00094-of-00163.safetensors +3 -0
model-00097-of-00163.safetensors +3 -0
model-00099-of-00163.safetensors +3 -0
model-00102-of-00163.safetensors +3 -0
model-00104-of-00163.safetensors +3 -0
model-00118-of-00163.safetensors +3 -0
model-00123-of-00163.safetensors +3 -0
model-00126-of-00163.safetensors +3 -0
model-00128-of-00163.safetensors +3 -0
model-00129-of-00163.safetensors +3 -0
model-00153-of-00163.safetensors +3 -0
model-00158-of-00163.safetensors +3 -0

model-00016-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f710f79a48832c6c85ee14cf18c0464e9262e9323f72973e0a9b7300c2e3aefc
+size 8795505224

model-00026-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d74f6b2f302c5c05aa5385a2e328e8afa0e8535bb13b71f0394fc2fb34053f07
+size 17179933872

model-00038-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e7ab54054f2595ff3b684d81857f5e191c2cdb6830ba2b6e98bbb88bdf683ac0
+size 17179933872

model-00055-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e9ef5bc0822a5ecc31c12e72a5ec9926bc3c5a0d79a341a42a98ef0f94c511b1
+size 17179933872

model-00058-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fa1c71e252e8a17437c4527142bb110376a8f371e237ca511332a5174e3defdb
+size 17179933872

model-00063-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f8166f004533787f00b7452b9a0b25184fc8930a97c4220d7dd7352e29198f61
+size 9198141656

model-00067-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6760d615b5dc911b341f80bb362c5854bd009d67bec2df0696f45b1a5d023f26
+size 8795505224

model-00070-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bbaa6a2eeb596b69166f39a4d1a0cd692a7fe618cd00260cb0fdafa6c01724b4
+size 17179933872

model-00082-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:aef244b303e557b015514e294f21395e69694f5b1169e7bc158a25b4b20f3b4c
+size 10151509016

model-00094-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:149282a42fd6eeda10b44551bb2c4eeeed204ae75d0c64fbed891ee1b0b719f6
+size 17179933872

model-00097-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ae8e28caabf5f9de9742db3edbd91c9627001e28a24953307cbeb9f7769bf621
+size 9198141656

model-00099-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2c475c0755c1624b4c644e78aa980791e851389a4ded5631c6a804a81d46f1eb
+size 10151509016

model-00102-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8661be04f020fe4311d7dbe3a6fbbeeee5c37d8ededb4c8ede30194ae7c3b102
+size 17179933872

model-00104-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bbf68f52c3a7b71117056485acc584448fac1132f46e62b7c8fe92352c1e01f7
+size 17179933872

model-00118-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7dbb981feed54c79158a49a71789b3b5649a5271c708592ccf9e21122861fc1f
+size 8795505224

model-00123-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5cb4a1ddd150f1c6f218d35bddaa45dadbd88f6c04e050c2e89f03d0abadfe00
+size 17179933872

model-00126-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:191aba9ad9b94a7a117df334c88f1b84d11137a4200afef1f62d3e155384330a
+size 17179933872

model-00128-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f416baa2d4dddd91318c72918969472729658dab01c5254dd3b85a9c04fcfae7
+size 17179933872

model-00129-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:12119a1d17d6d5381958964af26eafbf9597ab5c65180c8d68e76e9ef8c05a9e
+size 8795488344

model-00153-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:94488c25b31856f3ccd86b102e3e5a422dcdd68d2efcae1a63fb2b89aa14c209
+size 17179933872

model-00158-of-00163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3c1b9812d8917fc876e56d5e620baf347e8f0a4265e614ed8764fe682bb16a07
+size 10272071736