Instructions to use deepseek-ai/DeepSeek-V3.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deepseek-ai/DeepSeek-V3.1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="deepseek-ai/DeepSeek-V3.1", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V3.1", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V3.1", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use deepseek-ai/DeepSeek-V3.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deepseek-ai/DeepSeek-V3.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V3.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/deepseek-ai/DeepSeek-V3.1

SGLang

How to use deepseek-ai/DeepSeek-V3.1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "deepseek-ai/DeepSeek-V3.1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V3.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "deepseek-ai/DeepSeek-V3.1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V3.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use deepseek-ai/DeepSeek-V3.1 with Docker Model Runner:
```
docker model run hf.co/deepseek-ai/DeepSeek-V3.1
```

Script to download deepseek tensor files

#39

by gariandamo - opened Aug 30, 2025

Discussion

gariandamo

Aug 30, 2025

#This script is provided by gariandamo under MIT license in honor of the good work of the DeepSeek team
#create a file like deepseek3-1_download.sh
#copy this code inside and safe
#replace the dummy token with your token that you created within your huggingface
#make the file runable with right click onfile - properties - rights -allow to run as programm ->save
#-> Execute in the folder you like to have the download
#Enjoy a robust scripted download especially on sides that have low internet connection for the safetensor files, rest to download manually

#!/bin/bash

Konfiguration

BASE_URL="https://huggingface.co/deepseek-ai/DeepSeek-V3.1/resolve/main"
TOKEN="insert_your_huggingface_token_here"
LOGFILE="download.log"
START=00001
END=00163

Sicherstellen, dass das Log existiert

touch "$LOGFILE"

Hauptloop

for i in $(seq -w $START $END); do
FILE="model-${i}-of-000163.safetensors"

Überspringen, wenn bereits im Logfile steht

if grep -q "$FILE" "$LOGFILE"; then
echo "✅ $FILE bereits abgeschlossen – überspringe."
continue
fi

echo "⬇️ Starte Download: $FILE"
echo "🔗 URL: ${BASE_URL}/${FILE}?download=true"

Direktdownload mit wget

wget -c "${BASE_URL}/${FILE}?download=true"
--header="Authorization: Bearer $TOKEN"
-O "$FILE"

Erfolgsprüfung

if [ $? -eq 0 ]; then
echo "$FILE" >> "$LOGFILE"
echo "✔️ $FILE erfolgreich abgeschlossen."
else
echo "⚠️ $FILE möglicherweise unvollständig – wird später fortgesetzt."
fi

Zufällige Wartezeit (50–180 Sekunden)

DELAY=$((RANDOM % 131 + 50))
echo "⏳ Warte $DELAY Sekunden vor nächstem Download..."
sleep "$DELAY"
done

echo "🟢 Downloadvorgang abgeschlossen oder pausiert. Du kannst das Skript jederzeit neu starten."

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment