Instructions to use Nanbeige/Nanbeige4.1-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nanbeige/Nanbeige4.1-3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nanbeige/Nanbeige4.1-3B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Nanbeige/Nanbeige4.1-3B")
model = AutoModelForCausalLM.from_pretrained("Nanbeige/Nanbeige4.1-3B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Nanbeige/Nanbeige4.1-3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nanbeige/Nanbeige4.1-3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanbeige/Nanbeige4.1-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nanbeige/Nanbeige4.1-3B

SGLang

How to use Nanbeige/Nanbeige4.1-3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nanbeige/Nanbeige4.1-3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanbeige/Nanbeige4.1-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nanbeige/Nanbeige4.1-3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanbeige/Nanbeige4.1-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Nanbeige/Nanbeige4.1-3B with Docker Model Runner:
```
docker model run hf.co/Nanbeige/Nanbeige4.1-3B
```

Nanbeige4.1-3B

Commit History

Update tokenizer_config.json

9c4555c
verified

flust commited on Mar 25

Add syntax highlight in python markdown code snippets (#34)

6f3b2c3

leran1995

RahulSharma0 commited on Feb 26

increase the max_new_tokens in the demo (#33)

971753c

leran1995

bitsnaps commited on Feb 26

Update README.md

2d78edc
verified

leran1995 commited on Feb 22

Update README.md

7e64fe9
verified

leran1995 commited on Feb 19

Update README.md

988060c
verified

leran1995 commited on Feb 17

Update README.md

b32903a
verified

leran1995 commited on Feb 13

Upload Nanbeige4.1-3B-Report.pdf

57b5051
verified

leran1995 commited on Feb 13

Update config.json

0133423
verified

flust commited on Feb 11

Upload model_performance_comparison.png

5fa91af
verified

flust commited on Feb 11

Delete figures/model_performance_comparison.png

b7eedac
verified

flust commited on Feb 11

Update README.md

da13918
verified

flust commited on Feb 11

Upload model_performance_comparison.png

28f4b2e
verified

leran1995 commited on Feb 11

Update README.md

d02c34c
verified

leran1995 commited on Feb 11

Delete figures/1

8109f12
verified

leran1995 commited on Feb 11

Upload nbg.png

9c1da7f
verified

leran1995 commited on Feb 11

Create figures/1

810318d
verified

leran1995 commited on Feb 11

Upload tokenizer_config_search.json

253dcb4
verified

flust commited on Feb 11

Upload 2 files

665db78
verified

chenzongchao commited on Feb 10

Upload 2 files

1f158ff
verified

chenzongchao commited on Feb 10

Upload 6 files

346a63d
verified

chenzongchao commited on Feb 10

initial commit

7d222b4
verified

leran1995 commited on Feb 10

Commit History

Update tokenizer_config.json 9c4555c verified

Add syntax highlight in python markdown code snippets (#34) 6f3b2c3

increase the max_new_tokens in the demo (#33) 971753c

Update README.md 2d78edc verified

Update README.md 7e64fe9 verified

Update README.md 988060c verified

Update README.md b32903a verified

Upload Nanbeige4.1-3B-Report.pdf 57b5051 verified

Update config.json 0133423 verified

Upload model_performance_comparison.png 5fa91af verified

Delete figures/model_performance_comparison.png b7eedac verified

Update README.md da13918 verified

Upload model_performance_comparison.png 28f4b2e verified

Update README.md d02c34c verified

Delete figures/1 8109f12 verified

Upload nbg.png 9c1da7f verified

Create figures/1 810318d verified

Upload tokenizer_config_search.json 253dcb4 verified

Upload 2 files 665db78 verified

Upload 2 files 1f158ff verified

Upload 6 files 346a63d verified

initial commit 7d222b4 verified

Update tokenizer_config.json

9c4555c
verified

Add syntax highlight in python markdown code snippets (#34)

6f3b2c3

increase the max_new_tokens in the demo (#33)

971753c

Update README.md

2d78edc
verified

Update README.md

7e64fe9
verified

Update README.md

988060c
verified

Update README.md

b32903a
verified

Upload Nanbeige4.1-3B-Report.pdf

57b5051
verified

Update config.json

0133423
verified

Upload model_performance_comparison.png

5fa91af
verified

Delete figures/model_performance_comparison.png

b7eedac
verified

Update README.md

da13918
verified

Upload model_performance_comparison.png

28f4b2e
verified

Update README.md

d02c34c
verified

Delete figures/1

8109f12
verified

Upload nbg.png

9c1da7f
verified

Create figures/1

810318d
verified

Upload tokenizer_config_search.json

253dcb4
verified

Upload 2 files

665db78
verified

Upload 2 files

1f158ff
verified

Upload 6 files

346a63d
verified

initial commit

7d222b4
verified