Instructions to use Nanbeige/Nanbeige4.2-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nanbeige/Nanbeige4.2-3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nanbeige/Nanbeige4.2-3B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Nanbeige/Nanbeige4.2-3B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Nanbeige/Nanbeige4.2-3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nanbeige/Nanbeige4.2-3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanbeige/Nanbeige4.2-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nanbeige/Nanbeige4.2-3B

SGLang

How to use Nanbeige/Nanbeige4.2-3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nanbeige/Nanbeige4.2-3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanbeige/Nanbeige4.2-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nanbeige/Nanbeige4.2-3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanbeige/Nanbeige4.2-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Nanbeige/Nanbeige4.2-3B with Docker Model Runner:
```
docker model run hf.co/Nanbeige/Nanbeige4.2-3B
```

Amazing for its size, but spirals into questionable solutions once it fails something

#21

by Mk2Oracle - opened 4 days ago

Discussion

Mk2Oracle

4 days ago

So, honestly I hope the team creates a version 2 of this or maybe a better iteration. I actually liked this model. The main issue is that when it comes to git usage and submodules it started spiraling on undoing 2 commits to pluck a file out and send out the new changes with a force push.

The task was simple. Just put a file into .gitignore and align the submodule but... it started doing unnecessarily complex mutations not using git commands but literally started mutating the files INSIDE .git which was a massive red flag. because it "needed to fix the indexing"

Things that I loved:

Not perfect, but really loved that it went through pivotal thinking on how to solve the issue
kept it realistically simple when asked to plan ahead rather than "go in there and fix the issue"
analyzed the things I asked without expanding the scope

Project exposed to:

Wordpress site with 4 unique plugins
PHP language, nodejs, react for gutenberg blocks
No quantizations created or used (e.g. compress to q4), used the native safetensors provided with vLLM no additional parameters

leran1995

Nanbeige LLM Lab org 3 days ago

Hi, thanks a lot for sharing the detailed feedback. We're really happy to see our 3B model being tested in a real SWE workflow like this. The positive parts you mentioned, such as planning before acting and keeping the scope constrained, are exactly the kind of agent behavior we care about.

At the same time, the .git mutation behavior you described is definitely not what we want. In future iterations, we plan to further improve both the model's final task performance and its intermediate process discipline. In particular, better turn-level filtering in SFT data and stronger process/action-centric rewards during RL are on our roadmap. The goal is not only to make the final result better, but also to make the steps in between safer, cleaner, and aligned with standard engineering practice.

Thanks again for the very useful report. ❤️

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment