Instructions to use internlm/Intern-S2-Mobius with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use internlm/Intern-S2-Mobius with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="internlm/Intern-S2-Mobius", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModelForImageTextToText model = AutoModelForImageTextToText.from_pretrained("internlm/Intern-S2-Mobius", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use internlm/Intern-S2-Mobius with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "internlm/Intern-S2-Mobius" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/Intern-S2-Mobius", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/internlm/Intern-S2-Mobius
- SGLang
How to use internlm/Intern-S2-Mobius with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "internlm/Intern-S2-Mobius" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/Intern-S2-Mobius", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "internlm/Intern-S2-Mobius" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/Intern-S2-Mobius", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use internlm/Intern-S2-Mobius with Docker Model Runner:
docker model run hf.co/internlm/Intern-S2-Mobius
Add SGLang serving instructions
#3
by JustinTong - opened
README.md
CHANGED
|
@@ -76,6 +76,7 @@ Intern-S2-Mobius can be deployed using any of the following LLM inference framew
|
|
| 76 |
- LMDeploy
|
| 77 |
- Transformers
|
| 78 |
- vLLM
|
|
|
|
| 79 |
|
| 80 |
### Sampling Parameters
|
| 81 |
|
|
@@ -192,5 +193,32 @@ vllm serve \
|
|
| 192 |
--tool-call-parser qwen3_coder
|
| 193 |
```
|
| 194 |
|
|
|
|
| 195 |
|
|
|
|
| 196 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
- LMDeploy
|
| 77 |
- Transformers
|
| 78 |
- vLLM
|
| 79 |
+
- SGLang
|
| 80 |
|
| 81 |
### Sampling Parameters
|
| 82 |
|
|
|
|
| 193 |
--tool-call-parser qwen3_coder
|
| 194 |
```
|
| 195 |
|
| 196 |
+
### SGLang
|
| 197 |
|
| 198 |
+
Use the `lmsysorg/sglang:dev` Docker image or a recent source build with Intern-S2-Mobius support. See the [SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/InternLM/Intern-S2-Mobius) for Docker commands, verified deployment recipes, benchmarks, and usage examples.
|
| 199 |
|
| 200 |
+
- Serving With MTP (Recommended)
|
| 201 |
+
|
| 202 |
+
```bash
|
| 203 |
+
sglang serve \
|
| 204 |
+
--model-path internlm/Intern-S2-Mobius \
|
| 205 |
+
--trust-remote-code \
|
| 206 |
+
--tp 2 \
|
| 207 |
+
--reasoning-parser qwen3 \
|
| 208 |
+
--tool-call-parser qwen3_coder \
|
| 209 |
+
--speculative-algorithm NEXTN \
|
| 210 |
+
--speculative-num-steps 3 \
|
| 211 |
+
--speculative-eagle-topk 1 \
|
| 212 |
+
--speculative-num-draft-tokens 4
|
| 213 |
+
```
|
| 214 |
+
|
| 215 |
+
- Basic Serving Without MTP
|
| 216 |
+
|
| 217 |
+
```bash
|
| 218 |
+
sglang serve \
|
| 219 |
+
--model-path internlm/Intern-S2-Mobius \
|
| 220 |
+
--trust-remote-code \
|
| 221 |
+
--tp 2 \
|
| 222 |
+
--reasoning-parser qwen3 \
|
| 223 |
+
--tool-call-parser qwen3_coder
|
| 224 |
+
```
|