Instructions to use AXERA-TECH/MiniCPM-V-4.6-AX637 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AXERA-TECH/MiniCPM-V-4.6-AX637 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="AXERA-TECH/MiniCPM-V-4.6-AX637")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("AXERA-TECH/MiniCPM-V-4.6-AX637", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AXERA-TECH/MiniCPM-V-4.6-AX637 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AXERA-TECH/MiniCPM-V-4.6-AX637"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AXERA-TECH/MiniCPM-V-4.6-AX637",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/AXERA-TECH/MiniCPM-V-4.6-AX637

SGLang

How to use AXERA-TECH/MiniCPM-V-4.6-AX637 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AXERA-TECH/MiniCPM-V-4.6-AX637" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AXERA-TECH/MiniCPM-V-4.6-AX637",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AXERA-TECH/MiniCPM-V-4.6-AX637" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AXERA-TECH/MiniCPM-V-4.6-AX637",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use AXERA-TECH/MiniCPM-V-4.6-AX637 with Docker Model Runner:
```
docker model run hf.co/AXERA-TECH/MiniCPM-V-4.6-AX637
```

MiniCPM-V-4.6 on AXERA NPU

Ready-to-run deployment package for openbmb/MiniCPM-V-4.6 on AX637 / NPU1.

This release packages the AX637 axllm runtime together with the compiled text and vision .axmodel files.
The packaged text runtime uses the non-GPTQ w8a16 build.
The packaged vision runtime uses a fixed-shape 448x448 MiniCPM-V-4.6 vision encoder.
The package supports text-only chat, single-image understanding, and video understanding through the OpenAI-compatible axllm serve API.
The package is intended for board-side deployment through the bundled axllm runtime.

Supported Platform

AX637 / NPU1

Validated Devices

This package has been validated on the following AX637-based device:

AX637 / NPU1 development board

Performance

All measurements below were taken on AX637 / NPU1. TTFT stands for time to first token. In this table, TTFT is measured end-to-end from request arrival at axllm serve to the first generated token, so the multimodal rows include media preprocessing and vision encoding time.

The text-only smoke prompt, What is 1+1? Reply with the number only., was kept within one 128-token prefill chunk. To avoid one-time startup effects, the text row below excludes the first request after service startup. Its Decode figure was measured with longer text-only generations (max_tokens=256) to better reflect sustained decode throughput; very short smoke replies under-report decode speed because EOS and response-tail overhead become relatively larger. The image row was measured with the packaged fixed-shape 448x448 vision encoder, assets/sample.png, and What animal is the main subject in this image? Reply in Chinese only.. The video row used the packaged sample video with video:assets/red-panda-openai.mp4:2 and Describe this video briefly..

Scenario	Input tokens	Prefill chunks	TTFT	Decode
Text-only smoke prompt	`25`	`1 x 128`	`508.16 ms avg` (`507.56-508.57 ms`)	`7.24 token/s avg`
Image prompt	`97`	`1 x 128`	`1432.85 ms avg` (`1351.51-1593.75 ms`)	`7.33 token/s avg`
Video prompt	`1140`	`9 x 128`	`19469.71 ms avg` (`17852.00-22700.51 ms`)	`6.89 token/s avg`

The packaged runtime uses the following context layout:

prefill_len=128
kv_cache_len=1535
prefill_max_token_num=1152

Input tokens in the table above refers to the full request length after chat templating, not just the visual soft tokens. For the shipped 448x448 vision encoder, each selected image block contributes 64 visual soft tokens. Under the current packaged runtime settings, the sample video request in this README uses 1140 total input tokens and spans 9 prefill chunks.

Startup Runtime Footprint

Item	Value
`Flash total (text + post + vision axmodels)`	`1.31 GiB` (`1344.35 MiB`)
`Package flash total (excluding vision_cache/)`	`1.84 GiB` (`1884.46 MiB`)
`Runtime CMM increment during board-side startup`	`1.40 GiB` (`1435.39 MiB`)

The runtime CMM value above was measured during board-side startup on the validated AX637 board configuration and should be treated as a practical reference value.

Vision Encoder Latency

Measured on AX637 / NPU1 with /opt/bin/ax_run_model -m minicpmv4_6_vision_448.axmodel -g 0 -w 1 -r 5.

Model	Resolution	Soft Tokens	Time (ms)
`minicpmv4_6_vision_448.axmodel`	`448x448`	`64`	`543.970 ms avg`

For this packaged AX637 runtime, the visual token count is fixed by the shipped vision encoder configuration:

vision_width = 448
vision_height = 448
vision_patch_size = 14
patch grid = (448 / 14) x (448 / 14) = 32 x 32
raw patch tokens = 32 x 32 = 1024
current packaged build uses the 16x visual compression path
Soft Tokens = 1024 / 16 = 64

So, for the fixed-shape runtime shipped in this repository, the relation is:

Soft Tokens = (vision_width / patch_size) x (vision_height / patch_size) / 16

Input tokens in the performance table can be larger than the visual Soft Tokens because axllm counts the full templated request, including user text and chat-template tokens in addition to the visual tokens. For the packaged assets/sample.png request in this README, the runtime reports input_num_token=97, which still fits within a single 128-token prefill chunk.

Soft Tokens is not a runtime-configurable value in this package. This repository ships only minicpmv4_6_vision_448.axmodel, so the board-side AX637 runtime always uses 448x448 -> 64 soft tokens for image encoding.

Package Layout

.
├── README.md
├── bin/
│   ├── axllm
│   └── axllm.version.json
├── lib/
│   └── runtime shared libraries
├── assets/
│   ├── openai_api_demo.png
│   ├── red-panda-openai.mp4
│   └── sample.png
├── minicpmv4_6_vision_448.axmodel
├── qwen3_5_text_p128_l0_together.axmodel
├── ...
├── qwen3_5_text_p128_l23_together.axmodel
├── qwen3_5_text_post.axmodel
├── model.embed_tokens.weight.bfloat16.bin
├── config.json
├── post_config.json
└── minicpm_v46_tokenizer.txt

The packaged axllm runtime, runtime shared libraries, and compiled .axmodel files live at the repository root.

Sample Image

The axllm flow uses the sample image: assets/sample.png

Sample Video

The package also includes a packaged sample video for board-side video understanding validation:

assets/red-panda-openai.mp4

Direct Inference with `axllm`

The axllm workflow is still being refined. The instructions below reflect the current validated flow and may be adjusted as the packaging continues to evolve.

Download the Model Package

Download the release package from Hugging Face:

mkdir -p AXERA-TECH/MiniCPM-V-4.6-AX637
cd AXERA-TECH/MiniCPM-V-4.6-AX637
hf download AXERA-TECH/MiniCPM-V-4.6-AX637 --local-dir .

Install `axllm`

Option 1: use the validated binary included in this repository:

chmod +x ./bin/axllm

Option 2: install axllm from the public repository:

git clone -b axllm https://github.com/AXERA-TECH/ax-llm.git
cd ax-llm
./install.sh

Option 3: install with a one-line command:

curl -fsSL https://raw.githubusercontent.com/AXERA-TECH/ax-llm/axllm/install.sh | bash

Option 4: download the prebuilt binary from GitHub Actions CI:

If you do not have a local build environment, download the latest CI-generated axllm binary from GitHub Actions: https://github.com/AXERA-TECH/ax-llm/actions?query=branch%3Aaxllm Then run:

chmod +x axllm
sudo mv axllm /usr/bin/axllm

Run on the Board

The package root is already arranged for axllm, so no extra runtime path arguments are required.

For multimodal testing, you can use the packaged sample image shown above: ./assets/sample.png, or the packaged sample video: ./assets/red-panda-openai.mp4.

./bin/axllm run .

In interactive mode:

press Enter directly for text-only chat
input an image path for single-image chat
input video:/path/to/frames_dir or video:/path/to/video.mp4 for video chat

Serve with `axllm`

From the package root on the board:

./bin/axllm serve . --port 8000

Expected model id:

AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16

Health check:

curl http://127.0.0.1:8000/health

A typical startup log looks like this:

INF Init | LLM init start
INF Init | mixed attention enabled: full_attention_interval=4 ref_full_layer_idx=3
INF Init | attention config: layers=24 sliding=0 full=6 linear=18 sliding_window=0 ref_full_layer_idx=3
tokenizer_type = 3
huggingface tokenizer mode = gpt2_byte_bpe
...
INF Init | max_token_len : 1535
INF Init | kv_cache_size : 512, kv_cache_num: 1535
INF init_groups_from_model | prefill_token_num : 128
INF init_groups_from_model | prefill_max_token_num : 1152
INF Init | MiniCPM-V-4.6 token ids: image_pad=248056 video_pad=248057
INF Init | VisionModule init ok: type=MiniCPMV46VL, tokens_per_block=64, embed_size=1024, out_dtype=fp32
INF Init | LLM init ok
Starting server on port 8000 with model 'AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16'...
API URLs:
  GET  http://127.0.0.1:8000/health
  GET  http://127.0.0.1:8000/v1/models
  POST http://127.0.0.1:8000/v1/chat/completions
OpenAI API Server starting on http://0.0.0.0:8000
Max concurrency: 1
Models: AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16

You can then send requests to the server using the API endpoints shown in the log. For example, to check the health status and list the available models:

curl http://127.0.0.1:8000/health
curl http://127.0.0.1:8000/v1/models

Example output:

{
  "concurrency": 0,
  "max_concurrency": 1,
  "status": "healthy"
}
{
  "data": [
    {
      "created": 1780908633,
      "id": "AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16",
      "object": "model",
      "owned_by": "openai-api"
    }
  ],
  "object": "list"
}

Text Request

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "中国和美国的首都分别是哪里"}
        ]
      }
    ],
    "max_tokens": 32
  }'

Example output:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "中国的首都是北京，美国的首都是华盛顿。"
      },
      "finish_reason": "stop"
    }
  ],
  "model": "AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16",
  "object": "chat.completion"
}

Image Request

python3 - <<'PY'
import base64
import json
from pathlib import Path
from urllib.request import Request, urlopen

img = Path("assets/sample.png").read_bytes()
payload = {
    "model": "AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16",
    "messages": [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What animal is the main subject in this image? Reply in Chinese only."},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "data:image/png;base64," + base64.b64encode(img).decode()
                    },
                },
            ],
        }
    ],
    "max_tokens": 32,
}
req = Request(
    "http://127.0.0.1:8000/v1/chat/completions",
    data=json.dumps(payload).encode(),
    headers={"Content-Type": "application/json"},
)
with urlopen(req, timeout=60) as resp:
    print(resp.read().decode())
PY

Example output:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "龙虾"
      },
      "finish_reason": "stop"
    }
  ],
  "model": "AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16",
  "object": "chat.completion"
}

Video Request

axllm serve accepts either a frames directory or a raw video file:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "image_url", "image_url": {"url": "video:/path/to/frames_dir"}},
          {"type": "text", "text": "Describe this video briefly."}
        ]
      }
    ],
    "max_tokens": 128
  }'

For a raw video file, use video:/path/to/video.mp4. If you need to request a specific sampling FPS, use the form video:/path/to/video.mp4:2.

To test the packaged sample video from the package root, you can set:

VIDEO_PATH="$(pwd)/assets/red-panda-openai.mp4"

and then use video:${VIDEO_PATH}:2 in the request payload.

Example output:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Two red pandas are playing with a rope. One is on a branch, the other is on the ground."
      },
      "finish_reason": "stop"
    }
  ],
  "model": "AXERA-TECH/MiniCPM-V-4.6-AX637-C128-P1152-CTX1535-w8a16",
  "object": "chat.completion"
}

Conversion References

If you need the original model files or want to rebuild the deployment artifacts, start with:

Original Hugging Face model: openbmb/MiniCPM-V-4.6
AXERA conversion and deployment workflow: AXERA-TECH/MiniCPM-V-4.6.axera

Discussion

GitHub Issues
QQ group: 139953715

Downloads last month: 32

Inference Providers NEW

Image-Text-to-Text

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for AXERA-TECH/MiniCPM-V-4.6-AX637

Base model

openbmb/MiniCPM-V-4.6

Finetuned

(17)

this model

Collection including AXERA-TECH/MiniCPM-V-4.6-AX637

MiniCPM

Collection

8 items • Updated 7 days ago

MiniCPM-V-4.6 on AXERA NPU

Supported Platform

Validated Devices

Performance

Startup Runtime Footprint

Vision Encoder Latency

Package Layout

Sample Image

Sample Video

Direct Inference with axllm

Download the Model Package

Install axllm

Run on the Board

Serve with axllm

Text Request

Image Request

Video Request

Conversion References

Discussion

Model tree for AXERA-TECH/MiniCPM-V-4.6-AX637

Collection including AXERA-TECH/MiniCPM-V-4.6-AX637

Direct Inference with `axllm`

Install `axllm`

Serve with `axllm`