update readme

README.md

@@ -4,6 +4,9 @@ pipeline_tag: image-text-to-text
 tags:
 - minicpm-v
 - multimodal
+- On-Device Model
+- lightweight
+library_name: transformers
 ---
 
 A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone
@@ -11,6 +14,12 @@ A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Ph
 [GitHub](https://github.com/OpenBMB/MiniCPM-o) | [CookBook](https://github.com/OpenSQZ/MiniCPM-V-CookBook) | [Demo](https://huggingface.co/spaces/openbmb/MiniCPM-V-4.6-Thinking-Demo) |
 [Feishu (Lark)](https://raw.githubusercontent.com/openbmb/MiniCPM-V/main/assets/feishu_qrcode.png)
 
+## News
+
+* [2026.05.17] ⭐️⭐️⭐️ We release the API service of MiniCPM-V 4.6, with a **public free API key** together! Try [it](https://github.com/OpenBMB/MiniCPM-V/blob/main/docs/api.md) now.
+
+
+
 ## MiniCPM-V 4.6 Thinking
 
 **MiniCPM-V 4.6 Thinking** is the long chain-of-thought reasoning variant of [MiniCPM-V 4.6](https://huggingface.co/openbmb/MiniCPM-V-4.6). It generates an explicit reasoning trace before producing the final answer, substantially boosting performance on complex multimodal reasoning, math, and OCR-heavy tasks, while keeping the same edge-friendly architecture (SigLIP2-400M vision encoder + Qwen3.5-0.8B LLM) and the mixed 4x/16x visual token compression of MiniCPM-V 4.6.
@@ -198,9 +207,9 @@ You can customize image/video processing by passing additional parameters to `ap
 |-----------|---------|------------|-------------|
 | `downsample_mode` | `"16x"` | Image & Video | Visual token downsampling. `"16x"` merges tokens for efficiency; `"4x"` keeps 4× more tokens for finer detail. Must also be passed to `generate()`. |
 | `max_slice_nums` | `9` | Image & Video | Maximum number of slices when splitting a high-resolution image. Higher values preserve more detail for large images. Recommended: `36` for image, `1` for video. |
-| `max_num_frames` | `128` | Video only | Maximum number of main frames sampled from the video. |
-| `stack_frames` | `1` | Video only | Total sample points per second. `1` = main frame only (no stacking). `N` (N>1) = 1 main frame + N−1 sub-frames per second; the sub-frames are composited into a grid image and interleaved with main frames. Recommended: `3` or `5`. |
-| `use_image_id` | `True` | Image & Video | Whether to prepend `<image_id>N</image_id>` tags before each image/frame placeholder. Recommended: `True` for image, `False` for video. |
+| `max_num_frames` | `128` | Video only | The `max_num_frames` parameter dynamically controls the temporal context length and prevents VRAM overflow: <br> **Short Videos** (duration ≤ `max_num_frames` sec): The processor defaults to **1 FPS**, capturing second-by-second details without hitting the upper limit. <br> **Long Videos** (duration > `max_num_frames` sec): The processor automatically switches to **uniform sampling**, selecting exactly `max_num_frames` evenly spaced across the entire timeline. |
+| `stack_frames` | `1` | Video only | Total sample points per second. `1` = main frame only (no stacking). `N` (N>1) = 1 main frame + N−1 sub-frames per second; the sub-frames are composited into a grid image and interleaved with main frames. Recommended setting is `1` for short videos, and `3` or `5` for long videos. |
+| `use_image_id` | `True` | Image & Video | Whether to prepend `<image_id>N</image_id>` tags before each image/frame placeholder. Set `True` for image, `False` for video. |
 
 > **Note:** `downsample_mode` must be passed to **both** `apply_chat_template` (for correct placeholder count) and `generate` (for the vision encoder). All other parameters only need to be passed to `apply_chat_template`.
 
@@ -235,6 +244,57 @@ curl -s http://localhost:8000/v1/chat/completions \
   }'
 ```
 
+Tool calling example:
+
+```bash
+curl -s http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
+  "model": "openbmb/MiniCPM-V-4.6-Thinking",
+  "messages": [{"role": "user", "content": [
+    {"type": "text", "text": "the weather of Beijing"}
+  ]}],
+  "tools": [{
+    "type": "function",
+    "function": {
+      "name": "get_weather",
+      "description": "Get the current weather for a given location",
+      "parameters": {
+        "type": "object",
+        "properties": {
+          "location": {"type": "string", "description": "City name"}
+        },
+        "required": ["location"]
+      }
+    }
+  }]
+}'
+```
+
+The model returns a natural-language explanation followed by a structured <tool_call> block embedded in the content field. Note that a dedicated tool call parser for this format has not yet been added to the transformers library, so the tool calls need to be extracted manually via regex for now.
+
+```
+{
+    "id": "f4f09c7d-8045-4cb1-ade9-07aa5dee637d",
+    "choices": [
+        {
+            "finish_reason": "stop",
+            "index": 0,
+            "message": {
+                "content": "I need to check the current weather for Beijing, so I will call the get_weather function.\n\n<tool_call>\n<function=get_weather>\n<parameter=location>\nBeijing\n</parameter>\n</function>\n</tool_call>",
+                "role": "assistant"
+            }
+        }
+    ],
+    "created": 1778748859,
+    "model": "openbmb/MiniCPM-V-4.6-Thinking@main",
+    "object": "chat.completion",
+    "usage": {
+        "completion_tokens": 47,
+        "prompt_tokens": 283,
+        "total_tokens": 330
+    }
+}
+```
+
 #### Handling Escaped Newlines in Model Outputs <!-- omit in toc -->
 
 In some cases, the model might output escaped newline characters `\n` as string literals instead of actual newlines. To render the text correctly, especially in UI layers, you can use the following utility function. This function carefully replaces literal `\n` with real newlines while protecting scenarios where `\n` has specific semantic meaning.
@@ -269,6 +329,7 @@ def normalize_response_text(text: str) -> str:
 
 We have adapted MiniCPM-V 4.6 for deployment on **iOS, Android, and HarmonyOS** platforms, with **all edge adaptation code fully open-sourced**. Developers can reproduce the on-device experience in just a few steps. Visit our [edge deployment repository](https://github.com/OpenBMB/MiniCPM-V-edge-demo) for platform-specific build guides, or go to the [download page](https://github.com/OpenBMB/MiniCPM-V-edge-demo/blob/main/DOWNLOAD.md) to try pre-built apps directly.
 
+<a id="inference-and-training"></a>
 #### Use MiniCPM-V 4.6 in Other Inference and Training Frameworks <!-- omit in toc -->
 
 MiniCPM-V 4.6 supports multiple inference and training frameworks. Below are quick-start commands for each. For full details, see our [Cookbook](https://github.com/OpenSQZ/MiniCPM-V-CookBook).
@@ -407,7 +468,7 @@ swift sft --model_type minicpm-v-4_6 --dataset <your-dataset>
 
 **Technical Reports:** [MiniCPM-o 4.5](https://huggingface.co/papers/2604.27393) | [MiniCPM-V 4.5](https://arxiv.org/abs/2509.18154) | [MiniCPM-o 2.6](https://openbmb.notion.site/MiniCPM-o-2-6-A-GPT-4o-Level-MLLM-for-Vision-Speech-and-Multimodal-Live-Streaming-on-Your-Phone-185ede1b7a558042b5d5e45e6b237da9) | [MiniCPM-Llama3-V 2.5](https://arxiv.org/abs/2408.01800) | [MiniCPM-V 2.0](https://openbmb.vercel.app/minicpm-v-2)
 
-**Other Multimodal Projects:** [VisCPM](https://github.com/OpenBMB/VisCPM/tree/main) | [RLPR](https://github.com/OpenBMB/RLPR) | [RLHF-V](https://github.com/RLHF-V/RLHF-V) | [LLaVA-UHD](https://github.com/thunlp/LLaVA-UHD) | [RLAIF-V](https://github.com/RLHF-V/RLAIF-V)
+**Other Multimodal Projects:** [VisCPM](https://github.com/OpenBMB/VisCPM/tree/main) | [RLPR](https://github.com/OpenBMB/RLPR) | [RLHF-V](https://github.com/RLHF-V/RLHF-V) | [LLaVA-UHD](https://github.com/thunlp/LLaVA-UHD) | [RLAIF-V](https://github.com/RLHF-V/RLAIF-V) | [LLaVA-UHD-v4](https://arxiv.org/abs/2605.08985 )
 
 
 ## Citation <!-- omit in toc -->
@@ -415,13 +476,6 @@ swift sft --model_type minicpm-v-4_6 --dataset <your-dataset>
 If you find our model/code/paper helpful, please consider citing our papers 📝 and staring us ⭐️！
 
 ```bib
-@misc{cui2026minicpmo45realtimefullduplex,
-      title={MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction}, 
-      author={Junbo Cui and Bokai Xu and Chongyi Wang and Tianyu Yu and Weiyue Sun and Yingjing Xu and Tianran Wang and Zhihui He and Wenshuo Ma and Tianchi Cai and others},
-      year={2026},
-      url={https://arxiv.org/abs/2604.27393}, 
-}
-
 @proceedings{yu2025minicpmv45cookingefficient,
       title={MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe}, 
       author={Tianyu Yu and Zefan Wang and Chongyi Wang and Fuwei Huang and Wenshuo Ma and Zhihui He and Tianchi Cai and Weize Chen and Yuxiang Huang and Yuanqian Zhao and others},