Image-Text-to-Text
MLX
Safetensors
Korean
English
qwen3_5
gui-agent
web-agent
computer-use
browser
korean
conversational
4-bit precision
Instructions to use KORMo-VL/KORMo-Web-4B-mlx-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use KORMo-VL/KORMo-Web-4B-mlx-4bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("KORMo-VL/KORMo-Web-4B-mlx-4bit") config = load_config("KORMo-VL/KORMo-Web-4B-mlx-4bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use KORMo-VL/KORMo-Web-4B-mlx-4bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "KORMo-VL/KORMo-Web-4B-mlx-4bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "KORMo-VL/KORMo-Web-4B-mlx-4bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use KORMo-VL/KORMo-Web-4B-mlx-4bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "KORMo-VL/KORMo-Web-4B-mlx-4bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default KORMo-VL/KORMo-Web-4B-mlx-4bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use KORMo-VL/KORMo-Web-4B-mlx-4bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "KORMo-VL/KORMo-Web-4B-mlx-4bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "KORMo-VL/KORMo-Web-4B-mlx-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download tools.json from KORMo-VL/KORMo-Web-4B-mlx-4bit: direct link, hf CLI and curl.
- Browser
- Download file 12.4 kB
-
https://huggingface.co/KORMo-VL/KORMo-Web-4B-mlx-4bit/resolve/main/tools.json
- Command line
-
hf download hf://KORMo-VL/KORMo-Web-4B-mlx-4bit/tools.json
-
curl -L -o tools.json https://huggingface.co/KORMo-VL/KORMo-Web-4B-mlx-4bit/resolve/main/tools.json
12.4 kB
| [ | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "click", | |
| "description": "Left-click.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "x": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| }, | |
| "y": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| } | |
| }, | |
| "required": [ | |
| "x", | |
| "y" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "double_click", | |
| "description": "Double-click.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "x": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| }, | |
| "y": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| } | |
| }, | |
| "required": [ | |
| "x", | |
| "y" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "right_click", | |
| "description": "Right-click.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "x": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| }, | |
| "y": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| } | |
| }, | |
| "required": [ | |
| "x", | |
| "y" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "type", | |
| "description": "Type literal text into the input that currently has focus. It takes no coordinates: click (or focus) the target input first, then type. To replace existing text, use click → clear → type.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "text": { | |
| "type": "string" | |
| } | |
| }, | |
| "required": [ | |
| "text" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "hotkey", | |
| "description": "Press one key, or a key combination held down together, on the currently focused page/element. Pass the keys in order as a list: modifiers first, then the main key (e.g. [\"Control\", \"a\"]). Key names are case-sensitive and use these exact spellings (modifiers Control, Shift, Alt, Meta; named keys such as Enter, Escape, Tab, Backspace, Space, ArrowUp, ArrowDown, ArrowLeft, ArrowRight, Home, End, PageUp, PageDown, F1-F12; letters and digits in lowercase, e.g. a, 0, -). Keys and when to use them: Enter - submit a focused search box or form, or confirm a selection (use after type). Control+a - select all text in the focused input (e.g. before Backspace or before typing replacement text). Control+c - copy the current selection. Control+v - paste the clipboard into the focused input. Backspace - delete the character before the cursor, or the selected text. Tab / Shift+Tab - move focus to the next / previous element. Escape - close a popup, dialog, dropdown or autocomplete list. ArrowDown / ArrowUp - move through dropdown or autocomplete options, or scroll a little. ArrowLeft / ArrowRight - move the cursor in a text input, or switch slides/carousel items. Home / End - jump to the top / bottom of the page (or start / end of the input line). Control+Home / Control+End - jump to the very top / bottom of the page or document. PageDown / PageUp - scroll down / up by one screen. Space - toggle a focused checkbox/button, or scroll down. Control+f - open find-in-page. F3 - find the next match in find-in-page. Control+l / Alt+d / F6 - focus the address bar (prefer goto(url) to open a URL). Control+r / F5 - reload the current page. Control+Shift+r - hard reload the current page, ignoring the cache. Alt+ArrowLeft - go back in browser history (same as go_back). Alt+Home - open the browser home page. Control+t - open a new tab (prefer new_tab). Control+w - close the current tab (prefer close_tab). Control+n - open a new browser window. Control+Tab - switch to the next tab (prefer switch_tab). Control+1 ... Control+8 - switch to the tab at that position (prefer switch_tab). Control+- / Control+0 - zoom the page out / reset the zoom. F11 - toggle full-screen view. Control+h - open the browser history page. Control+j - open the browser downloads page. Control+d - bookmark the current page. Control+p - open the print dialog. Control+u - view the page source. F12 / Control+Shift+i / Control+Shift+j / Control+Shift+c - open the browser developer tools (console / element picker). Alt / Meta / Meta+r / Control+Escape / Alt+Tab / Control+Alt+t - operating-system or browser-menu keys (open the menu bar or start menu, run dialog, switch windows, open a terminal); they usually do not help with a web task. j / k / w - single-letter shortcuts that some websites define (for example next / previous item). Do not use hotkey to type text; use type for that. Clicking an input first gives it focus.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "keys": { | |
| "type": "array", | |
| "items": { | |
| "type": "string" | |
| }, | |
| "minItems": 1, | |
| "description": "Keys pressed together, modifiers first. Examples: [\"Enter\"], [\"Control\", \"a\"], [\"Escape\"], [\"ArrowDown\"], [\"Control\", \"r\"]." | |
| } | |
| }, | |
| "required": [ | |
| "keys" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "scroll", | |
| "description": "Scroll by signed pixel deltas.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "scroll_dy": { | |
| "type": "number", | |
| "description": "Signed pixel delta, not a normalized coordinate." | |
| }, | |
| "scroll_dx": { | |
| "type": "number", | |
| "description": "Signed pixel delta, not a normalized coordinate." | |
| } | |
| }, | |
| "required": [ | |
| "scroll_dy" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "wait", | |
| "description": "Wait for the specified duration.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "seconds": { | |
| "type": "number", | |
| "minimum": 0 | |
| } | |
| }, | |
| "required": [ | |
| "seconds" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "finished", | |
| "description": "Finish the task, optionally returning an answer.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "answer": { | |
| "type": "string" | |
| } | |
| }, | |
| "required": [], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "fail", | |
| "description": "Terminate because the task cannot be completed.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": {}, | |
| "required": [], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "unknown", | |
| "description": "Signal that a reliable action cannot be determined.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": {}, | |
| "required": [], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "go_back", | |
| "description": "Go back in browser history.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": {}, | |
| "required": [], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "go_forward", | |
| "description": "Go forward in browser history.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": {}, | |
| "required": [], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "goto", | |
| "description": "Navigate to a URL.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "url": { | |
| "type": "string" | |
| } | |
| }, | |
| "required": [ | |
| "url" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "hover", | |
| "description": "Move the pointer without clicking.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "x": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| }, | |
| "y": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| } | |
| }, | |
| "required": [ | |
| "x", | |
| "y" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "drag", | |
| "description": "Drag from (x,y) to (x2,y2).", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "x": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| }, | |
| "y": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| }, | |
| "x2": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| }, | |
| "y2": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| } | |
| }, | |
| "required": [ | |
| "x", | |
| "y", | |
| "x2", | |
| "y2" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "upload", | |
| "description": "Upload a local file to the web page. Use it right after the page has opened a file picker (e.g. after clicking a button such as 'Upload', 'Choose file', or '컴퓨터에서 선택'); the file is attached to that picker. Use only a file name or path that the user gave in the request — never invent one. If no file picker is open yet, click the page's upload button first.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "path": { | |
| "type": "string", | |
| "description": "File name or path exactly as given by the user, e.g. \"puppy.jpg\"." | |
| } | |
| }, | |
| "required": [ | |
| "path" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "focus", | |
| "description": "Focus the element at the coordinates.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "x": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| }, | |
| "y": { | |
| "type": "number", | |
| "minimum": 0, | |
| "maximum": 1000, | |
| "description": "Normalized 0-1000 screenshot coordinate." | |
| } | |
| }, | |
| "required": [ | |
| "x", | |
| "y" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "clear", | |
| "description": "Clear all text in the currently focused input. Give the input focus first (click or focus on it), then clear, then type.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": {}, | |
| "required": [], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "EXIT", | |
| "description": "Explicit final termination.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": {}, | |
| "required": [], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "new_tab", | |
| "description": "Open a new blank browser tab.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": {}, | |
| "required": [], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "switch_tab", | |
| "description": "Switch to the browser tab at the given 0-based index (see Available tabs in the observation).", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "index": { | |
| "type": "integer", | |
| "minimum": 0, | |
| "description": "0-based tab index." | |
| } | |
| }, | |
| "required": [ | |
| "index" | |
| ], | |
| "additionalProperties": false | |
| } | |
| } | |
| }, | |
| { | |
| "type": "function", | |
| "function": { | |
| "name": "close_tab", | |
| "description": "Close the current browser tab.", | |
| "parameters": { | |
| "type": "object", | |
| "properties": {}, | |
| "required": [], | |
| "additionalProperties": false | |
| } | |
| } | |
| } | |
| ] |