Instructions to use Xylolabs/KORMo-Web-11B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Xylolabs/KORMo-Web-11B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
Use Docker
docker model run hf.co/Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use Xylolabs/KORMo-Web-11B-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Xylolabs/KORMo-Web-11B-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Xylolabs/KORMo-Web-11B-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
- Ollama
How to use Xylolabs/KORMo-Web-11B-GGUF with Ollama:
ollama run hf.co/Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use Xylolabs/KORMo-Web-11B-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Xylolabs/KORMo-Web-11B-GGUF with Docker Model Runner:
docker model run hf.co/Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
- Lemonade
How to use Xylolabs/KORMo-Web-11B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.KORMo-Web-11B-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use Xylolabs/KORMo-Web-11B-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Xylolabs/KORMo-Web-11B-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "Xylolabs/KORMo-Web-11B-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
GGUF quants of KORMo-VL/KORMo-Web-11B for llama.cpp on CUDA, Metal or CPU; stock llama.cpp runs the text model, and its vision projector needs llava-onevision.patch from this repository.
Files
Each quant is one file with the text model, its tokenizer and its chat template.
| File | Size |
|---|---|
KORMo-Web-11B-Q8_0.gguf |
11.44 GB |
KORMo-Web-11B-Q6_K.gguf |
8.83 GB |
KORMo-Web-11B-Q5_K_M.gguf |
7.66 GB |
KORMo-Web-11B-Q4_K_M.gguf |
6.56 GB |
KORMo-Web-11B-Q4_K_S.gguf |
6.23 GB |
KORMo-Web-11B-IQ4_XS.gguf |
5.90 GB |
KORMo-Web-11B-Q3_K_M.gguf |
5.31 GB |
KORMo-Web-11B-IQ3_M.gguf |
4.98 GB |
mmproj-KORMo-Web-11B-F16.gguf (vision projector, F16) |
871.52 MB |
mmproj-KORMo-Web-11B-Q8_0.gguf (vision projector, Q8_0) |
592.85 MB |
KORMo-Web-11B.imatrix.gguf (importance matrix) |
13.23 MB |
llava-onevision.patch (llama.cpp patch) |
19.69 kB |
LICENSE |
11.36 kB |
NOTICE |
309 bytes |
How to run
The patch applies to llama.cpp commit e4b620af19f02960dea7a3a81de7c5805aa3a333.
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git checkout e4b620af19f02960dea7a3a81de7c5805aa3a333
curl -LO https://huggingface.co/Xylolabs/KORMo-Web-11B-GGUF/resolve/main/llava-onevision.patch
git apply llava-onevision.patch
cmake -B build -DCMAKE_BUILD_TYPE=Release # add -DGGML_CUDA=ON for NVIDIA GPUs
cmake --build build -j --target llama-server llama-mtmd-cli
./build/bin/llama-server -m KORMo-Web-11B-Q4_K_M.gguf --mmproj mmproj-KORMo-Web-11B-F16.gguf --jinja -ngl 99 -c 32768
./build/bin/llama-mtmd-cli -m KORMo-Web-11B-Q4_K_M.gguf --mmproj mmproj-KORMo-Web-11B-F16.gguf --jinja -ngl 99 --image page.png -p "Describe this page."
The model thinks by default. Add --chat-template-kwargs '{"enable_thinking":false}' to llama-server to answer directly.
Accuracy
| Quant | PPL | Mean KL vs BF16 | Same top-1 vs BF16 | Answer check |
|---|---|---|---|---|
| Q8_0 | 11.7514 | 0.0014 | 98.30% | 7/7 |
| Q6_K | 11.7590 | 0.0028 | 97.58% | 7/7 |
| Q5_K_M | 11.7727 | 0.0070 | 95.99% | 7/7 |
| Q4_K_M | 11.8013 | 0.0169 | 94.06% | 7/7 |
| Q4_K_S | 11.8435 | 0.0223 | 93.11% | 7/7 |
| IQ4_XS | 11.8753 | 0.0279 | 92.33% | 7/7 |
| Q3_K_M | 12.0835 | 0.0513 | 89.69% | 7/7 |
| IQ3_M | 12.0083 | 0.0928 | 86.69% | 7/7 |
KL, top-1 and PPL are against our BF16 GGUF (PPL 11.7595) on wikitext-2 test, the last 255 tokens of each of 80 chunks of 512; the answer check is seven short English and Korean questions answered greedily with thinking off, scored against BF16, which answers all seven.
License and notices
The model is under the Apache License 2.0, copied in LICENSE; we converted its weights to GGUF and quantized them, NOTICE names the source model and its copyright holder, llava-onevision.patch is offered under llama.cpp's MIT license, and Xylolabs is not affiliated with or endorsed by KAIST MLP Lab.
KAIST MLP Lab's original model description
This is KAIST MLP Lab's card for the original model, copied unchanged from KORMo-VL/KORMo-Web-11B (Apache-2.0) except that its tools.json and HARNESS.md links now point to that repository.
KORMo-Web-11B
One Model. Three Capabilities: LLM, VLM, and Web Agent.
KORMo-Web-11B๋ KAIST MLP ์ฐ๊ตฌ์ค์ด ์ฒ์๋ถํฐ ๊ตฌ์ถํ KORMo ํ์ด๋ฐ์ด์ ๋ชจ๋ธ์ ๊ธฐ๋ฐ์ผ๋ก, LLMยทVLMยทGUI Agent ๋ฅ๋ ฅ์ ๋ชจ๋ ๊ฐ์ถ ๋ฒ์ฉ ๋ฉํฐ๋ชจ๋ฌ AI ๋ชจ๋ธ์ ๋๋ค.
- ํ๋์ ๋ชจ๋ธ, ์ธ ๊ฐ์ง ๋ฅ๋ ฅ: ์ผ๋ฐ์ ์ธ ์ง๋ฌธ์๋ LLM์ผ๋ก ๋ต๋ณํ๊ณ , ์ด๋ฏธ์ง ์ดํด๊ฐ ํ์ํ ๊ฒฝ์ฐ VLM์ผ๋ก, ์น ์กฐ์์ด ํ์ํ ์์ฒญ์๋ GUI Agent๋ก ์๋ํฉ๋๋ค.
- ๋ ์ ํ์ด๋ฐ์ด์ ๋ชจ๋ธ ๊ธฐ๋ฐ: From-scratch๋ก ํ์ตํ KORMo-10B๋ฅผ KORMo-VL๋ก ํ์ฅํ๊ณ , ๊ธฐ์กด ์ธ์ดยท์๊ฐ ์ดํด ๋ฅ๋ ฅ์ ์ ์งํ๋๋ก ํ์ตํ ์น ์์ด์ ํธ๋ก ์๊ฐ<Think> ๋ํ ์ ์ผํ๊ฒ ํ๊ตญ์ด๋ก ํ๋ ๋ชจ๋ธ์ ๋๋ค.
- ๋๊ท๋ชจ ์น ์ฌ์ ํ์ต: 1,000๋ง ๊ฑด ์ด์์ ํ์ ์น ํ์ต ์ํ๋ก UI ์์์ ์์นยท์๋ฏธยท๊ตฌ์กฐยท๊ธฐ๋ฅ ๋ฑ์ ์ธ๋ฐํ๊ฒ ํ์ตํ์ต๋๋ค.
- ์ค์ ์น ํ๋ ํ์ต: 10๋ง ๊ฑด ์ด์์ ํ์ ์น ์ฌ์ฉ ๊ถค์ ์ ํ์ฉํด ๋ณต์กํ ๋ค๋จ๊ณ ์น ์์ ์ ์ํํฉ๋๋ค.
- ์์ ๊ณต๊ฐ ๋ฐ ์์ ์ ํ์ฉ: ๋ชจ๋ธ ๊ฐ์ค์น์ ํ์ตยทํ๊ฐ ๋ฆฌ์์ค๋ฅผ ๊ณต๊ฐํ๋ฉฐ, Apache-2.0 ๋ผ์ด์ ์ค์ ๋ฐ๋ผ ์์ ์ ์ด์ฉ๊ณผ ์์ ยท์ฌ๋ฐฐํฌ๊ฐ ๊ฐ๋ฅํฉ๋๋ค.
KORMo-Web-11B is a general-purpose bilingual AI model built on the KORMo foundation model family, developed by KAIST MLP Lab starting from a language model trained entirely from scratch. Unlike our Qwen-based KORMo-Web-4B and 9B models, which focus on maximizing web task performance, KORMo-Web-11B is designed to preserve its original LLM and VLM capabilities while learning to act as a GUI agent. It can understand Korean and English text, interpret images, and complete real browser tasks by visually navigating webpagesโall within a single model.
What Makes KORMo-Web-11B Different?
- Built on Our Own Foundation Model: Developed from the KORMo-10B language model, trained from scratch by KAIST MLP Lab, and extended through KORMo-VL into a web agent.
- Three Capabilities in One Model: Designed to retain general LLM reasoning and VLM understanding while adding GUI interaction capabilities, rather than optimizing exclusively for web navigation.
- 10M+ Atomic-Level Web Pretraining Samples: Large-scale Korean and English web understanding training, covering individual UI elements, their locations, layouts, functions, and states.
- 100K+ Web Interaction Trajectories: Bilingual browser demonstrations for learning real-world, multi-step web tasks.
- Open and Commercially Usable: Model weights and supporting resources are openly released under Apache-2.0, allowing commercial use, modification, and redistribution subject to the license terms.
From a Korean LLM to a General-Purpose Web Agent
KORMo-Web-11B extends our independently developed KoreanโEnglish foundation model through three stages:
| Stage | Model | Capabilities |
|---|---|---|
| 1. Language | KORMo-10B | KoreanโEnglish LLM trained from scratch with a Korean-optimized tokenizer |
| 2. Vision | KORMo-VL | Multimodal understanding through a SigLIP vision encoder and KORMo language backbone |
| 3. Web Agent | KORMo-Web-11B | Large-scale web pretraining and action learning for visual browser interaction |
The goal is not just a better web agent, but a versatile model that can understand language, interpret visual information, and interact with the web.
Performance
We evaluated KORMo-Web-11B on 612 live-website tasks (300 English, 312 Korean) from Online-Mind2Web, using the same LLM judge across all evaluated models.
| Model | Size | Online-Mind2Web EN | Online-Mind2Web KO |
|---|---|---|---|
| Qwen3.5-9B (base) | 9B | 24.7 | 16.7 |
| Fara-1.5-4B | 4B | 20.7 | 7.7 |
| UI-TARS-1.5-7B | 7B | 22.3 | 20.2 |
| OpenWebRL-4B | 4B | 32.7 | 15.4 |
| KORMo-Web-11B (ours) | 11B | 31.3 | 33.7 |
| GPT-6-luna (proprietary) | โ | 63.0 | 65.7 |
Success rates are calculated over the full task set, including failed or incomplete runs. Each agent uses its own prompt and action format.
KORMo-Web-11B achieves 33.7% Korean task success, exceeding the strongest non-KORMo open-weight baseline listed above by 13.5 percentage points.
While our specialized KORMo-Web-4B and 9B models achieve higher web task success, KORMo-Web-11B prioritizes versatility across language, vision, and GUI interaction, rather than web-only optimization.
Why KORMo-Web-11B?
- Korean from the Ground Up: Built upon our independently pretrained KoreanโEnglish foundation model.
- General-Purpose Multimodal AI: Supports text-based language tasks, visual understanding, and browser interactions within one model.
- Visual-First Web Interaction: Operates primarily through screenshots without requiring full DOM or accessibility trees.
- Think, Then Act: Generates an inspectable reasoning step before executing browser actions.
- Commercial Use Permitted: Released under Apache-2.0, supporting academic research, commercial applications, fine-tuning, and redistribution.
- Open and Ready to Run: Includes model weights, Transformers 5 support, a vLLM inference path, browser harness, and reference inference settings.
KORMo-Web-11B โ From a Korean Foundation Model to a General-Purpose Web Agent.
How to Start: Chrome Extension
The fastest way to try KORMo-Web is the KORMo-Web Chrome extension: type a request in Chrome's side panel and watch the agent work in your own browser. This video walks through the whole setup, from serving the model to running a task and controlling it from Discord.
Setup guide (about 15 min, Korean narration). Parts that showed account tokens have been blurred.
1. Serve the model on a Mac or on an NVIDIA GPU.
Mac (Apple silicon)
KORMo-Web-11B is served with vLLM on an NVIDIA GPU (see below); a Mac setup for 11B has not been tested. To try KORMo-Web on a Mac, run KORMo-Web-4B or KORMo-Web-9B with vllm-mlx 0.6.0 or later. Pick the one that fits your Mac's memory:
| Model | Weights | Memory in use | Recommended Mac memory |
|---|---|---|---|
KORMo-Web-4B-mlx-4bit (4-bit) |
2.8 GB | about 3.8 GB | 16 GB or more |
KORMo-Web-4B (bf16) |
9.6 GB | about 9.5 GB | 24 GB or more |
KORMo-Web-9B (bf16) |
17.5 GB | about 19 GB | 48 GB or more (32 GB is tight) |
uv venv --python 3.12 && source .venv/bin/activate
uv pip install "vllm-mlx>=0.6.0" huggingface_hub
MODEL=KORMo-Web-4B-mlx-4bit # or KORMo-Web-4B / KORMo-Web-9B
hf download KORMo-VL/$MODEL --local-dir ~/models/$MODEL
vllm-mlx serve ~/models/$MODEL --port 8010 --served-model-name KORMo-VL/$MODEL \
--enable-auto-tool-choice --tool-call-parser auto
Keep this terminal open. --served-model-name is the name the extension must request.
NVIDIA GPU (local PC or server)
vLLM needs a one-time conversion for the KORMo architecture: follow steps 1 and 2 of Run the agent (vLLM) in the Quickstart below, which serve the model as KORMo-Web-11B on port 8000.
Check that it is up with curl http://localhost:8010/v1/models (Mac) or curl http://<server>:8000/v1/models (vLLM). If you changed the port, use that port.
2. Install the extension. git clone https://github.com/MLP-Lab/KORMo-Web-Extension.git, open chrome://extensions, turn on Developer mode, click Load unpacked and choose the cloned folder. Open the GUI ์์ด์ ํธ side panel, type a task and press ์คํ (Run).
3. Point the extension at your model. Open the panel's ๊ฐ๋ฐ์ ์ค์ (Developer settings).
- Mac: keep the server URL
http://localhost:8010/v1andtool_choiceauto. Set the model name to your--served-model-name(the defaultKORMo-VL/KORMo-Web-4B-mlx-4bitalready matches the 4-bit build). - NVIDIA GPU: set the server URL to
http://<server>:8000/v1(orlocalhost), the model name toKORMo-Web-11B, andtool_choicetonone.
4. Discord (optional). Run node relay/discord-relay.mjs in the cloned folder and follow the prompts to send tasks to the agent by mentioning a bot in a channel.
Full step-by-step instructions (Korean) are in the extension README.
Quickstart
Tested with transformers 5.12.1 and vLLM 0.23.0. The model runs on a single GPU.
transformers does not ship the KORMo text model, so this repo includes it (_modeling_kormo.py) together with a small loader, kormo_load.py:
kormo_load.register(path)registers the KORMo text model. Call it before loading the processor or the model.kormo_load.patch_vision(model)feeds the SigLIP features after post-layernorm to the projector, as in training. Without it the model still runs but misreads screenshots.
Inference example (one step, transformers)
pip install -U "transformers>=5.12" torch pillow huggingface_hub
hf download KORMo-VL/KORMo-Web-11B --local-dir KORMo-Web-11B
import json, sys, torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor
M = "KORMo-Web-11B"
sys.path.insert(0, M)
import kormo_load
kormo_load.register(M)
processor = AutoProcessor.from_pretrained(M)
model = AutoModelForImageTextToText.from_pretrained(M, dtype=torch.bfloat16).to("cuda").eval()
kormo_load.patch_vision(model)
SYSTEM = open(f"{M}/system_prompt.md").read().strip()
TOOLS = json.load(open(f"{M}/tools.json"))
obs = ("<observation>\nscreen size: 1280 x 720\nCurrent tab: 0\nAvailable tabs:\n"
"- Tab 0 (active): https://www.aladin.co.kr/\n\nurl: https://www.aladin.co.kr/\nscreenshot:\n")
messages = [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": [
{"type": "text", "text": "<user_request>\n์๋ผ๋ ์ฌ์ดํธ์์ 'ํด์ปค์ค ํ ์ต RC'๋ฅผ ๊ฒ์ํ ๋ค ๊ฐ์ฅ ๋จผ์ ํ์๋๋ ๋์์ ์ถ๊ฐ์ผ์ ์๋ ค์ค.\n</user_request>\n" + obs},
{"type": "image", "image": Image.open("screen.png").convert("RGB")},
{"type": "text", "text": "\n</observation>"},
]},
]
inputs = processor.apply_chat_template(messages, tools=TOOLS, add_generation_prompt=True,
tokenize=True, return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=16384, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))
A real first step from our Online-Mind2Web run (Aladin home page):
<think>I see the Aladin homepage, with the search field labeled โํตํฉ๊ฒ์โ near the top. I have not yet searched for the book, so I still need to enter โํด์ปค์ค ํ ์ต RC,โ inspect the first resultโs publication date, and report that date. My plan:
1. Click the search field
2. Enter โํด์ปค์ค ํ ์ต RCโ and submit the search
3. Read the publication date of the first listed book
This click targets the visible search field and is a safe, relevant first action.
</think>
<tool_call>
<function=click>
<parameter=x>
482
</parameter>
<parameter=y>
212
</parameter>
</function>
</tool_call>
Coordinates are normalized to 0โ1000. Convert them to pixels with x / 1000 ร width and y / 1000 ร height.
Run the agent (vLLM)
vLLM does not include the KORMo architecture either. The KORMo text layers have the Llama structure, so vllm/convert_to_vllm.py renames two parameter names and writes a folder vLLM loads natively. vllm/sitecustomize.py turns SigLIP post-layernorm back on inside vLLM.
1. Convert (once)
pip install -U "vllm>=0.23.0" openai playwright safetensors
playwright install chromium
python KORMo-Web-11B/vllm/convert_to_vllm.py KORMo-Web-11B KORMo-Web-11B-vllm
2. Start the model server
PYTHONPATH=$PWD/KORMo-Web-11B/vllm vllm serve ./KORMo-Web-11B-vllm --served-model-name KORMo-Web-11B \
--max-model-len 65536 --limit-mm-per-prompt '{"image":1,"video":0}'
Check that the server log prints [kormo-vllm] SigLIP post_layernorm enabled.
3. Run a browser task
python KORMo-Web-11B/examples/agent_playwright.py --model KORMo-Web-11B \
--start-url https://www.danawa.com/ \
--task "๋ค๋์์์ 50๋ง ์ ์ดํ, 10์ธ์น ์ด์ ํ๋ธ๋ฆฟ 3๊ฐ๋ฅผ ์ฐพ์์ ๊ฐ๊ฒฉ์ ์๋ ค์ค"
The agent drives a headless Chromium, prints each action as it goes, and finishes with the model's answer. Add --headful to watch the browser window. The script sends tool_choice="none", which vLLM needs: the tool schema stays in the prompt and the call comes back as text (sending tools without tool_choice returns HTTP 400 unless the server was started with --enable-auto-tool-choice).
How It Works
user <user_request> task </user_request> + observation (tabs, url, screenshot)
assistant <think> what I see ยท what is done ยท plan </think> <tool_call> one action </tool_call>
tool action result + new observation (newest screenshot only)
โฆ repeat until finished(answer)
- Action space: 22 browser tools: click, type, hotkey, scroll, drag, goto, tab control,
finished, and others (tools.json). - Memory: the full text history of thoughts, actions and observations is kept. Only the current screenshot is sent.
- Recommended settings: temperature 0.6, top_p 0.95, top_k 20, thinking on, up to 16k tokens per step and 50 steps. These are already set in
generation_config.json.
The exact message format, parsing rules and feedback strings are in HARNESS.md. Follow it if you build your own harness.
Evaluation Details
| Tasks | Online-Mind2Web: 300 English, 312 Korean |
| Judge | GPT-6 |
Files
| File | Description |
|---|---|
model.safetensors ยท config.json |
Weights and config (LLaVA-OneVision: SigLIP + KORMo-10B text model) |
_configuration_kormo.py ยท _modeling_kormo.py |
KORMo text model code for transformers 5 |
kormo_load.py |
register() and patch_vision() |
chat_template.jinja |
Chat template (XML tool calls, thinking on) |
system_prompt.md ยท tools.json |
System prompt and 22-tool schema |
HARNESS.md |
Full harness specification |
examples/agent_playwright.py |
Reference browser agent (Playwright + OpenAI-compatible API). --tool-choice none for vLLM (default); --headful shows the browser. It does not execute upload. |
vllm/convert_to_vllm.py ยท vllm/sitecustomize.py |
Convert for vLLM and the post-layernorm fix |
generation_config.json ยท *processor_config.json ยท tokenizer*.json ยท special_tokens_map.json |
Sampling, processor and tokenizer configs |
Acknowledgements
Built on KORMo-VL and KORMo-10B by KAIST MLP Lab. Evaluated on Online-Mind2Web and its Korean counterpart.
Contact
- KyungTae Lim, Professor at KAIST.
ktlim@kaist.ac.kr
Contributor
- Hangyeol Yoo (hg.yoo@kaist.ac.kr)
- Minkyu Kim (minkyukim@kaist.ac.kr)
- Minkyung Cho (minkyung.cho@kaist.ac.kr)
- Yerin Nam (yerinnam@kaist.ac.kr)
- Seungwoo Song (sswoo@kaist.ac.kr)
- Gyun Lee (gyunini@kaist.ac.kr)
- KyungTae Lim (ktlim@kaist.ac.kr)
- Downloads last month
- 164
