--- license: apache-2.0 library_name: transformers base_model: Qwen/Qwen3.8-27B base_model_relation: finetune pipeline_tag: text-classification tags: - decision-model - system-one - web-agent - browser-agent model-index: - name: karpo-decision-27B-act results: - task: type: text-classification name: web action decision (one choice per step among the page's actions and 7 endings) dataset: type: kbb name: Karpo BrowserBench (KBB) 1.0, private split (1,794 scored instructions) metrics: - type: completion_rate value: 35.5 name: "Instruction completion rate, every step right (self-measured; not on the KBB board)" - type: step_accuracy value: 78.5 name: "Step accuracy (self-measured; not on the KBB board)" source: name: Self-measured (not on the KBB board) url: https://karpo.ai/benchmarks/browserbench - task: type: text-classification name: web action decision (one choice per step among the page's actions and 7 endings) dataset: type: kbb name: Karpo BrowserBench (KBB) 1.0, all 2,748 scored instructions (private and public splits) metrics: - type: completion_rate value: 35.3 name: "Instruction completion rate, every step right (self-measured; not on the KBB board)" - type: step_accuracy value: 78.3 name: "Step accuracy (self-measured; not on the KBB board)" source: name: Self-measured (not on the KBB board) url: https://karpo.ai/benchmarks/browserbench --- # Karpo Decision 27B Act

Benchmark: KBB (Karpo BrowserBench) License: Apache 2.0

**The System 1 model of karpo-browser**, an open-source browser agent from Karpo's production work, released together with this model. With DeepSeek V4.1 Flash as System 2, karpo-browser gets **84%** of the 38 core tasks of [Browserbase Benchmark v2](https://stagehand.dev/evals) right (mean of 8 runs), in **218 s** and for **$0.096** in model cost per task. On the board of 19 November 2026, none of the 91 records gets 80% or more within 250 s and $0.15 per task. These are our own runs, not board entries: our time is the agent's own run time, and each board record is one run. See [how we measured](#browserbase-benchmark-v2-karpo-browser-end-to-end). A web action decision model: at every step on a page it reads the numbered list of the page's elements, a screenshot, a summary of the goal so far and the page changes caused by the last few actions, and picks the most probable of the listed options, an action or one of seven endings. One forward pass, about one second, no sampling, no generated text. Fine-tuned for web actions from Karpo Decision 27B, Karpo's general decision model (not released separately), which was post-trained from [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B), a multimodal model with its own vision encoder. It works like [Karpo Decision 31B](https://huggingface.co/MachinePulseAI/karpo-decision-31B): one forward pass, probabilities read directly over the given options, the same request interface and the same kind of readout (a linear readout over option codes). - **Parameters:** 26.1B in bf16, plus a 255 × 5,120 readout in fp32 - **Input:** text and a screenshot: the goal, the values it may fill, the actions so far and their effects, the numbered page list. Trained with and without screenshots, so it also works when no screenshot is sent - **Questions:** one choice per step (element actions, page actions, 7 endings); a follow-up choice for `FILL` and `SELECT`; a yes/no question on whether it is going in circles - **Request size in karpo-browser:** at most about 32,000 tokens and 255 options per question; median input 7.3k to 7.7k tokens, screenshot included - **Context:** 32,768 tokens per sequence (the inference code's default); a longer request is split by questions, each part with the full state, and a question that does not fit with its state is refused, never truncated - **Latency:** median 931 ms per decision on KBB (one request at a time, no inference acceleration) - **Licence:** [Apache-2.0](LICENSE) ## Results ### Browserbase Benchmark v2: karpo-browser end to end [Browserbase Benchmark v2](https://stagehand.dev/evals) measures whether a whole task gets done: a one-sentence task on a live website, an answer, and a grader that checks it against a rubric written in advance. karpo-browser with DeepSeek V4.1 Flash as the System 2 model and Karpo Decision 27B Act as the System 1 model: | Task set | Accuracy | Time per task | Model cost per task | Runs | |---|---|---|---|---| | 38 core tasks (the board's set) | **84%** | **218 s** | **$0.096** | mean of 8 | | All 102 tasks | **86%** | **201 s** | **$0.096** | mean of 3 (final configuration) | All 8 runs covered the 102 tasks: 84 to 93 passed, 88.8 on average; of the 38 core tasks, 29 to 35 passed (76% to 92%), 31.8 on average. The 3 runs in the second row are the final configuration, the group with the shortest mean time; the most accurate group averaged 89% over 2 runs. Each record on the board is one run of the 38 core tasks. Among its 91 records (19 November 2026), the highest accuracy is 92% (claude-fable-5-2 + claude code, 378 s, $1.242), the cheapest at 80% or more is gpt-6.2-sol + codex (87%, 267 s, $0.386), and the fastest at 85% or more takes 223 s. In the 3 runs of the final configuration, System 1 did about 78% of the page actions (opening URLs excluded) in 14% of the time per task (27.5 s of 201 s) and for about 13% of the model cost. Over their 8,089 decision calls, the server logged zero output tokens. How we measured, and how it differs from the board: - **Tasks:** the official set has 102 tasks, 38 of them core. 99 tasks ran on 19 November 2026. The 3 tasks that start on petsmart.com and then look up the same product's price on amazon.com (1 of them core) were re-run on 20 November, each run keeping its own settings; so 37 of the 38 core tasks ran on 19 November. - **Grading:** Stagehand's open-source eval program (version of 10 November 2026, plus a patch that plugs in karpo-browser), each task's rubric (including the v1.2 notes), judge model gemini-3.5-flash (the program's default). We have not verified that the board's records used the same grader version and judge model. The 3 runs of the final configuration also passed the grader the start pages System 2 read and a screenshot at the end of each browsing segment; the other 5 runs differ slightly in wait settings and in the screenshots passed to the grader. - **Timing:** the agent's own run time only, grading excluded; 450 s limit per task. We found no statement of the board's timing boundary. - **Cost:** the System 2 model at OpenRouter's public price on the day, times the actual tokens, plus the System 1 model at $0.06 per million input tokens. Karpo Decision 27B Act has no public API; that price is the OpenRouter input price of a Qwen model of the same size, for comparison only. At actual GPU time it is $0.11 to $0.14 per task. Browserbase browser cost (about $0.33 to $0.38 per task) and grading cost are not included. The board prices each model at its public price. - **The board itself:** it is run by Browserbase, the developer of Stagehand, which also wrote the grader, and each record is a single run of the 38 core tasks. - **Our setup:** our DeepSeek V4.1 Flash is the 17 October version with image input on, and karpo-browser's harness was tuned on this task set, so this is not a controlled comparison with the board's deepseek-v4.1-flash records (other harnesses, 12 September). ### KBB (Karpo BrowserBench), as the System 1 model of karpo-browser [KBB](https://karpo.ai/benchmarks/browserbench) measures web action-decision ability: runs recorded on real websites are replayed page by page, each step giving the model the same kind of request the System 1 model gets in karpo-browser, and an instruction counts as completed only if every step is right. **Self-measured: Karpo Decision 27B Act is not on the KBB board.** Private split, 1,794 scored instructions; board numbers from the KBB page of 17 November 2026. | Model | Completion | |---|---| | The four general LLMs on the board | 38.1% to 41.0% | | **Karpo Decision 27B Act** | **35.5%** | | The decision model with the highest completion on the board | 28.8% | Karpo Decision 27B Act has 78.5% step accuracy, costs $0.0039 per instruction (by KBB's common cost method) and takes 931 ms per decision at the median (one request at a time). The board's decision models read only the page text, while Karpo Decision 27B Act, like the general LLMs, also reads the screenshot; part of its lead over those decision models comes from that. ### General decisions Specialisation has a cost: on a sample of general decision questions, Karpo Decision 27B Act scores 1.7 points lower than its starting model. For general decision tasks, use [Karpo Decision 31B](https://huggingface.co/MachinePulseAI/karpo-decision-31B). ## The Karpo Decision family | Model | Base | Parameters | For | |---|---|---|---| | **karpo-decision-27B-act** | Qwen3.8 27B, via Karpo Decision 27B (not released separately) | 26.1B | Web actions, System 1 model of karpo-browser | | [karpo-decision-31B](https://huggingface.co/MachinePulseAI/karpo-decision-31B) | Gemma 4 31B | 31.3B | General decisions | | [karpo-decision-12B](https://huggingface.co/MachinePulseAI/karpo-decision-12B) | Gemma 4 12B | 12.0B | General decisions | | [karpo-decision-9B](https://huggingface.co/MachinePulseAI/karpo-decision-9B) | Qwen3.5 9B | 8.4B | General decisions | | [karpo-decision-4B](https://huggingface.co/MachinePulseAI/karpo-decision-4B) | Qwen3.5 4B | 4.5B | General decisions | | [karpo-decision-2B](https://huggingface.co/MachinePulseAI/karpo-decision-2B) | Qwen3.5 2B | 2.2B | General decisions | | [karpo-decision-0.8B](https://huggingface.co/MachinePulseAI/karpo-decision-0.8B) | Qwen3.5 0.8B | 0.85B | General decisions | ## How it works ### System 1 decides, System 2 plans In browser agents such as [browser-use](https://github.com/browser-use/browser-use) and [Stagehand](https://github.com/browserbase/stagehand), a general LLM decides every step on a page, so cost and time grow with the number of steps. For scale: in our Browserbase Benchmark v2 runs, one System 2 turn (DeepSeek V4.1 Flash) took about 3.4 s, read about 19.5k input tokens and wrote 476 output tokens. karpo-browser splits the work between two models, named after the psychology terms "fast thinking" (System 1) and "slow thinking" (System 2): this model is the System 1 model, and a general LLM is the System 2 model. | Work | Who | |---|---| | Break the task into goals | System 2 | | Write the text to type (search terms, form values) | System 2 | | Open URLs | System 2 | | Pick the element and the action at every step | System 1 | | Pick which input gets which value | System 1, among the values System 2 gave | | Judge whether the goal is done, or why it cannot go on | System 1 judges, System 2 checks | | Read lists, compare, summarise, write the final answer | System 2 | | Remember information across goals | System 2 | System 1's options include no "open URL", no free text and no click at coordinates; it does not read long lists or compare, and the text it types can only come from values System 2 gave. This is deliberate: the more fixed the options, the faster the model and the fewer unexpected actions; the price is that someone has to give it goals and values. ### Action space: every step is one choice question Options are generated from the numbered elements of the page list and written as "action number", for example `CLICK 5`. Actions on one element: | Action | Effect | Offered on | |---|---|---| | `CLICK n` | Click | Clickable elements, inputs included | | `FILL n` | Fill an input with a value | Editable inputs, not read-only ones | | `PRESS_ENTER n` | Press Enter in an input | Inputs | | `SELECT n` | Choose an item of a dropdown | Native `