| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3.5-2B |
| base_model_relation: finetune |
| library_name: transformers |
| tags: |
| - wiss-ai |
| - qwen3.5 |
| - tool-use |
| - function-calling |
| - reasoning |
| - coding |
| - vision |
| - lora |
| - unsloth |
| --- |
| |
| # Wiss 2B |
|
|
| **Wiss 2B** is a small (2B) assistant fine-tuned from [Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) for |
| **coding, multi-step tool use and short, finished reasoning**. It keeps the base model's image understanding |
| (the image encoder and projector are unchanged). |
|
|
| It is a narrow fine-tune, not a general upgrade: on standard benchmarks it is **behind the base model on instruction |
| following, hard math and code**, and ahead on BBH-style logic puzzles. The tables below show every number we measured. |
|
|
| > **A note from the team:** we were stumped by these results. We aimed for a model that was better at coding and tool |
| > use without giving up general ability, and on standard benchmarks it fell short of the base model in several places. |
| > We are releasing it anyway, openly and with all the numbers, because it is still useful for what it was built for. |
| > Our next model, Wiss2, will try our best to fix this: more general-purpose training data, replay of earlier data to stop |
| > forgetting, and release checks measured against the base model. |
| > Due to some technical issues, we are unable to release the model right now. |
|
|
| ## Benchmarks |
|
|
| All models were run by us with the same settings: [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) |
| on vLLM, each model's own chat template, **thinking mode off**, greedy decoding, bf16. IFEval through MMLU-Pro are the |
| [Open LLM Leaderboard v2](https://huggingface.co/docs/leaderboards/open_llm_leaderboard/about) tasks (raw scores, not the |
| leaderboard's normalized ones). GSM8K is 5-shot strict match; HumanEval is pass@1 (instruct prompt). IFEval, MATH-hard, |
| GSM8K and HumanEval use the full test sets. BBH, MuSR and MMLU-Pro use only the first 60 questions of every subtask / |
| subject (the same questions for both models), so differences of a few points on those are noise. |
|
|
| | Benchmark | Wiss 2B | Qwen3.5-2B (base) | Difference | |
| |---|---|---|---| |
| | IFEval, prompt-level strict | 49.5 | **67.3** | −17.8 | |
| | IFEval, instruction-level strict | 60.4 | **76.0** | −15.6 | |
| | IFEval, prompt-level loose | 54.0 | **71.2** | −17.2 | |
| | IFEval, instruction-level loose | 64.4 | **79.4** | −15.0 | |
| | MATH-hard (exact match) | 11.9 | **31.2** | −19.3 | |
| | GSM8K | 61.8 | **63.8** | −2.0 | |
| | HumanEval (pass@1) | 42.7 | **50.0** | −7.3 | |
| | BBH (acc_norm) | **48.4** | 39.4 | +9.0 | |
| | MuSR (acc_norm) | **41.7** | 38.3 | +3.3 | |
| | MMLU-Pro (acc) | 25.0 | **26.7** | −1.7 (within noise) | |
|
|
| ### With thinking on (GSM8K, first 500 test questions) |
|
|
| Standard prompt ("reason step by step, and put your final answer within \boxed{}"), greedy decoding, a 2,048-token |
| budget for both models. An answer counts only if the model finished thinking and its final answer is right. |
|
|
| | Model | GSM8K | |
| |---|---| |
| | Wiss 2B | **66.8** | |
| | Qwen3.5-2B (base) | 17.8 | |
|
|
| The base model's low score here is mostly a budget effect: it thinks at such length that most answers are cut off |
| at 2,048 tokens before a final answer. Wiss thinks briefly and finishes inside the budget. This is **not** evidence |
| that Wiss is a better mathematician than the base model with a larger budget (see the non-thinking table above, where |
| the base model is ahead on math). |
|
|
| ## Usage |
|
|
| Needs `transformers >= 5`. |
|
|
| ```python |
| from transformers import AutoModelForImageTextToText, AutoProcessor |
| |
| model_id = "Wiss-2B" # path or Hub id of this model |
| processor = AutoProcessor.from_pretrained(model_id) |
| model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto", device_map="auto") |
| |
| messages = [{"role": "user", "content": [{"type": "text", "text": "Write a Python function that merges two sorted lists."}]}] |
| # pictures: add {"type": "image", "image": "photo.jpg"} to the content list |
| inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, |
| return_tensors="pt", enable_thinking=True).to(model.device) |
| out = model.generate(**inputs, max_new_tokens=1024, do_sample=True, temperature=0.6, top_p=0.95, top_k=20) |
| print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| Recommended sampling: thinking on `temperature=0.6, top_p=0.95, top_k=20`; thinking off |
| `temperature=0.7, top_p=0.8, top_k=20`. Tools are passed in the standard OpenAI format through the chat template |
| (`tools=[...]`). |
|
|
| For llama.cpp (text and images), use the GGUF build with its image projector: |
|
|
| ```bash |
| llama-server -m Wiss-2B-Q5_K_M.gguf --mmproj mmproj-Wiss-2B-f16.gguf --jinja --image-max-tokens 2048 |
| ``` |
|
|
| ## Training |
|
|
| - **Method:** LoRA (r=32, α=64) on every linear projection of the language model, including the Gated DeltaNet |
| linear-attention layers, merged into the weights. The image encoder and projector are unchanged from Qwen3.5-2B. |
| - **Schedule:** 2 epochs, peak LR 1e-4 with cosine decay, max length 2,048 tokens, loss only on assistant turns. |
| - **Compute:** one RTX 4090, about 22 minutes, with [Unsloth](https://github.com/unslothai/unsloth). |
| - **Data:** 2,974 verified synthetic conversations (2,362 with reasoning, 697 without), plus 85 held out; multi-step |
| conversations train every assistant turn, giving 4,442 training examples. Every row passed an automatic check |
| (unit tests, exact tool-call matching or answer matching) before it was kept. |
|
|
| | Family | Conversations | |
| |---|---| |
| | Code | 800 | |
| | Reasoning | 697 | |
| | Agent (multi-step tool use) | 678 | |
| | Instruction following | 346 | |
| | Grounded answers | 253 | |
| | Single-call tools | 200 | |
|
|
| Exact and near duplicates were removed, and the set was checked for 13-gram overlap with GSM8K, MATH-500, |
| HumanEval, MBPP and IFEval (0 overlaps found). It was not checked against MMLU-Pro, GPQA, BBH or MuSR. |
| - **Not trained on:** general-knowledge text, competition-level math, or any replay of the base model's own outputs. |
|
|
| ## Limitations |
|
|
| - **Weaker than the base model at instruction following and hard math** (see the benchmark table), and somewhat |
| weaker at code on HumanEval. If you need strict format-following or competition math, use Qwen3.5-2B. |
| - Tool use and vision were the training goals, but the benchmarks above do not measure them. |
| - A 2B model: it makes small mistakes, especially in code details. Test what it writes. |
| - Vision was not fine-tuned. It describes pictures well but can invent details in dense screenshots; give it a high |
| image-token budget or crop to the part that matters. |
| - English only in training; other languages come from the base model. |
| - Single runs; the multiple-choice sets are small, so differences of a few points are within noise. |
|
|
| ## License |
|
|
| Apache 2.0, the license of the base model [Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B). Synthetic training data |
| was generated with [gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b) (Apache 2.0), Pixel Canary (a model offered |
| on the Vercel AI Gateway) and Ling 3.1 Flash (inclusionAI). |
