--- language: - sl - en license: other license_name: mixed-see-provenance library_name: transformers base_model: SicariusSicariiStuff/Qwen3.5-4B_Abliterated pipeline_tag: text-generation tags: - slovenian - qwen3 - assistant - on-device - tool-calling - reasoning - gguf --- # Sokol — Slovenian Assistant LLM (4B, on-device) **Sokol** (*falcon*) is a small **4B** Slovenian assistant built for **fully-local, on-device use** (desktop via **GGUF** / llama.cpp / LM Studio). It chats in fluent Slovenian, translates EN↔SL, reasons through tasks (``), and makes **native tool / function calls** (Qwen3.6 XML format). It is a **text LLM**, not a speech model — pair it with your own ASR + TTS to build a local voice assistant (it routes `` to `reasoning_content`, so reasoning is never spoken). This single repo ships both the **full-precision weights** (safetensors) and **quantized GGUFs** (under [`gguf/`](./gguf)), so you can run it with transformers, or fully local with llama.cpp / LM Studio. ## What it is - **Base:** `SicariusSicariiStuff/Qwen3.5-4B_Abliterated` (dense, `qwen3_5` arch, natively supported in transformers ≥ 5.6; multimodal-capable). - **Phase 2 — CPT (full-parameter):** continued pretraining on **~1.78 B tokens** of Slovenian text for language adaptation (full-FT, DeepSpeed ZeRO-2, 1 epoch). - **Phase 3 — SFT (LoRA):** chat + translation + tool-calling + reasoning + identity (~250 k examples). - **Deployment:** **GGUF** (desktop, llama.cpp / LM Studio) and full safetensors (transformers). ## Files in this repo | path | format | use | |---|---|---| | `*.safetensors` (+ `config.json`, tokenizer, `chat_template.jinja`) | full BF16 | transformers, further fine-tuning, requantization | | `gguf/sokol-4b-slo-Q8_0.gguf` | GGUF Q8_0 | highest-quality local (largest) | | `gguf/sokol-4b-slo-Q6_K.gguf` | GGUF Q6_K | near-lossless, smaller | | `gguf/sokol-4b-slo-Q5_K_M.gguf` | GGUF Q5_K_M | balanced | | `gguf/sokol-4b-slo-Q4_K_M.gguf` | GGUF Q4_K_M | smallest, fastest | > GGUFs are exported with `--no-mtp` (the multi-token-prediction head is dropped), which is required > for the `qwen3_5` GDN-hybrid architecture to load cleanly in current llama.cpp. ## Run it (GGUF, llama.cpp) ```bash # thinking + native tool calls; --reasoning-budget routes to reasoning_content llama-server -m gguf/sokol-4b-slo-Q5_K_M.gguf --jinja --reasoning-budget 8192 \ -c 8192 --temp 0.4 --top-p 0.9 ``` Or open the `.gguf` directly in **LM Studio**. For clean, non-thinking output pass `--reasoning-budget 0` (useful when driving TTS in a voice stack, so `` is never spoken). ## Capabilities - **Slovenian chat** — fluent, formal Slovenian conversation. - **Translation** — bidirectional EN↔SL. - **Reasoning** — emits `` traces (distilled from a large Qwen3.8-Max teacher); pass `enable_thinking=False` for direct answers. - **Tool / function calling** — native Qwen3.6 XML, e.g. `Ljubljana`. - **Identity** — presents as *Sokol*, a local Slovenian assistant. ## Training data (provenance) ### CPT — Slovenian text (~1.68 M docs / ~1.78 B tokens) | dataset | docs | source | license | |---|---|---|---| | `sl_wiki` | 160,539 | Slovenian Wikipedia | CC BY-SA | | `sl_fineweb` | 1,523,000 | FineWeb2 (sl subset) | ODC-BY | ### SFT — ~250 k examples | dataset | examples | what | source | |---|---|---|---| | `sft_chat_sl` | 79,324 | Slovenian conversation | `cjvt/GaMS-Nemotron-Chat` | | `sft_translate` | 120,000 | bidirectional EN↔SL translation | OPUS-derived | | `sft_toolcall_hermes` | 5,086 | tool calling | Hermes function-calling | | `sft_toolcall_apigen_sokol` | 4,829 | tool calling (length-filtered) | APIGen-MT | | `sft_toolcall_toolace` | 9,171 | tool calling | ToolACE | | `qwen38max_reasoning_sl` | 31,255 | reasoning `` | distilled from Qwen3.8-Max | | `identity_sokol_sl` | 56 | Sokol identity | hand-written | Nothing is truncated: all SFT examples are ≤ 7942 content tokens (cutoff 8192). ## Evaluation ### Slovenian-LLM-Eval — acc_norm (`cjvt/slovenian-llm-eval`, 200/task) Sokol (CPT + SFT) vs the abliterated base, same eval, via `flywheel-sl`: | task | base Qwen3.5-4B | **Sokol** | Δ | |---|---|---|---| | arc_easy | 0.565 | **0.610** | +0.045 | | arc_challenge | 0.425 | **0.450** | +0.025 | | hellaswag | 0.460 | **0.550** | +0.090 | | piqa | 0.605 | **0.665** | +0.060 | | openbookqa | 0.400 | **0.445** | +0.045 | | winogrande | 0.525 | **0.590** | +0.065 | | boolq | 0.825 | 0.825 | +0.000 | | **AVG** | **0.544** | **0.591** | **+0.047** | Wins 6/7 tasks, ties 1, no regressions. **CPT** also halved Slovenian perplexity (6.996 → 3.438 on held-out SL text, −50.9%). ### Judge track — chat quality (1–5) Open-ended answers to 32 general Slovenian questions (8 categories), judged by a stronger **27B Slovenian judge model** on correctness / helpfulness / language: | metric | base Qwen3.5-4B | **Sokol** | Δ | |---|---|---|---| | jezik (fluency) | 4.88 | **4.97** | +0.09 | | koristnost (helpfulness) | 4.84 | 4.81 | −0.03 | | pravilnost (correctness) | 4.78 | 4.63 | −0.16 | Sokol clearly wins **language fluency** and **tool-calling** (per-category +0.67) and the objective acc_norm; the small aggregate correctness dip comes almost entirely from two n=4 categories (grammar, instruction-format), i.e. 1–2 questions each — noise at this eval size, not a systematic regression. (v1 is 32 questions; per-category deltas need a larger v2 to be reliable.) ## ⚠️ Disclaimers & license - **General assistant.** Outputs may be incorrect or fabricated; not professional advice. Verify anything important. - **Mixed data-license provenance** (Wikipedia CC BY-SA, FineWeb2 ODC-BY, GaMS-Nemotron-Chat, OPUS, Hermes/APIGen/ToolACE, Qwen3.8-Max distillation). Marked `license: other`; **resolve data terms before public/commercial use.** Derivative of Qwen3.5-4B — the upstream Qwen license also applies. - Provided **AS IS**, no warranty. See `LICENSES.md`. ## Citation ``` @misc{sokol_4b_slovenian_assistant, title = {Sokol: a small on-device Slovenian assistant LLM (4B)}, author = {Tadej Fius}, year = {2026}, note = {Derivative of Qwen3.5-4B. Slovenian CPT+SFT. General Slovenian assistant LLM.} } ```