README.md

---
language:
- sl
- en
license: other
license_name: mixed-see-provenance
library_name: transformers
base_model: SicariusSicariiStuff/Qwen3.5-4B_Abliterated
pipeline_tag: text-generation
tags:
- slovenian
- qwen3
- assistant
- on-device
- tool-calling
- reasoning
- gguf
---

# Sokol — Slovenian Assistant LLM (4B, on-device)

**Sokol** (*falcon*) is a small **4B** Slovenian assistant built for **fully-local, on-device use**
(desktop via **GGUF** / llama.cpp / LM Studio). It chats in fluent Slovenian, translates EN↔SL, reasons
through tasks (`<think>`), and makes **native tool / function calls** (Qwen3.6 XML format).

It is a **text LLM**, not a speech model — pair it with your own ASR + TTS to build a local voice
assistant (it routes `<think>` to `reasoning_content`, so reasoning is never spoken).

This single repo ships both the **full-precision weights** (safetensors) and **quantized GGUFs**
(under [`gguf/`](./gguf)), so you can run it with transformers, or fully local with llama.cpp / LM Studio.

## What it is

- **Base:** `SicariusSicariiStuff/Qwen3.5-4B_Abliterated` (dense, `qwen3_5` arch, natively supported
  in transformers ≥ 5.6; multimodal-capable).
- **Phase 2 — CPT (full-parameter):** continued pretraining on **~1.78 B tokens** of Slovenian text
  for language adaptation (full-FT, DeepSpeed ZeRO-2, 1 epoch).
- **Phase 3 — SFT (LoRA):** chat + translation + tool-calling + reasoning + identity (~250 k examples).
- **Deployment:** **GGUF** (desktop, llama.cpp / LM Studio) and full safetensors (transformers).

## Files in this repo

| path | format | use |
|---|---|---|
| `*.safetensors` (+ `config.json`, tokenizer, `chat_template.jinja`) | full BF16 | transformers, further fine-tuning, requantization |
| `gguf/sokol-4b-slo-Q8_0.gguf` | GGUF Q8_0 | highest-quality local (largest) |
| `gguf/sokol-4b-slo-Q6_K.gguf` | GGUF Q6_K | near-lossless, smaller |
| `gguf/sokol-4b-slo-Q5_K_M.gguf` | GGUF Q5_K_M | balanced |
| `gguf/sokol-4b-slo-Q4_K_M.gguf` | GGUF Q4_K_M | smallest, fastest |

> GGUFs are exported with `--no-mtp` (the multi-token-prediction head is dropped), which is required
> for the `qwen3_5` GDN-hybrid architecture to load cleanly in current llama.cpp.

## Run it (GGUF, llama.cpp)

```bash
# thinking + native tool calls; --reasoning-budget routes <think> to reasoning_content
llama-server -m gguf/sokol-4b-slo-Q5_K_M.gguf --jinja --reasoning-budget 8192 \
             -c 8192 --temp 0.4 --top-p 0.9
```

Or open the `.gguf` directly in **LM Studio**. For clean, non-thinking output pass
`--reasoning-budget 0` (useful when driving TTS in a voice stack, so `<think>` is never spoken).

## Capabilities

- **Slovenian chat** — fluent, formal Slovenian conversation.
- **Translation** — bidirectional EN↔SL.
- **Reasoning** — emits `<think>` traces (distilled from a large Qwen3.8-Max teacher); pass
  `enable_thinking=False` for direct answers.
- **Tool / function calling** — native Qwen3.6 XML, e.g.
  `<tool_call><function=get_weather><parameter=city>Ljubljana</parameter></function></tool_call>`.
- **Identity** — presents as *Sokol*, a local Slovenian assistant.

## Training data (provenance)

### CPT — Slovenian text (~1.68 M docs / ~1.78 B tokens)
| dataset | docs | source | license |
|---|---|---|---|
| `sl_wiki` | 160,539 | Slovenian Wikipedia | CC BY-SA |
| `sl_fineweb` | 1,523,000 | FineWeb2 (sl subset) | ODC-BY |

### SFT — ~250 k examples
| dataset | examples | what | source |
|---|---|---|---|
| `sft_chat_sl` | 79,324 | Slovenian conversation | `cjvt/GaMS-Nemotron-Chat` |
| `sft_translate` | 120,000 | bidirectional EN↔SL translation | OPUS-derived |
| `sft_toolcall_hermes` | 5,086 | tool calling | Hermes function-calling |
| `sft_toolcall_apigen_sokol` | 4,829 | tool calling (length-filtered) | APIGen-MT |
| `sft_toolcall_toolace` | 9,171 | tool calling | ToolACE |
| `qwen38max_reasoning_sl` | 31,255 | reasoning `<think>` | distilled from Qwen3.8-Max |
| `identity_sokol_sl` | 56 | Sokol identity | hand-written |

Nothing is truncated: all SFT examples are ≤ 7942 content tokens (cutoff 8192).

## Evaluation

### Slovenian-LLM-Eval — acc_norm (`cjvt/slovenian-llm-eval`, 200/task)

Sokol (CPT + SFT) vs the abliterated base, same eval, via `flywheel-sl`:

| task | base Qwen3.5-4B | **Sokol** | Δ |
|---|---|---|---|
| arc_easy | 0.565 | **0.610** | +0.045 |
| arc_challenge | 0.425 | **0.450** | +0.025 |
| hellaswag | 0.460 | **0.550** | +0.090 |
| piqa | 0.605 | **0.665** | +0.060 |
| openbookqa | 0.400 | **0.445** | +0.045 |
| winogrande | 0.525 | **0.590** | +0.065 |
| boolq | 0.825 | 0.825 | +0.000 |
| **AVG** | **0.544** | **0.591** | **+0.047** |

Wins 6/7 tasks, ties 1, no regressions. **CPT** also halved Slovenian perplexity
(6.996 → 3.438 on held-out SL text, −50.9%).

### Judge track — chat quality (1–5)

Open-ended answers to 32 general Slovenian questions (8 categories), judged by a stronger **27B
Slovenian judge model** on correctness / helpfulness / language:

| metric | base Qwen3.5-4B | **Sokol** | Δ |
|---|---|---|---|
| jezik (fluency) | 4.88 | **4.97** | +0.09 |
| koristnost (helpfulness) | 4.84 | 4.81 | −0.03 |
| pravilnost (correctness) | 4.78 | 4.63 | −0.16 |

Sokol clearly wins **language fluency** and **tool-calling** (per-category +0.67) and the objective
acc_norm; the small aggregate correctness dip comes almost entirely from two n=4 categories
(grammar, instruction-format), i.e. 1–2 questions each — noise at this eval size, not a systematic
regression. (v1 is 32 questions; per-category deltas need a larger v2 to be reliable.)

## ⚠️ Disclaimers & license

- **General assistant.** Outputs may be incorrect or fabricated; not professional advice.
  Verify anything important.
- **Mixed data-license provenance** (Wikipedia CC BY-SA, FineWeb2 ODC-BY, GaMS-Nemotron-Chat, OPUS,
  Hermes/APIGen/ToolACE, Qwen3.8-Max distillation). Marked `license: other`; **resolve data terms
  before public/commercial use.** Derivative of Qwen3.5-4B — the upstream Qwen license also applies.
- Provided **AS IS**, no warranty. See `LICENSES.md`.

## Citation

```
@misc{sokol_4b_slovenian_assistant,
  title  = {Sokol: a small on-device Slovenian assistant LLM (4B)},
  author = {Tadej Fius},
  year   = {2026},
  note   = {Derivative of Qwen3.5-4B. Slovenian CPT+SFT. General Slovenian assistant LLM.}
}
```