Instructions to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG",
	filename="Qwen3.5-0.8B-GEC-KAZ-RUS-ENG.Q4_0.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
# Run inference directly in the terminal:
llama cli -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
# Run inference directly in the terminal:
llama cli -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
# Run inference directly in the terminal:
./llama-cli -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0

Use Docker

docker model run hf.co/loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0

LM Studio
Jan

vLLM

How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0

Ollama
How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with Ollama:
```
ollama run hf.co/loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
```

Unsloth Studio

How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG to start chatting

How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with Docker Model Runner:
```
docker model run hf.co/loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
```

Lemonade

How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0

Run and chat with the model

lemonade run user.Qwen3.5-0.8B-GEC-KAZ-RUS-ENG-Q4_0

List all available models

lemonade list

Qwen3.5-0.8B GEC for Kazakh, Russian, and English

A compact 0.8B grammatical-error-correction / proofreading model for Kazakh, Russian, and English. It takes raw text — voice-dictation output, chat messages, quick notes — and cleans it up: restores punctuation, capitalization, and paragraph breaks, and fixes typos and spelling errors — while preserving your words, meaning, language, and tone. It does not rewrite, paraphrase, translate, soften, or add anything of its own.

Published as GGUF (Q4_0) for CPU inference with llama.cpp, LM Studio, and Ollama. ~537 MB, ~44 tokens/s on CPU.

By Loqira Labs — everyday AI you own, not rent. Runs on the hardware you already have — no cloud, no frontier API.

⚡ TL;DR

Task: grammar / spelling / punctuation / capitalization / paragraphing correction (GEC, proofreading).
Word-preserving: keeps your exact words, meaning and tone; it corrects, it does not rewrite.
Languages: Kazakh, Russian, English (incl. Kazakh special letters).
One request → one response. This is a single-turn model: use it for one-shot calls, not multi-turn chat.
Must-have runtime settings: the exact system prompt below, thinking OFF, temperature 0.
Format: GGUF Q4_0, ~537 MB, ~44 tok/s CPU.

What it does

Restores punctuation, capitalization, and paragraph structure.
Fixes typos and spelling errors.
Word-preserving — keeps your words, meaning, tone, and language intact.
Adds nothing, invents nothing, and does not soften or rephrase.

⚠️ Single-turn model — one request, one response

This model is trained and meant to be used as a one-shot call: send one input, get one corrected output. It is not a conversational model — do not build multi-turn dialogues, do not feed prior turns back in, and do not chain calls in a running chat context. Each correction is an independent request with a fresh context and the same system prompt.

Languages

Explicitly trained and evaluated: Kazakh (kk), Russian (ru), English (en) — including Kazakh-specific letters (қ, ө, ұ, ү, ә, і, ң, ғ, һ), which the model restores.
Other languages: the base model (Qwen3.5-0.8B) additionally covers 198+ languages. The correction behaviour may transfer to them, but this was not tested — treat any non-KK/RU/EN use as experimental.

Model details


Base model	Qwen/Qwen3.5-0.8B (hybrid: GatedDeltaNet + full attention, 24 layers, hidden 1024)
Relation	finetune (quantization-aware)
Method	QAT at 4/8/16-bit with train==deploy parity — our int4/int8 codes are injected into the GGUF, not re-quantized
Format	GGUF, Q4_0 weights + Q8_0 / F32 (short-conv, norms)
Size	~537 MB
Speed	~44 tokens/s on CPU
Chat template + stop token	Both baked into the GGUF

The stop token is <|im_end|> — clients that read the GGUF metadata pick it up automatically.

Usage

1. System prompt — required (Russian, verbatim)

The model was trained to always receive this exact system prompt. Pass it as the system message. It is a single unified prompt — the same for every input and every language (Kazakh, Russian, and English input all use this one prompt).

Отформатируй текст голосового ввода: расставь пунктуацию и заглавные буквы, разбей на абзацы, исправь опечатки и орфографические ошибки. Сохрани язык, слова и смысл, ничего не добавляй от себя.

The prompt is in Russian by design — that is what the model was trained on, so it gives the best results. A version of the prompt translated into another language also works, but noticeably worse — keep the Russian prompt unless you have a specific reason not to. Without a system prompt at all (or with an unrelated one) the model is out-of-distribution and quality drops sharply.

2. Thinking OFF — required

The model was trained with an empty reasoning block (<think>\n\n</think>\n\n). A client with "thinking / reasoning" enabled will inject <think>\n and push the model out-of-distribution (garbled prefix, quality loss). Keep thinking OFF and use temperature 0 (greedy, deterministic).

3. llama.cpp (`llama-cli`)

llama-cli -m Qwen3.5-0.8B-GEC-KAZ-RUS-ENG.Q4_0.gguf \
  -sys "Отформатируй текст голосового ввода: расставь пунктуацию и заглавные буквы, разбей на абзацы, исправь опечатки и орфографические ошибки. Сохрани язык, слова и смысл, ничего не добавляй от себя." \
  -p "привте всем сегдня я хочю расказать про наш новый праект над котрым мы работаем уже почти пол года" \
  -st -no-cnv --reasoning off --reasoning-format none \
  --chat-template-kwargs "{\"enable_thinking\": false}" \
  --temp 0 --top-k 1 --repeat-penalty 1.0 -n 512

For long inputs, raise -n (e.g. 1024–2048) so the output is not truncated.

4. llama-server (OpenAI-compatible API)

Start the server with --reasoning-format none. In each request send the system prompt above, temperature: 0, and "chat_template_kwargs": {"enable_thinking": false}. Strip any leading <think>...</think> from the returned content.

5. Ollama

A ready Modelfile is included — it bakes in the correct chat template (think-off), the system prompt, and greedy decoding.

ollama create qwen3.5-gec -f Modelfile
ollama run qwen3.5-gec "привте всем сегдня я хочю расказать про наш новый праект"

6. LM Studio

Load the GGUF, disable "thinking / reasoning", set temperature 0, and set the system prompt above. Verify that the rendered template produces <think>\n\n</think>\n\n (an empty block), not <think>\n.

Compatibility

This model uses the Qwen3.5 architecture (qwen35 — hybrid GatedDeltaNet + attention), supported in mainline llama.cpp and the apps built on it. Just use a recent llama.cpp / LM Studio / Ollama. If you ever see unknown model architecture 'qwen35', your app bundles an outdated llama.cpp — update it. Verified on the CPU path; on a GPU backend, run the golden-probe examples below first.

Examples (temperature 0, thinking off)

Russian — typos:

in : привте всем сегдня я хочю расказать про наш новый праект над котрым мы работаем уже почти пол года
out: Привет всем! Сегодня я хочу рассказать про наш новый проект, над которым мы работаем уже почти пол года.

English — typos:

in : helo evryone i jsut wanted to tel you abuot the meting we have tomorow mornig at nine oclock
out: Hello everyone, I just wanted to tell you about the meeting we have tomorrow morning at nine o'clock.

Kazakh — special letters restored (қ / ө / і):

in : мен калага бардым онда коп адам бар еди кыс кези суык болды
out: Мен қалаға бардым, Онда көп адам бар, Енді қыс кезі суық болды.

Preserves tone — does not soften or rewrite:

in : да ты издеваешься что ли опять всё сломал я же говорил не трогай этот код
out: Да ты издеваешься что ли? Опять всё сломал. Я же говорил: «Не трогай этот код».

You can use these four as a quick golden probe: if your client produces materially different output at temperature 0, the prompt or think-off setting is wrong — check the system prompt and thinking-off first.

Known limitations

Mixed script within one word (e.g. Cyrillic + Latin "блin") is not corrected.
Segmentation around URLs / @handles can be unstable (though URLs and handles themselves are preserved byte-for-byte).
"HH MM" time without a separator may get an extra dot.
~10% of isolated in-word typos may remain (not systematic); rare Kazakh word substitutions.
0.8B / 4-bit ceiling — this is a small, quantized model built for speed and privacy, not a frontier LLM.

License

Apache-2.0. Base model: Qwen/Qwen3.5-0.8B.

About Loqira Labs

Loqira Labs builds capable AI that runs on the hardware you already have — no cloud, no frontier API, no vendor to answer to. Everyday AI you own, not rent.

🌐 loqira.tech

Downloads last month: -

GGUF

Model size

0.8B params

Architecture

qwen35

Hardware compatibility

4-bit

Model tree for loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG

Base model

Qwen/Qwen3.5-0.8B-Base

Finetuned

Qwen/Qwen3.5-0.8B

Finetuned

(271)

this model

Qwen3.5-0.8B GEC for Kazakh, Russian, and English

⚡ TL;DR

What it does

⚠️ Single-turn model — one request, one response

Languages

Model details

Usage

1. System prompt — required (Russian, verbatim)

2. Thinking OFF — required

3. llama.cpp (llama-cli)

4. llama-server (OpenAI-compatible API)

5. Ollama

6. LM Studio

Compatibility

Examples (temperature 0, thinking off)

Known limitations

License

About Loqira Labs

Model tree for loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG

3. llama.cpp (`llama-cli`)