Instructions to use UnimeType/Transliteration-4B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use UnimeType/Transliteration-4B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf UnimeType/Transliteration-4B-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf UnimeType/Transliteration-4B-GGUF:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf UnimeType/Transliteration-4B-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf UnimeType/Transliteration-4B-GGUF:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf UnimeType/Transliteration-4B-GGUF:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf UnimeType/Transliteration-4B-GGUF:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf UnimeType/Transliteration-4B-GGUF:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf UnimeType/Transliteration-4B-GGUF:Q8_0

Use Docker

docker model run hf.co/UnimeType/Transliteration-4B-GGUF:Q8_0

LM Studio
Jan

vLLM

How to use UnimeType/Transliteration-4B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "UnimeType/Transliteration-4B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "UnimeType/Transliteration-4B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/UnimeType/Transliteration-4B-GGUF:Q8_0

Ollama
How to use UnimeType/Transliteration-4B-GGUF with Ollama:
```
ollama run hf.co/UnimeType/Transliteration-4B-GGUF:Q8_0
```
Unsloth Desktop

How to use UnimeType/Transliteration-4B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf UnimeType/Transliteration-4B-GGUF:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "UnimeType/Transliteration-4B-GGUF:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use UnimeType/Transliteration-4B-GGUF with Docker Model Runner:
```
docker model run hf.co/UnimeType/Transliteration-4B-GGUF:Q8_0
```

Lemonade

How to use UnimeType/Transliteration-4B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull UnimeType/Transliteration-4B-GGUF:Q8_0

Run and chat with the model

lemonade run user.Transliteration-4B-GGUF-Q8_0

List all available models

lemonade list

Hermes Agent

How to use UnimeType/Transliteration-4B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf UnimeType/Transliteration-4B-GGUF:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default UnimeType/Transliteration-4B-GGUF:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use UnimeType/Transliteration-4B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf UnimeType/Transliteration-4B-GGUF:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "UnimeType/Transliteration-4B-GGUF:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

UnimeType Transliteration-4B · GGUF

Type Pinyin, Romaji, Romanized Hindi, Hinglish, or Arabizi and convert it into the target writing system. Transliteration-4B is UnimeType's 4B-class local Convert model. It is designed to preserve English words, URLs, code, numbers, and punctuation inside mixed-language messages. Review the result before replacing your text; preservation is not guaranteed.

Downloads

File	Format
Transliteration-4B-Convert-Q8_0.gguf	8-bit quantized weights

Other formats: Safetensors · MLX 6-bit.

Run with llama.cpp

We tested this file with native llama.cpp Metal on an Apple M5 Pro MacBook Pro with 48 GB memory. The tested llama.cpp revision is 9e0e220594af405a62835dc3a27495729fd8506b.

llama-server -m Transliteration-4B-Convert-Q8_0.gguf \
  --host 127.0.0.1 --port 18347 -ngl 99 -c 16384 -np 1 \
  --reasoning off --chat-template-kwargs '{"enable_thinking":false}'

Send conversion requests to the local OpenAI-compatible /v1/chat/completions endpoint. Specify one target language, ask for replacement text only, disable thinking, and use temperature 0. This model is for conversion, not translation, Polish, or Explain.

curl http://127.0.0.1:18347/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"system","content":"Convert Pinyin to Simplified Chinese. Preserve clear English, URLs, code, numbers, punctuation and line breaks. Return only the converted text."},{"role":"user","content":"jintian yao finish README"}],"temperature":0,"max_tokens":128}'

This standalone example does not reproduce UnimeType's production prompt or evaluation pipeline. NVIDIA CUDA, AMD ROCm, iOS, Android, and LM Studio GGUF inference are unverified for this file. No minimum-memory or speed claim is made.

Observed Q8_0 results

We ran UnimeType's production Convert prompts and evaluator on September 6, 2026, using the Apple silicon environment above.

Product-pipeline check	Result
Fixed conversion suite, exact accepted output	101 / 127
Conversational-context suite, valid-reference exact agreement	14 / 71
Context structure and protected-literal count checks	74 / 76
Protected-literal count and order, eligible cases	31 / 32
Latin-word preservation, eligible fixed-suite cases	47 / 48
URL preservation, eligible context cases	16 / 16
Code-span preservation, eligible context cases	16 / 16
Numeric-literal preservation, eligible context cases	16 / 16
Punctuation-sequence preservation, eligible context cases	15 / 16

The fixed suite includes 20 locally routed cases, so these are pipeline results rather than raw-model accuracy. Five invalid reference answers are excluded from context agreement, but all 76 cases remain in structure checks. Context references are model-generated and not native-speaker reviewed. The suites have been used in development and are not an independent blind benchmark.

The full runs used production prompt code, but alias ordering was not fixed across processes. Their scores cannot establish a controlled improvement over MLX. A subsequent targeted comparison of the floating-point source fixed the request bytes, except the model identifier.

Preservation checks compare literal counts and order within eligible cases; categories overlap. Latin-word checks cover case-sensitive words shared by the input and all accepted references, excluding URLs, email addresses, and code spans.

Known limitations

An Arabic context case produced an extra trailing newline and failed the production format check. A local BF16 export reproduced this under a controlled production request. Short English fragments can be converted instead of preserved. A Japanese context case lost punctuation and an emoji. Ambiguous names, dialect choices, and short fragments can produce the wrong conversion. Inspect protected text and line breaks before accepting a replacement.

Origin and license

This GGUF file comes from the fine-tuned Safetensors checkpoint, which was exported from the retained floating-point fusion used to create the existing MLX 6-bit release. Fine-tuning used LoRA on a quantized 4-bit Qwen3.5-4B base. Floating-point storage does not restore the original base model's pre-quantization precision. This file is not a dequantization of the public 6-bit model or a substitution with the untuned Qwen model.

The exact original base snapshot revision was not recorded. PROVENANCE.json records source hashes and the converter revision. The export is text-only, with auxiliary MTP layers excluded. SHA256SUMS contains public-file checksums.

The base model's Apache 2.0 license is included in LICENSE.

训练数据集未公开。 The training dataset is not publicly released. No datasets, data-generation scripts, training code, or training logs are included.

UnimeType · MLX setup guide

Downloads last month: 82

GGUF

Model size

4B params

Architecture

qwen35

Hardware compatibility

8-bit

Model tree for UnimeType/Transliteration-4B-GGUF

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Quantized

(412)

this model