Instructions to use ssuasd/Caphlon-Model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use ssuasd/Caphlon-Model with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ssuasd/Caphlon-Model:IQ2_S
# Run inference directly in the terminal:
llama cli -hf ssuasd/Caphlon-Model:IQ2_S

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ssuasd/Caphlon-Model:IQ2_S
# Run inference directly in the terminal:
llama cli -hf ssuasd/Caphlon-Model:IQ2_S

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ssuasd/Caphlon-Model:IQ2_S
# Run inference directly in the terminal:
./llama-cli -hf ssuasd/Caphlon-Model:IQ2_S

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ssuasd/Caphlon-Model:IQ2_S
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ssuasd/Caphlon-Model:IQ2_S

Use Docker

docker model run hf.co/ssuasd/Caphlon-Model:IQ2_S

Llama new
LM Studio
Jan

vLLM

How to use ssuasd/Caphlon-Model with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ssuasd/Caphlon-Model"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ssuasd/Caphlon-Model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ssuasd/Caphlon-Model:IQ2_S

Ollama
How to use ssuasd/Caphlon-Model with Ollama:
```
ollama run hf.co/ssuasd/Caphlon-Model:IQ2_S
```
Unsloth Desktop

How to use ssuasd/Caphlon-Model with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ssuasd/Caphlon-Model:IQ2_S

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ssuasd/Caphlon-Model:IQ2_S"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use ssuasd/Caphlon-Model with Docker Model Runner:
```
docker model run hf.co/ssuasd/Caphlon-Model:IQ2_S
```

Lemonade

How to use ssuasd/Caphlon-Model with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ssuasd/Caphlon-Model:IQ2_S

Run and chat with the model

lemonade run user.Caphlon-Model-IQ2_S

List all available models

lemonade list

Hermes Agent

How to use ssuasd/Caphlon-Model with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ssuasd/Caphlon-Model:IQ2_S

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ssuasd/Caphlon-Model:IQ2_S

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use ssuasd/Caphlon-Model with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ssuasd/Caphlon-Model:IQ2_S

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ssuasd/Caphlon-Model:IQ2_S" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Caphlon-Model / README.md

ssuasd

Upload README.md with huggingface_hub

896ab24 verified 2 days ago

preview code

Raw History Blame Contribute Delete

3.95 kB

metadata

license: apache-2.0
base_model: Qwen/Qwen3.8-27B
library_name: llama.cpp
language:
  - en
pipeline_tag: text-generation
tags:
  - gguf
  - llama.cpp
  - qwen3
  - qwen
  - 27b
  - iq2_s
  - quantized
  - lightweight
  - local-llm
  - text-generation
  - hybrid-attention
  - layer-pruning

Caphlon-Model

A 6.87 GiB GGUF build of Qwen 3.8 27B — full 27B reasoning quality, laptop-sized footprint.

Built and benchmarked on 25 October 2026. No distillation, no student model: the 27B topology survives. Footprint was cut by removing representation redundancy — vocabulary merge-table reduction and hybrid-interval-aware layer pruning — on top of an Unsloth Dynamic IQ2_S base.

Model details


Base model	Qwen 3.8 27B (Apache-2.0)
Intermediate	Unsloth Dynamic IQ2_S
Architecture	60-layer hybrid: gated-delta-net (majority) + full-attention (minority)
File	`Caphlon-Model-IQ2_S.gguf` — 6.87 GiB (7,375,122,272 bytes)
SHA-256	`8487b4e443834659b230e0bd529674ad65e068fc00380b996aaf11c873d63bc0`
Format	GGUF (llama.cpp)
Context	up to 32K tested
Language	English

KV cache lives only on the full-attention layers; gated-delta-net layers carry fixed-size recurrent state, so memory stays small and context scales cheaply.

Changes vs. the base quant

Step	Change	Saved
1	Vocabulary merge table reduced to a 100k target (closure-preserving selection)	−649 MiB
2	Layers 12–15 removed (4 layers, aligned to the hybrid interval)	−302 MiB
3	Weights kept at IQ2_S — no further quantization damage	—

Benchmarks

Measured 2026-10-25, Tesla T4 (16 GB), llama.cpp, temperature 0, thinking disabled.

Test	Caphlon-Model	Base quant	Result
English perplexity (`wiki.test.raw`, 512 ctx × 40 chunks)	6.6088	6.3638	+3.85%
HumanEval, 30 problems, pass@1	29/30	30/30	96.7%
MMLU, 502-question subset	344/502	370/502	92.9% retained
File-edit throughput (`--spec-type ngram-mod`)	29.7 tok/s	12.7 tok/s	2.34×
Free-form prose throughput	12.4 tok/s	12.7 tok/s	—

Hardware requirements

Setup	Minimum	Comfortable
GPU, full offload	8 GB VRAM (8K ctx)	12 GB VRAM (32K ctx)
CPU only	8 GB RAM (mmap)	16 GB RAM
Apple Silicon	16 GB unified	24 GB+
Disk	7 GB	—

Usage

llama-server -m Caphlon-Model-IQ2_S.gguf \
  -ngl 99 -c 32768 -np 1 \
  -b 4096 -ub 1024 --flash-attn on \
  --spec-type ngram-mod --spec-ngram-mod-n-match 24 \
  --spec-ngram-mod-n-min 48 --spec-ngram-mod-n-max 64 \
  --jinja --port 8080

Send chat_template_kwargs: {"enable_thinking": false} on chat requests, otherwise the model spends the token budget on internal reasoning.

Optional, only when context-bound: --cache-type-k q8_0 --cache-type-v q4_0 (measured gain −152 MB, but greedy output is then no longer byte-identical).

License and attribution

Weights are a derivative of Qwen 3.8 27B (Apache-2.0); the upstream license governs them and permits commercial use with notices retained. Documentation, naming, and evaluation scripts by the author are Apache-2.0.

"Caphlon" is a project mark. Apache-2.0 grants no trademark rights: you may redistribute the weights, but you may not present a modified build as official, endorsed, or authored by this project.

Citation

@misc{caphlon2026,
  title  = {Caphlon-Model: A 6.87 GiB lightweight build of Qwen 3.8 27B},
  author = {demiralpdev},
  year   = {2026},
  month  = {10},
  note   = {GGUF, IQ2\_S base, vocabulary-reduced and layer-pruned},
  url    = {https://github.com/demiralpdev/Caphlon-Model}
}

GitHub: https://github.com/demiralpdev/Caphlon-Model
Project page: https://demiralpdev.github.io/Caphlon-Model/