Instructions to use ZAQERIS/ZQN-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ZAQERIS/ZQN-12B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ZAQERIS/ZQN-12B", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("ZAQERIS/ZQN-12B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ZAQERIS/ZQN-12B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ZAQERIS/ZQN-12B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ZAQERIS/ZQN-12B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ZAQERIS/ZQN-12B
- SGLang
How to use ZAQERIS/ZQN-12B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ZAQERIS/ZQN-12B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ZAQERIS/ZQN-12B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ZAQERIS/ZQN-12B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ZAQERIS/ZQN-12B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use ZAQERIS/ZQN-12B with Docker Model Runner:
docker model run hf.co/ZAQERIS/ZQN-12B
- ZQN-12B (
ZAQERIS/ZQN-12B)- نبذة عن النموذج والشفافية التقنية (Overview in Arabic)
- 1. Actual Capabilities & Known Limitations (ما يفعله النموذج فعلياً وحدوده)
- 2. Architecture & Parameter Accounting (
verify_parameter_count_and_m4_expansion.py) - 3. Empirical Measurements & Offline Benchmarks (Verified Locally — Zero Synthetic Numbers)
- 4. Quickstart Usage (8 GB+ GPU or CPU)
- 5. Technical Documentation & Verification Suite
- نبذة عن النموذج والشفافية التقنية (Overview in Arabic)
ZQN-12B (ZAQERIS/ZQN-12B)
48-Layer Bilingual Text Model Compressed with ZAQERIS M3-GPTQ for 8 GB GPUs
Official Model Name: ZQN-12B | Repository: ZAQERIS/ZQN-12B | Model Family: ZQN (ZaqionDialecticForCausalLM)
Developed by: ZAQERIS AI (Independent AI Research Group, Cairo, Egypt)
Predecessor: ZAQERIS-AXIOM (79.43B total / 12.39B active Sparse MoE, 36 layers, M3 3.16 bpp)
Training Origin: Trained from scratch by ZAQERIS AI; all pre-training and SFT datasets are kept strictly private in-house and are not distributed in this repository.
Tokenizer Ownership (External): This tokenizer is NOT ours. ZQN-12B reuses an external pre-existing 152,064-token multilingual BPE tokenizer (vocab_size = 152,064, <|User|>, <|Assistant|>, <think>, </think>, <|end▁of▁sentence|>) for compatibility only.
License: Apache 2.0
Total Physical Parameters: 15,546,031,104 (15.55B unfolded: 13.21B 48-layer backbone + 1.56B embed/head (14.77B dense core) + 0.75B DVMU + 21.02M LTBDD)
Quantization: ZAQERIS M3-GPTQ (3x8b 8D RVQ + Block-Hadamard $g=128$ + OBQ salience 0.5% (0.4% lm_head) + Block-8 Cholesky + row normalization, 3.2484 bpw)
Compressed Disk Footprint: 6.479 GiB (6,957,289,320 bytes across 5 Safetensors shards)
Verified Consumer 8 GB Runtime: 6.032 GiB loaded CUDA VRAM / 6.614 GiB peak eval VRAM / 25.47 tok/s (9-prompt Stage-4 live suite, prefill=1.373 s) on RTX 5060 Laptop GPU, 8 GB
Context Window: 131,072 tokens Native (128K) / 262,144 tokens Extended via YaRN (256K, $m_{\text{scale}} = 1.069315$)
Quickstart Notebook: Quickstart & Verification Notebook
Quantization Paper: ZAQERIS M3-GPTQ Technical Paper
نبذة عن النموذج والشفافية التقنية (Overview in Arabic)
ZQN-12B هو النموذج الثاني من مجموعة ZAQERIS AI البحثية المستقلة (القاهرة، مصر) بعد الإصدار الأول ZAQERIS-AXIOM:
- التدريب والبيانات: تم تدريب نموذج ZQN-12B بالكامل من الصفر (Trained from Scratch) بواسطة ZAQERIS AI، مع الاحتفاظ بجميع بيانات التدريب والضبط الدقيق (Pre-training & SFT Corpora) خاصةً بنا داخلياً دون نشر أي ملفات تدريب في هذا المستودع العام.
- التوكنايزر (ملكية خارجية): التوكنايزر المرفق ليس من تطويرنا ولا نملكه؛ بل يعتمد النموذج فقط على توكنايزر BPE خارجي جاهز (
vocab_size = 152,064) ورموزه الخاصة القياسية (<|User|>,<|Assistant|>,<think>,</think>,<|end▁of▁sentence|>) لضمان التوافق البرمجي فقط. - الهيكلة والحجم الفعلي: رغم تسمية السلسلة التجارية
ZQN-12B، تضم النسخة الحالية 48 طبقة (d_model = 5120,intermediate_size = 13824,40رأس استعلام /8رؤوس KV،head_dim = 128) بإجمالي15,546,031,104معامل (14.77Bنواة كثيفة +0.75BDVMU +21.02MLTBDD =15.55B)، مضغوطة عبر ZAQERIS M3-GPTQ (3.2484 bpw) إلى6.479 GiBعلى القرص (6.032 GiBVRAM بعد التحميل).
1. Actual Capabilities & Known Limitations (ما يفعله النموذج فعلياً وحدوده)
Designed Capabilities (Text-Only Causal LM):
- Bilingual Arabic & English Reasoning + Dialogue: Supports step-by-step
<think>...</think>reasoning formatting and direct answers in Arabic and English. - Local Consumer 8 GB Deployment: Compressed with ZAQERIS M3-GPTQ (
3.2484 bpwlinear projections + CPU-offloadedINT8embed_tokenson<= 9 GiBCUDA GPUs +INT4 G64DVMU adapters), achieving6.614 GiBpeak VRAM and25.47 tok/s(Stage-49-prompt suite). - Structured Compact Tool Markup (
<call .../>/<plan ...>): Can emit structured XML-style tool calls that an external host or localZaqionAgentToolboxcan parse. StandaloneZaqionAgentToolboximplements opt-in localsh,linux_diag,read_file,write_file, andsearchonly; browser, email/outbox, web search, and media tools require an external host orchestrator. - Deterministic Lightweight Helpers Included in Modeling Code:
ZaqionSelfCorrectionEngine: a lightweight rule/lookup pre-normalizer (COMMON_CORRECTIONStable + trailingخ->كrule + mash-string rejection triggers), tested10/10on its specific 10-case unit file.ZaqionDialecticDeliberationEngine(LTBDD): a21.02M-parameter residual bottleneck block prior tolm_head.
Current Limitations (Realistic Scope):
- Early Checkpoint / Modest Academic Benchmark Scores: Stage-3
40-layer reference benchmarks achieved15.24%on HumanEval,19.00%on GSM8K (100tasks),45.50%on MMLU (200tasks),55.50%on ARC-Challenge (200tasks), and58.33%(7/12) on tool selection. It is not a frontier SOTA model. - Stage-4 Quantitative Coverage: Stage-4 (
48-layer) weights have been verified for compression (3.2484 bpw,mean_cos = 0.9731,calib_ppl = 10.6807), held-out perplexity (35.5500vs43.6063in Stage-3), throughput (25.47 tok/s), and2/5tool-tag emission onlive_eval_report.jsonagent prompts; the full 5-suite benchmark sweep (HumanEval/GSM8K/MMLU/ARC/Agent) has not yet been re-run on Stage-4. - No Built-In Web Search, Vision, or Audio Generation: Standalone
ZQN-12Bweights are text-only causal weights (search_enabled: false). - No Hardware Power Metering Recorded:
config.jsonsets"power_metering_recorded": false; no fabricated J/token or USD/1M cost metrics are claimed.
2. Architecture & Parameter Accounting (verify_parameter_count_and_m4_expansion.py)
| Component | Exact Parameters | Precision / Packing |
|---|---|---|
48-Layer Dense Backbone (layers.0..47 attn + MLP) |
13,212,893,184 (~13.21B) |
M3-GPTQ 3x8b (0.5% OBQ, normalize_rows) |
Embeddings (embed_tokens) + Output Head (lm_head) |
1,557,140,480 (~1.56B) |
INT8 per-row (embed_tokens) + M3-GPTQ 3x8b (lm_head, 0.4% OBQ) |
DVMU Virtual Manifold Experts (layers.0..47) |
754,974,720 (~0.75B) |
Group-64 Symmetric INT4 |
| LTBDD Tri-Branch Deliberation Engine | 21,022,720 (~21.02M) |
float16 |
| Total Physical Unfolded Parameters | 15,546,031,104 (15.55B) |
Total Disk Size: 6.479 GiB (5 Safetensors shards) |
Shard Layout (6.479 GiB / 6,957,289,320 Bytes Total)
model-00001-of-00005.safetensors:1,618,173,454bytes (1.51 GiB)model-00002-of-00005.safetensors:1,616,174,450bytes (1.51 GiB)model-00003-of-00005.safetensors:1,611,934,110bytes (1.50 GiB)model-00004-of-00005.safetensors:1,628,396,864bytes (1.52 GiB)model-00005-of-00005.safetensors:482,610,442bytes (0.45 GiB)
3. Empirical Measurements & Offline Benchmarks (Verified Locally — Zero Synthetic Numbers)
Evaluated locally on NVIDIA GeForce RTX 5060 Laptop GPU (8 GB VRAM), Windows 11, Python 3.12.10, PyTorch 2.11.0+cu128:
| Metric | Observed Result | Notes & Provenance |
|---|---|---|
| M3-GPTQ Effective Bitrate (Stage-4) | 3.2484 bpw | m3_compression_report.json (0.5% OBQ, damp=0.02, 16 RVQ iters) |
| Mean Matrix Weight Cosine (Stage-4) | 0.9731 | Across 48 layers + lm_head (32,768 calib tokens; calib_ppl=10.6807) |
| Loaded CUDA VRAM / Peak VRAM (Stage-4) | 6.032 GiB / 6.614 GiB | embed_tokens auto-hosted on CPU when GPU <= 9 GiB |
| Held-Out Perplexity (M3-GPTQ Stage-4) | 35.5500 | Improved from 43.6063 (Stage-3) on 148-token held-out sequence (prefill=1.373 s) |
| Decode Speed (Stage-4 Live Suite) | 25.47 tok/s (9 prompts) |
NVRTC Hadamard-LUT + Static T=1 CUDA Graph |
HumanEval (164/164 Stage-3 reference) |
15.24% (25/164) |
Greedy pass@1 in isolated 5s subprocess (benchmark_report.json; Stage-4 sweep pending) |
GSM8K (100/1,319 Stage-3 reference) |
19.00% (19/100) |
Exact numeric match (benchmark_report.json; Stage-4 sweep pending) |
MMLU (200/14,042 Stage-3 reference) |
45.50% (91/200) |
Deterministic 1-pass A/B/C/D log-prob (benchmark_report.json; Stage-4 sweep pending) |
AI2 ARC-Challenge (200/1,172 Stage-3 reference) |
55.50% (111/200) |
Deterministic 1-pass log-prob (benchmark_report.json; Stage-4 sweep pending) |
Arabic/English Agent Tool Suite (12 tasks, Stage-3) |
58.33% (7/12) / 100% JSON |
Unseen multi-tool calling (benchmark_report.json; Stage-4 live emitted calls on 2/5 agent prompts) |
| ASSC Rule Normalizer Unit Suite | 10 / 10 (100%) | evaluation_evidence/self_correction_typo_benchmark.jsonl |
| RoPE KV-Cache Position Consistency | 0.0e+00 drift | scripts/test_rope_kv_cache_consistency.py |
| YaRN 256K Frequency Stability | Verified (mscale = 1.069315) |
scripts/test_yarn_long_context.py |
| DVMU Active Tensor Divergence | Verified | scripts/test_dvmu_real_forward_pass.py |
(Note: generation_config.json sets repetition_penalty = 1.05, do_sample = False, and eos_token_id = [151645, 151643]. Full ablation details are in EVALUATION_BENCHMARK_COMPARISON.md.)
4. Quickstart Usage (8 GB+ GPU or CPU)
import sys
import torch
from transformers import AutoTokenizer
from huggingface_hub import snapshot_download
local_dir = snapshot_download("ZAQERIS/ZQN-12B")
sys.path.insert(0, local_dir)
from modeling_zaqeris_zaqion_12b import ZaqionDialecticForCausalLM
# External 152,064-vocab BPE tokenizer reused for compatibility (not trained by ZAQERIS AI)
tokenizer = AutoTokenizer.from_pretrained(local_dir, trust_remote_code=True)
model = ZaqionDialecticForCausalLM.from_m3_pretrained(
local_dir,
device="cuda" if torch.cuda.is_available() else "cpu",
).eval()
# Optional deterministic pre-normalizer
noisy_prompt = "ما قدراتخ وما هو اسم النموذج؟"
correction = model.correct_prompt(noisy_prompt)
print("Normalized Prompt:", correction["corrected_prompt"])
# Generate response using ZQN-12B chat template (preventing duplicate BOS)
messages = [{"role": "user", "content": correction["corrected_prompt"]}]
formatted = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
inputs = tokenizer(formatted, return_tensors="pt", add_special_tokens=False).to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=192,
do_sample=False,
repetition_penalty=1.05,
eos_token_id=[151645, 151643],
)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
5. Technical Documentation & Verification Suite
- Paper: ZAQERIS M3-GPTQ Compression
- Paper 01: Latent Tri-Branch Deliberation Block (LTBDD)
- Paper 03: Compact Tool Parsing & Sandbox Toolbox
- Paper 04: Lineage & Foundations of ZAQERIS AI (AXIOM & ZQN-12B)
- Paper 05: Deterministic Rule Self-Correction Pre-Processor (ASSC)
- Dialectical Deliberation Specification
- Inference Compatibility & Runtime Guide
- Training Provenance & Privacy Policy
- Evaluation Methodology & Environment
- Evaluation Benchmarks & Ablation Tables
- Hardware Footprint & Throughput Report
- Self-Correction Unit Suite Report
Executable Verification Suite:
python scripts/verify_parameter_count_and_m4_expansion.py
python scripts/test_rope_kv_cache_consistency.py
python scripts/test_yarn_long_context.py
python scripts/test_dvmu_real_forward_pass.py
python evaluation_evidence/reproduce_m3_benchmarks.py --model Qwen/Qwen2.5-Coder-1.5B
- Downloads last month
- 520