Instructions to use KorshLAB/KorshZero-0.33B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use KorshLAB/KorshZero-0.33B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="KorshLAB/KorshZero-0.33B")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("KorshLAB/KorshZero-0.33B") model = AutoModelForCausalLM.from_pretrained("KorshLAB/KorshZero-0.33B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use KorshLAB/KorshZero-0.33B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf KorshLAB/KorshZero-0.33B:Q4_K_M # Run inference directly in the terminal: llama cli -hf KorshLAB/KorshZero-0.33B:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf KorshLAB/KorshZero-0.33B:Q4_K_M # Run inference directly in the terminal: llama cli -hf KorshLAB/KorshZero-0.33B:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf KorshLAB/KorshZero-0.33B:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf KorshLAB/KorshZero-0.33B:Q4_K_M
Use Docker
docker model run hf.co/KorshLAB/KorshZero-0.33B:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use KorshLAB/KorshZero-0.33B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "KorshLAB/KorshZero-0.33B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KorshLAB/KorshZero-0.33B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/KorshLAB/KorshZero-0.33B:Q4_K_M
- SGLang
How to use KorshLAB/KorshZero-0.33B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "KorshLAB/KorshZero-0.33B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KorshLAB/KorshZero-0.33B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "KorshLAB/KorshZero-0.33B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KorshLAB/KorshZero-0.33B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Ollama
How to use KorshLAB/KorshZero-0.33B with Ollama:
ollama run hf.co/KorshLAB/KorshZero-0.33B:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use KorshLAB/KorshZero-0.33B with Docker Model Runner:
docker model run hf.co/KorshLAB/KorshZero-0.33B:Q4_K_M
- Lemonade
How to use KorshLAB/KorshZero-0.33B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull KorshLAB/KorshZero-0.33B:Q4_K_M
Run and chat with the model
lemonade run user.KorshZero-0.33B-Q4_K_M
List all available models
lemonade list
- Atomic Chat
π KorshZero-0.33B: The Sovereign Pocket Titan
100% Trained From Scratch β’ Zero Corporate Distillation β’ 450+ tok/s Edge Inference
β‘ The Sovereign AI Manifesto
KorshZero-0.33B represents technological autonomy. While most small language models are distilled from proprietary corporate APIs (OpenAI, Anthropic, or Meta), KorshZero-0.33B was initialized and trained 100% from scratch.
Developed by KorshLAB, this 325-million parameter, 33-layer pocket titan fuses high-efficiency neural topology with deep domain specialization in modern software engineering, Bento grid UI design, and native autonomous agent protocols.
"Your infrastructure. Your weights. Complete technological sovereignty. Zero external API calls. Zero data leakage."
π― Why Choose KorshZero-0.33B?
- ποΈ Blazing Speed (448+ tokens/second): Engineered for ultra-low latency with sub-15ms time-to-first-token on standard consumer hardware and over 6,600 tokens/second prompt evaluation throughput.
- π¦ Minimal Footprint (173 MB to 331 MB in GGUF): Runs seamlessly on micro-servers, ultra-portable laptops, Raspberry Pi 5, mobile platforms, and local edge clusters without saturating memory.
- π€ Native Autonomous Agent Engine: Pre-trained with direct support for execution blocks and tool calling tokens:
<|skill|>,<|tool_call|>, and<|tool_result|>. - π¨ Mastery of Modern Web Architecture: Specialized in Tailwind CSS dark mode (
darkMode: 'class'), Bento grid structures, pure glassmorphism design tokens, vector SVG icons (zero emojis), and strict WCAG AAA accessibility. - π Sovereignty & Complete Privacy: Released under the permissive Apache 2.0 license. Deployable for commercial or private applications with zero external telemetry.
π GGUF Quantization Download Matrix
The full suite of quantized binaries is hosted directly in this repository:
| GGUF File | Quantization | File Size | RAM / VRAM Required | Recommended Deployment |
|---|---|---|---|---|
KorshZero-0.33B-instruct-F16.gguf |
F16 (16-bit) | 621 MB | ~800 MiB | Full precision reference, evaluation, and fine-tuning |
KorshZero-0.33B-instruct-Q8_0.gguf |
Q8_0 (8-bit) | 331 MB | ~450 MiB | Default Recommended: Near-lossless fidelity with peak throughput |
KorshZero-0.33B-instruct-Q6_K.gguf |
Q6_K (6-bit) | 311 MB | ~400 MiB | High-fidelity compression for local servers |
KorshZero-0.33B-instruct-Q5_K_M.gguf |
Q5_K_M (5-bit) | 239 MB | ~320 MiB | Optimal balance for laptops and mini-PCs |
KorshZero-0.33B-instruct-Q4_K_M.gguf |
Q4_K_M (4-bit) | 221 MB | ~280 MiB | Universal standard for edge and mobile inference |
KorshZero-0.33B-instruct-Q3_K_M.gguf |
Q3_K_M (3-bit) | 187 MB | ~240 MiB | Resource-constrained edge devices |
KorshZero-0.33B-instruct-Q2_K.gguf |
Q2_K (2-bit) | 173 MB | ~200 MiB | Extreme micro-memory constraints |
π οΈ Architecture & Specifications
KorshZero-0.33B Decoder-Only Transformer Topology:
βββ Parameters: 325,085,824 (Initialized & Trained From Scratch)
βββ Hidden Size: 896
βββ Intermediate Size (FFN): 2816 (SwiGLU Activation)
βββ Hidden Layers: 33
βββ Attention Heads: 14 (Query) / 2 (Key-Value) [Grouped-Query Attention]
βββ Head Dimension: 64
βββ Context Window: 2,048 tokens
βββ Normalization: RMSNorm (eps=1e-5)
βββ Positional Encoding: RoPE (theta=10000.0)
βββ Vocabulary: 16,384 Byte-Level BPE
βββ Word Embeddings: Tied (Tied Input/Output Weights)
π Quickstart Guides
1. Python with Hugging Face transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "KorshLAB/KorshZero-0.33B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
prompt = """<|bos|><|system|>You are KorshZero-0.33B, a sovereign software and UI/UX architect.<|eos|>
<|user|>Design a modern glassmorphic Bento grid card in Tailwind CSS.<|eos|>
<|assistant|>"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=300,
temperature=0.2,
top_p=0.9,
do_sample=True
)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=False)
print(response)
2. Ultra-Fast Local Inference with llama.cpp
Download any quantized .gguf file from this repository and run locally with zero Python overhead:
# Download the recommended Q8_0 quantization
huggingface-cli download KorshLAB/KorshZero-0.33B KorshZero-0.33B-instruct-Q8_0.gguf --local-dir .
# Run with GPU acceleration (CUDA, Metal, or Vulkan):
llama-cli -m KorshZero-0.33B-instruct-Q8_0.gguf \
-p "<|bos|><|system|>You are KorshZero-0.33B, a sovereign UI/UX architect.<|eos|><|user|>Create a modern responsive Bento grid header with functional dark mode.<|eos|><|assistant|>" \
-ngl 99 -c 2048 --temp 0.2 -n 300
3. One-Click Deployment with Ollama
An optimized Modelfile is included directly in this repository:
# Download the GGUF model and Modelfile
huggingface-cli download KorshLAB/KorshZero-0.33B KorshZero-0.33B-instruct-Q8_0.gguf Modelfile --local-dir .
# Build the model in Ollama
ollama create korshzero -f ./Modelfile
# Run interactively
ollama run korshzero "Design a minimalist Bento grid dashboard with glassmorphism in Tailwind CSS"
π License & Citation
This model is licensed under the Apache 2.0 License, granting full freedom for academic research and commercial products.
@misc{korshzero2026,
author = {KorshLAB},
title = {KorshZero-0.33B: A Sovereign 325M Parameter Language Model Trained from Scratch for Modern Web Architecture and Autonomous Agents},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/KorshLAB/KorshZero-0.33B}}
}
- Downloads last month
- -
