Text Generation
GGUF
llama.cpp
qwen3
cpu-inference
ssd-offload
ssd-streaming
low-memory
conversational
Instructions to use HelloSun/qwen3-0.6b-ssd-streaming with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use HelloSun/qwen3-0.6b-ssd-streaming with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf HelloSun/qwen3-0.6b-ssd-streaming:Q8_0 # Run inference directly in the terminal: llama cli -hf HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf HelloSun/qwen3-0.6b-ssd-streaming:Q8_0 # Run inference directly in the terminal: llama cli -hf HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf HelloSun/qwen3-0.6b-ssd-streaming:Q8_0 # Run inference directly in the terminal: ./llama-cli -hf HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf HelloSun/qwen3-0.6b-ssd-streaming:Q8_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
Use Docker
docker model run hf.co/HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
- LM Studio
- Jan
- vLLM
How to use HelloSun/qwen3-0.6b-ssd-streaming with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "HelloSun/qwen3-0.6b-ssd-streaming" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HelloSun/qwen3-0.6b-ssd-streaming", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
- Ollama
How to use HelloSun/qwen3-0.6b-ssd-streaming with Ollama:
ollama run hf.co/HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
- Unsloth Desktop
- Pi
How to use HelloSun/qwen3-0.6b-ssd-streaming with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "HelloSun/qwen3-0.6b-ssd-streaming:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use HelloSun/qwen3-0.6b-ssd-streaming with Docker Model Runner:
docker model run hf.co/HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
- Lemonade
How to use HelloSun/qwen3-0.6b-ssd-streaming with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
Run and chat with the model
lemonade run user.qwen3-0.6b-ssd-streaming-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use HelloSun/qwen3-0.6b-ssd-streaming with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use HelloSun/qwen3-0.6b-ssd-streaming with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf HelloSun/qwen3-0.6b-ssd-streaming:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "HelloSun/qwen3-0.6b-ssd-streaming:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download validate/validation.json from HelloSun/qwen3-0.6b-ssd-streaming: direct link, hf CLI and curl.
- Browser
- Download file 8.25 kB
-
https://huggingface.co/HelloSun/qwen3-0.6b-ssd-streaming/resolve/main/validate/validation.json
- Command line
-
hf download hf://HelloSun/qwen3-0.6b-ssd-streaming/validate/validation.json
-
curl -L -o validation.json https://huggingface.co/HelloSun/qwen3-0.6b-ssd-streaming/resolve/main/validate/validation.json
8.25 kB
| { | |
| "generated_at": "2026-10-05T08:20:02.346927+00:00", | |
| "model": "Qwen/Qwen3-0.6B-GGUF/Qwen3-0.6B-Q8_0.gguf", | |
| "llama_cpp_commit": "0504396140d1c882f5f6ee34466a42db7ae90114", | |
| "prompt": "請只回答數字:2 加 2 等於多少?", | |
| "response": { | |
| "choices": [ | |
| { | |
| "finish_reason": "stop", | |
| "index": 0, | |
| "message": { | |
| "role": "assistant", | |
| "content": "2 + 2 = 4" | |
| } | |
| } | |
| ], | |
| "created": 1791188402, | |
| "model": "qwen3-0.6b-q8_0", | |
| "system_fingerprint": "b1-0504396", | |
| "object": "chat.completion", | |
| "usage": { | |
| "completion_tokens": 12, | |
| "prompt_tokens": 28, | |
| "total_tokens": 40, | |
| "prompt_tokens_details": { | |
| "cached_tokens": 0 | |
| } | |
| }, | |
| "id": "chatcmpl-Zu2dxEGy1d50m76STbg5oxBlf1Bet1NS", | |
| "timings": { | |
| "cache_n": 0, | |
| "prompt_n": 28, | |
| "prompt_ms": 121.671, | |
| "prompt_per_token_ms": 4.3453928571428575, | |
| "prompt_per_second": 230.1287899335092, | |
| "predicted_n": 12, | |
| "predicted_ms": 328.544, | |
| "predicted_per_token_ms": 29.86763636363636, | |
| "predicted_per_second": 33.48105580987631 | |
| } | |
| }, | |
| "response_text": "2 + 2 = 4", | |
| "elapsed_seconds": 0.465, | |
| "usage": { | |
| "completion_tokens": 12, | |
| "prompt_tokens": 28, | |
| "total_tokens": 40, | |
| "prompt_tokens_details": { | |
| "cached_tokens": 0 | |
| } | |
| }, | |
| "timings": { | |
| "cache_n": 0, | |
| "prompt_n": 28, | |
| "prompt_ms": 121.671, | |
| "prompt_per_token_ms": 4.3453928571428575, | |
| "prompt_per_second": 230.1287899335092, | |
| "predicted_n": 12, | |
| "predicted_ms": 328.544, | |
| "predicted_per_token_ms": 29.86763636363636, | |
| "predicted_per_second": 33.48105580987631 | |
| }, | |
| "memory": { | |
| "sample_interval_seconds": 0.1, | |
| "sample_count": 14, | |
| "peak_rss_bytes": 713650176, | |
| "peak_rss_mib": 680.59, | |
| "peak_metric": "max(smaps_rollup.Rss, VmHWM)", | |
| "peak_anon_bytes": 60903424, | |
| "peak_process_swap_bytes": 0, | |
| "all_process_swap_samples_zero": true, | |
| "memory_limit_gib": 1.0, | |
| "within_memory_limit": true, | |
| "samples": [ | |
| { | |
| "time_unix": 1791188400, | |
| "VmHWM": 1458176, | |
| "VmRSS": 1458176, | |
| "VmSwap": 0, | |
| "Rss": 1789952, | |
| "Anonymous": 159744 | |
| }, | |
| { | |
| "time_unix": 1791188400, | |
| "VmHWM": 35917824, | |
| "VmRSS": 35917824, | |
| "VmSwap": 0, | |
| "Rss": 35917824, | |
| "Anonymous": 22663168 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 39743488, | |
| "VmRSS": 39743488, | |
| "VmSwap": 0, | |
| "Rss": 40239104, | |
| "Anonymous": 26787840 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 67735552, | |
| "VmRSS": 67735552, | |
| "VmSwap": 0, | |
| "Rss": 67735552, | |
| "Anonymous": 54153216 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 72695808, | |
| "VmRSS": 72695808, | |
| "VmSwap": 0, | |
| "Rss": 72695808, | |
| "Anonymous": 59047936 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 124624896, | |
| "VmRSS": 124624896, | |
| "VmSwap": 0, | |
| "Rss": 126427136, | |
| "Anonymous": 54571008 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 344309760, | |
| "VmRSS": 271564800, | |
| "VmSwap": 0, | |
| "Rss": 249901056, | |
| "Anonymous": 56467456 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 344309760, | |
| "VmRSS": 89395200, | |
| "VmSwap": 0, | |
| "Rss": 89395200, | |
| "Anonymous": 56913920 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 344309760, | |
| "VmRSS": 89395200, | |
| "VmSwap": 0, | |
| "Rss": 89395200, | |
| "Anonymous": 56913920 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 344309760, | |
| "VmRSS": 89395200, | |
| "VmSwap": 0, | |
| "Rss": 89395200, | |
| "Anonymous": 56913920 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 344309760, | |
| "VmRSS": 121397248, | |
| "VmSwap": 0, | |
| "Rss": 128864256, | |
| "Anonymous": 59080704 | |
| }, | |
| { | |
| "time_unix": 1791188402, | |
| "VmHWM": 344309760, | |
| "VmRSS": 243027968, | |
| "VmSwap": 0, | |
| "Rss": 314331136, | |
| "Anonymous": 60903424 | |
| }, | |
| { | |
| "time_unix": 1791188402, | |
| "VmHWM": 708186112, | |
| "VmRSS": 537792512, | |
| "VmSwap": 0, | |
| "Rss": 712339456, | |
| "Anonymous": 60903424 | |
| }, | |
| { | |
| "time_unix": 1791188402, | |
| "VmHWM": 708186112, | |
| "VmRSS": 634343424, | |
| "VmSwap": 0, | |
| "Rss": 713650176, | |
| "Anonymous": 60903424 | |
| } | |
| ] | |
| }, | |
| "system_swap_before_inference": { | |
| "SwapTotal": 2147479552, | |
| "SwapFree": 53248, | |
| "active_devices": 1, | |
| "used_bytes": 2147426304 | |
| }, | |
| "system_swap_after_inference": { | |
| "SwapTotal": 2147479552, | |
| "SwapFree": 53248, | |
| "active_devices": 1, | |
| "used_bytes": 2147426304 | |
| }, | |
| "storage": { | |
| "model_path": "models/Qwen3-0.6B-Q8_0.gguf", | |
| "model_size_bytes": 639446688, | |
| "kv_path": "models/kv-ssd.cache", | |
| "kv_file_bytes": 62390272, | |
| "weight_mmap_release_enabled": true, | |
| "kv_ssd_mmap_forced": true | |
| }, | |
| "memory_session_peak": { | |
| "sample_interval_seconds": 0.1, | |
| "sample_count": 14, | |
| "peak_rss_bytes": 713650176, | |
| "peak_rss_mib": 680.59, | |
| "peak_metric": "max(smaps_rollup.Rss, VmHWM)", | |
| "peak_anon_bytes": 60903424, | |
| "peak_process_swap_bytes": 0, | |
| "all_process_swap_samples_zero": true, | |
| "memory_limit_gib": 1.0, | |
| "within_memory_limit": true, | |
| "samples": [ | |
| { | |
| "time_unix": 1791188400, | |
| "VmHWM": 1458176, | |
| "VmRSS": 1458176, | |
| "VmSwap": 0, | |
| "Rss": 1789952, | |
| "Anonymous": 159744 | |
| }, | |
| { | |
| "time_unix": 1791188400, | |
| "VmHWM": 35917824, | |
| "VmRSS": 35917824, | |
| "VmSwap": 0, | |
| "Rss": 35917824, | |
| "Anonymous": 22663168 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 39743488, | |
| "VmRSS": 39743488, | |
| "VmSwap": 0, | |
| "Rss": 40239104, | |
| "Anonymous": 26787840 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 67735552, | |
| "VmRSS": 67735552, | |
| "VmSwap": 0, | |
| "Rss": 67735552, | |
| "Anonymous": 54153216 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 72695808, | |
| "VmRSS": 72695808, | |
| "VmSwap": 0, | |
| "Rss": 72695808, | |
| "Anonymous": 59047936 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 124624896, | |
| "VmRSS": 124624896, | |
| "VmSwap": 0, | |
| "Rss": 126427136, | |
| "Anonymous": 54571008 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 344309760, | |
| "VmRSS": 271564800, | |
| "VmSwap": 0, | |
| "Rss": 249901056, | |
| "Anonymous": 56467456 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 344309760, | |
| "VmRSS": 89395200, | |
| "VmSwap": 0, | |
| "Rss": 89395200, | |
| "Anonymous": 56913920 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 344309760, | |
| "VmRSS": 89395200, | |
| "VmSwap": 0, | |
| "Rss": 89395200, | |
| "Anonymous": 56913920 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 344309760, | |
| "VmRSS": 89395200, | |
| "VmSwap": 0, | |
| "Rss": 89395200, | |
| "Anonymous": 56913920 | |
| }, | |
| { | |
| "time_unix": 1791188401, | |
| "VmHWM": 344309760, | |
| "VmRSS": 121397248, | |
| "VmSwap": 0, | |
| "Rss": 128864256, | |
| "Anonymous": 59080704 | |
| }, | |
| { | |
| "time_unix": 1791188402, | |
| "VmHWM": 344309760, | |
| "VmRSS": 243027968, | |
| "VmSwap": 0, | |
| "Rss": 314331136, | |
| "Anonymous": 60903424 | |
| }, | |
| { | |
| "time_unix": 1791188402, | |
| "VmHWM": 708186112, | |
| "VmRSS": 537792512, | |
| "VmSwap": 0, | |
| "Rss": 712339456, | |
| "Anonymous": 60903424 | |
| }, | |
| { | |
| "time_unix": 1791188402, | |
| "VmHWM": 708186112, | |
| "VmRSS": 634343424, | |
| "VmSwap": 0, | |
| "Rss": 713650176, | |
| "Anonymous": 60903424 | |
| } | |
| ] | |
| }, | |
| "system_swap_after_session": { | |
| "SwapTotal": 2147479552, | |
| "SwapFree": 57344, | |
| "active_devices": 1, | |
| "used_bytes": 2147422208 | |
| }, | |
| "all_passed": false | |
| } | |