Instructions to use AtomicChat/SenseNova-Loop-9B-Preview-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AtomicChat/SenseNova-Loop-9B-Preview-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
Use Docker
docker model run hf.co/AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use AtomicChat/SenseNova-Loop-9B-Preview-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AtomicChat/SenseNova-Loop-9B-Preview-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AtomicChat/SenseNova-Loop-9B-Preview-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
- Ollama
How to use AtomicChat/SenseNova-Loop-9B-Preview-GGUF with Ollama:
ollama run hf.co/AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use AtomicChat/SenseNova-Loop-9B-Preview-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use AtomicChat/SenseNova-Loop-9B-Preview-GGUF with Docker Model Runner:
docker model run hf.co/AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
- Lemonade
How to use AtomicChat/SenseNova-Loop-9B-Preview-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.SenseNova-Loop-9B-Preview-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use AtomicChat/SenseNova-Loop-9B-Preview-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use AtomicChat/SenseNova-Loop-9B-Preview-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "AtomicChat/SenseNova-Loop-9B-Preview-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Q4_K_M: ssm_alpha / ssm_beta are Q4_K — possible missing F32 pin; plus a metadata/layout question
Hi AtomicChat team!!!
Thanks for publishing these -- the README is unusually clear, and the mHC routing precision note is exactly the right thing to document. I downloaded SenseNova-Loop-9B-Preview-Q4_K_M.gguf (SHA-256 f3b321f001b04fdc8915b4067f0592f2b7cf55676b584080bc3129b964bf1d4b, 5,628,723,104 bytes) and inspected the actual GGUF tensor types so I have two observations
- The SSM alpha / beta gates are not F32 in Q4_K_M
The model has 24 SSM layers, and each has ssm_alpha.weight and ssm_beta.weight, shape [4096, 32] — 48 tensors total. In your Q4_K_M:
ssm_alpha: 24 × Q4_K
ssm_beta: 24 × Q4_K
By contrast, the 96 mHC tensors are correctly F32, and all 24 ssm_conv1d tensors are F32. So the precision intent is clearly there; the SSM gates look like a narrow omission rather than a general “quantize everything” choice.
Why this may be happening: in the upstream qwen35 path, only ssm_conv1d is special-cased to F32. There is no equivalent rule for ssm_alpha / ssm_beta, so unless the patched build adds an explicit rule, they land at Q4_K. A regex that matches blk.N.ssm_alpha.weight / blk.N.ssm_beta.weight and forces F32 should be enough.
The cost is tiny: 48 × ~0.43 MiB extra versus Q4_K = ~20.6 MiB on a 5.2 GB file. That is cheap enough that I would pin them even if the benefit were only suspected.
For context: in our own patched sensenova_loop builds, pinning these 48 gates to F32 (together with our allocator) produced a qualitative generation improvement, including under MTP; we are still isolating the exact contribution, but the direction and the price both point the same way. Your mHC result suggests you already found the same class of issue for routing — this looks like the same “small tensor, large behavioral effect” pattern.
Could you add explicit F32 rules for ssm_alpha and ssm_beta in the next quant revision? A small type dump per quant would also make this easy to verify.
- Metadata/layout: qwen35 + num_loops vs sensenova_loop
This is the part that puzzles me. Your Q4_K_M reports:
general.architecture = qwen35
tensor count = 523
qwen35.block_count = 32
qwen35.num_loops = 2
qwen35.loop_start = 4
qwen35.loop_end = 28
A sensenova_loop GGUF from our patched build reports:
general.architecture = sensenova_loop
tensor count = 538
sensenova_loop.block_count = 57
block names = blk.0 … blk.27, blk.52 … blk.56
Your README says the decoder stores 32 layers and executes layers 4–27 twice, and that MTP/vision are omitted. That explains the 15-tensor difference (our build includes the MTP block), but it does not explain the different architecture key or the different block numbering. Is qwen35 + num_loops the intended GGUF representation produced by your patch, or is it an artifact of the converter? Which runtime/build is expected to load it — only your patched engine, or any sensenova_loop-aware build?
Could you share the exact patched llama.cpp fork/PR (you mention base commit 5047748f11)? I would like to reproduce your path and compare the same graph rather than comparing two different tensor layouts. If the qwen35 metadata is intentional, that is fine — we just want to know whether the files are cross-compatible or engine-specific.
- Things I verified as correct
- 96 mHC tensors are F32.
- 24 ssm_conv1d tensors are F32.
- No vision tower / mmproj, as documented.
- No MTP block, as documented.
So this is not a complaint about the text-only scope; it is specifically about the SSM gates and the architecture/layout metadata
4. Small verification snippet
If useful, this is the check I used:
from collections import Counter
from gguf import GGUFReader
r = GGUFReader("SenseNova-Loop-9B-Preview-Q4_K_M.gguf")
gates = [t for t in r.tensors
if "ssm_alpha" in t.name or "ssm_beta" in t.name]
print(len(gates), Counter(t.tensor_type.name for t in gates))
48 Counter({'Q4_K': 48})
Happy to send the full tensor/type dump or test a revised quant thanks again for the release - the architecture notes made this much easier to audit