Instructions to use NeosQ/SenseNova-Loop-9B-MLX-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use NeosQ/SenseNova-Loop-9B-MLX-4bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("NeosQ/SenseNova-Loop-9B-MLX-4bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use NeosQ/SenseNova-Loop-9B-MLX-4bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "NeosQ/SenseNova-Loop-9B-MLX-4bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "NeosQ/SenseNova-Loop-9B-MLX-4bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use NeosQ/SenseNova-Loop-9B-MLX-4bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "NeosQ/SenseNova-Loop-9B-MLX-4bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "NeosQ/SenseNova-Loop-9B-MLX-4bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NeosQ/SenseNova-Loop-9B-MLX-4bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use NeosQ/SenseNova-Loop-9B-MLX-4bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "NeosQ/SenseNova-Loop-9B-MLX-4bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default NeosQ/SenseNova-Loop-9B-MLX-4bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use NeosQ/SenseNova-Loop-9B-MLX-4bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "NeosQ/SenseNova-Loop-9B-MLX-4bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "NeosQ/SenseNova-Loop-9B-MLX-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
SenseNova-Loop-9B — Unofficial MLX 4-bit
This is an unofficial, text-only MLX conversion of sensenova/SenseNova-Loop-9B, quantized to 4-bit weights. It is not an official SenseNova release.
Important runtime requirement
SenseNova-Loop-9B uses a custom looped, dual-stream mHC architecture. Stock mlx-lm does not support this architecture. These weights require an MLX/oMLX runtime with a compatible SenseNova_loop implementation.
The customized implementation used in our tests is included as mlx_sensenova_loop.py for integration and reference. Merely downloading this repository into a stock mlx-lm installation is not sufficient.
The conversion contains text-generation weights only; vision inputs are not supported by this package.
Quantization
- Tool:
mlx_lm.convert - Method: MLX affine weight quantization
- Bits: 4
- Group size: 64
- Effective size reported by the converter: 4.511 bits per weight, including quantization metadata
- Source: BF16 Hugging Face weights
- Custom runtime flags used in testing:
SENSENOVA_MHC_METAL=1andSENSENOVA_MHC_MERGE_METAL=1
Recommended generation settings
The source repository's default generation configuration is temperature=1.0, top_p=0.95, with sampling enabled. In our Chinese long-form tests, that setting could produce irrelevant foreign words in all tested precisions, including unquantized BF16.
For formal Chinese long-form generation, we recommend:
{
"temperature": 0,
"top_p": 1
}
This recommendation targets language artifacts; it does not guarantee factual correctness. Reasoning may also be in English while the final answer is entirely Chinese.
Speed on the test machine
Test environment: Apple Silicon Mac with 128 GB unified memory, oMLX 0.7.3, customized mlx-lm, MLX 0.32.4. Each result is the median of three OpenAI-compatible requests generating 64 tokens. Results depend on hardware, prompt, runtime, and workload.
| Variant | Generation speed |
|---|---|
| BF16 baseline | 8.50 token/s |
| 8-bit | 13.51 token/s |
| 4-bit | 20.90 token/s |
For this 4-bit variant, enabling compiled mHC router subgraphs increased speed from 16.44 to 20.90 token/s (about 27.1% faster). The compilation optimization passed 32-token output and logit-equivalence checks. That equivalence applies to compilation, not to quantization.
Quantization quality checks
Evaluation covered 614 token positions across Chinese, English, Python, and math text. Lower perplexity is better; top-1 agreement is measured against BF16.
| Variant | Perplexity | Top-1 agreement with BF16 |
|---|---|---|
| BF16 | 2.7127 | reference |
| 8-bit | 2.6938 | 99.02% |
| 4-bit | 2.7875 | 93.81% |
Small differences in perplexity can arise from evaluation variation; the table should not be interpreted as proof that 8-bit improves the base model.
Chinese long-form language tests
Official default sampling (temperature=1.0, top_p=0.95)
Three fixed seeds were run for each precision. All nine essays finished naturally. Every matched foreign word below was manually reviewed and was inappropriate in context.
| Variant | Foreign words across 3 essays | Foreign words per 1,000 Chinese characters |
|---|---|---|
| BF16 | 7 | 1.21 |
| 8-bit | 23 | 3.32 |
| 4-bit | 18 | 2.76 |
This small sample shows that the artifact exists in the base model too. Quantization may aggravate it, but the results do not show a monotonic relationship with quantization depth.
Deterministic decoding (temperature=0, top_p=1)
This 4-bit variant was tested on five different approximately 2,000-character Chinese tasks: two book reflections, one argumentative essay, one literary essay, and one popular-science article.
- 5/5 articles finished naturally.
- Total output: 12,109 characters, including 10,034 Chinese characters.
- No meaningless foreign words, scripts, or Unicode replacement characters were found.
- Contextually appropriate terms such as
ChatGPT,AI, andREMwere not counted as errors.
Five tasks do not prove that artifacts are impossible under all prompts.
Optional MTP speculative draft
This repository includes model-mtp.safetensors, an optional MLX BF16 conversion of the official MTP draft weights. The main model remains 4-bit; the draft is deliberately kept in BF16 because it is a small file and quantizing it did not address the measured bottleneck.
MTP is disabled by default and requires SENSENOVA_MTP=1 with the customized runtime. The included mlx_mtp_speculative.py provides the tested single-request runner. The current oMLX generic MTP toggle does not recognize this custom architecture.
Use depth 2. Deeper drafts did not increase useful acceptance and became slower; depths above 3 are not recommended.
SENSENOVA_MTP=1 python mlx_mtp_speculative.py \
--model /path/to/SenseNova-Loop-9B-MLX-4bit \
--prompt "Your prompt" --depth 2
Add --baseline to run the same prompt without speculative decoding.
Three-run medians for greedy code generation with a 4-bit target:
| Task | Normal decoding | MTP depth 2 | Output match |
|---|---|---|---|
| LRU cache implementation | 36.00 token/s | 39.43 token/s (about 9.6% faster) | exact in 3/3 runs |
| Merge intervals | 33.34 token/s | 31.88 token/s (4.4% slower) | exact in 3/3 runs |
Long Chinese prose and several other tasks were slower with MTP. This is an optional, workload-dependent code-generation experiment, not a general speed setting.
Limitations
- Quantization is not lossless, especially at 4-bit.
- Deterministic decoding does not prevent factual errors, fabricated quotations, or invented personal experiences. Review factual content before use.
- The language tests targeted foreign-token artifacts, not overall writing quality or factual accuracy.
- A customized runtime is required; stock mlx-lm is insufficient.
Download
hf download NeosQ/SenseNova-Loop-9B-MLX-4bit \
--local-dir SenseNova-Loop-9B-MLX-4bit
License and attribution
Apache-2.0. Please also follow the base model's license and terms. All quantization and testing described here are unofficial community work.
- Downloads last month
- 76
4-bit
Model tree for NeosQ/SenseNova-Loop-9B-MLX-4bit
Base model
sensenova/SenseNova-Loop-9B