Instructions to use gitcommit90/SenseNova-Loop-9B-Preview-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use gitcommit90/SenseNova-Loop-9B-Preview-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="gitcommit90/SenseNova-Loop-9B-Preview-NVFP4", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("gitcommit90/SenseNova-Loop-9B-Preview-NVFP4", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use gitcommit90/SenseNova-Loop-9B-Preview-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "gitcommit90/SenseNova-Loop-9B-Preview-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gitcommit90/SenseNova-Loop-9B-Preview-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/gitcommit90/SenseNova-Loop-9B-Preview-NVFP4

SGLang

How to use gitcommit90/SenseNova-Loop-9B-Preview-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "gitcommit90/SenseNova-Loop-9B-Preview-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gitcommit90/SenseNova-Loop-9B-Preview-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "gitcommit90/SenseNova-Loop-9B-Preview-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gitcommit90/SenseNova-Loop-9B-Preview-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use gitcommit90/SenseNova-Loop-9B-Preview-NVFP4 with Docker Model Runner:
```
docker model run hf.co/gitcommit90/SenseNova-Loop-9B-Preview-NVFP4
```

SenseNova-Loop-9B-Preview-NVFP4

Unofficial community quantization. The original model, architecture and weights are by SenseNova: sensenova/SenseNova-Loop-9B-Preview (revision 27abea779b1e6634b136805b6519371739d551f7), licensed Apache-2.0. We did not train this model. This repository contains modified weights (post-training quantization only — no fine-tuning, QAT or distillation).

A calibrated, packed NVFP4 (ModelOpt) checkpoint of SenseNova-Loop-9B-Preview with the MTP head kept BF16 so MTP speculative decoding works, validated on an NVIDIA DGX Spark (GB10, SM121) with native W4A4 FP4 GEMMs.

Downloads and companion source

Weights and complete model files: https://huggingface.co/gitcommit90/SenseNova-Loop-9B-Preview-NVFP4
Runtime, quantization recipe, evaluation harness and charts: https://github.com/gitcommit90/sensenova-loop-9b-nvfp4-dgx-spark

Updated October 31: runtime v2 is included below, alongside the preserved October 30 functional-release baseline. Weights live on Hugging Face; the GitHub companion does not duplicate the 9.38 GB checkpoint.

Runtime v2 — final optimization results (October 31)

Same 9.38 GB NVFP4 weights; new runtime kernels and a smaller MTP draft vocabulary. No retraining, recalibration, reduced loop passes, or changed KV/state precision. MTP is supported in all profiles. The target still verifies over the full vocabulary. Only greedy decoding was evaluated.

Final matched measurement	Frozen NVFP4 + MTP	v2 exact + MTP	v2 fast + MTP
Single-request decode, ~1K input (tok/s, median)	12.4	27.3	28.9
Single-request code decode, ~90 input (tok/s, median)	16.0	33.4	35.9
Single-request document decode, ~4K input (tok/s, median)	14.3	30.6	33.3
Eight-request aggregate, ~1K input (tok/s)	64.3	117.2	119.4
Eight-request code aggregate (tok/s)	97.4	172.7	196.7
Single-request TTFT, ~1K input (seconds, median)	0.74	0.51	0.32

Exactly 256 output tokens; unique prompts; no prefix-cache hits; zero errors in these final performance runs. Aggregate throughput is combined across requests, not each user's speed. These are the final matched runs, not the higher preliminary screening peaks. Full sample counts, token lengths and raw measurements: runtime-v2/RESULTS.md and runtime-v2/runs/.

Which profile should I use?

MHC_MODE=exact — recommended conservative profile: fused Sinkhorn with reference operation order; cuBLAS routing projection and reference merge retained. Reference routing math matched bitwise in differential tests; the 12-case final feature check matched the frozen output. This is not a universal promise of identical generations under every batch/hardware configuration.
MHC_MODE=fast — faster, numerically different: also fuses routing projection/merge. FP32 is preserved, but summation order changes and greedy text can diverge. Choose this explicitly if that trade-off suits you. The supplied v2 launcher's inherited default is fast; commands below explicitly select exact.
runtime/ — frozen v1: preserved for reproduction and rollback. Both v2 profiles use the same pinned image.

Quality: report the losses and uncertainty, not just the speedup

On the same 300-item suite, frozen v1 scored 251/300 (83.7%), v2 exact 249/300 (83.0%), and v2 fast 247/300 (82.3%). Paired-bootstrap 95% delta intervals vs frozen: exact −0.67 pp [−3.00, +1.67]; fast −1.33 pp [−4.33, +1.67]. These small, noisy subsets do not establish losslessness or full-benchmark equivalence.

Fast's first MMLU-Pro run fell from 70.0% to 58.3% (−11.7 pp) and truncations rose from 14 to 20 across the suite. Those exceed the original first-run per-task and ref+3 truncation tripwires. Fast MMLU-Pro reruns scored 71.7% and 65.0%; frozen reruns scored 70.0% and 65.0%. All runs are retained. Variability and the token budget complicate interpretation; they do not erase the first-run regression. See RESULTS.md.

The improvements come from fused FP32 mHC/Sinkhorn kernels and a 65,536-token MTP draft shortlist selected from training-split corpora. The draft is restricted; the verifying target is not. The exact kernel preserves all 20 configured normalization iterations. No learned model weights changed.

Source, numerical tests, optimized launcher, token-map builder, and experiment ledger are in runtime-v2/. Numerical tests and final feature/quality/performance results are included. Figures follow figures4papers, with figures/plot_runtime_v2.py, PNG previews and vector PDF/SVG exports.

Original quantization baseline (October 30; retained)

These older figures compare BF16 to the original v1 NVFP4 runtime, not runtime v2. Their source script remains figures/plot_benchmarks.py.

Provenance

Item	Value
Base model	`sensenova/SenseNova-Loop-9B-Preview` @ `27abea779b1e6634b136805b6519371739d551f7` (weights stored FP32 upstream; loaded as BF16 backbone + FP32 mHC, as the upstream card prescribes)
Quantization tool	NVIDIA Model Optimizer `nvidia-modelopt==0.48.0rc1` (`NVFP4_DEFAULT_CFG`, `max` calibration), PyTorch 2.11.0+cu130, Transformers 5.18.0
Format	NVFP4: E2M1 weights, group size 16, FP8-E4M3 block scales, FP32 per-tensor global scale (`weight_scale_2`) and static activation global scale (`input_scale`); activations quantized to NVFP4 at runtime (W4A4); KV cache not quantized (BF16)
Export	2026-10-30, `export_hf_checkpoint`, single `model.safetensors` (9.38 GB)
Hardware	NVIDIA DGX Spark: GB10 (SM121), 128 GB unified memory, driver 580.159.03, CUDA 13.0

Changes from the original (Apache-2.0 §4(b) notice)

Decoder linear-layer weights quantized to NVFP4 (see exclusions); config.json gains quantization_config; hf_quant_config.json added. Upstream FP32-stored tensors that were not quantized are stored as BF16, except the mHC routing parameters, which remain FP32.
mtp.* (MTP head) is included in model.safetensors as BF16 (upstream shipped it as a separate FP32 mtp.safetensors).
Tokenizer, processor configs, chat template, generation config and modeling_sensenova_loop.py are byte-identical to the pinned upstream revision (see SHA256SUMS).
runtime-v2/ adds fused routing kernels and a 64K draft token map, with no checkpoint weight changes.
runtime/ contains SGLang source files from the open SGLang PR #47281 (Apache-2.0, sgl-project) plus two small patches of ours (runtime/patches/), needed to serve this checkpoint.

Architecture-specific handling

Looped block (layers 4–27 run twice, shared weights): calibration ran through the real two-pass forward; every looped layer was observed on both visits for every sample, so each shared quantizer's amax is the max over both visits. One physical copy of each layer is stored (32 layers, not 56); per-visit KV/recurrent state remains independent in the runtime.
mHC routers (mhc_attn, mhc_ffn): FP32 parameters and FP32 routing math (Sinkhorn, 20 iterations), untouched.
No scale folding / smoothing / graph transforms were applied (plain max calibration), so nothing assumes a single visit.

Precision exclusions

Module group	Kept in	Reason
Vision encoder + merger (`model.visual*`)	BF16	fragile, small share of compute
`lm_head`, `embed_tokens`	BF16	output/embedding sensitivity
mHC routing (`mhc_attn`, `mhc_ffn`, 96 tensors)	FP32	required by the architecture
Gated-DeltaNet `in_proj_a`, `in_proj_b`, `conv1d`, `A_log`, `dt_bias`, norms	BF16 / FP32	recurrent-state gates, tiny
All RMSNorms	BF16	not linear
MTP head (`mtp.*`, 15 tensors)	BF16	draft head; keeps MTP acceptance
Quantized (NVFP4): full-attention `q/k/v/o_proj`, GDN `in_proj_qkv`, `in_proj_z`, `out_proj`, MLP `gate/up/down_proj` in all 32 layers (200 linears)	NVFP4

Parameters quantized: 73.4 % of the backbone loaded by Transformers (6.91 B of 9.41 B), 71.6 % including the MTP head. Checkpoint size: 9.38 GB (upstream: 38.1 GB FP32 on disk ≈ 19.3 GB as BF16). Weight memory in SGLang: 8.8 GB (vs 17.5 GiB BF16 weights in Transformers).

Calibration data

256 chat-formatted examples (calib_v1, sha256 9a4d2112ef71820a…, see quantization/calib_v1.jsonl.sha256), rendered with the model's own chat template/processor: 98,458 tokens incl. 39 images. Calibration data is not redistributed.

Share	Source	Split / revision	Licence
30 % chat (en/zh/es/ca)	OpenAssistant/oasst2	train @ 179dd21f	Apache-2.0
20 % code	google-research-datasets/mbpp	full/train @ 4bb6404f	CC-BY-4.0
20 % math (half with `<think>` traces)	openai/gsm8k	main/train @ 740312ad	MIT
15 % tool-calling trajectories	glaiveai/glaive-function-calling-v2	@ e7f4b645	Apache-2.0
15 % image QA	lmms-lab/textvqa	train shard 0 @ 9c0699cd	CC-BY-4.0 (images: OpenImages, CC-BY-2.0)

Disjoint from all evaluation items (different splits/datasets; 13-gram overlap check found only shared tool-schema boilerplate).

Quality vs BF16 reference (same SGLang build and flags, greedy, thinking off)

Small fixed suite (300 items; data hashes in evaluation/eval_data_MANIFEST.json). Gates were declared before any NVFP4 result was seen (evaluation/QUALITY_GATES.md). Paired-bootstrap 95 % CI for the delta. These are subsets, not full-benchmark scores; differences inside the CI are "not detectable at this n", not "lossless".

Task	n	BF16	NVFP4	Δ (pp) [95 % CI]
IFEval subset (prompt-level strict, re-implemented checker)	60	78.3	75.0	−3.3 [−13.3, +6.7]
GSM8K test subset	80	92.5	95.0	+2.5 [−2.5, +7.5]
MMLU-Pro test subset	60	65.0	68.3	+3.3 [−6.7, +13.3]
HumanEval subset (pass@1)	40	82.5	80.0	−2.5 [−7.5, 0.0]
Function calling (20 hand-written BFCL-style cases)	20	95.0	95.0	0.0 [−15, +15]
TextVQA val subset	40	90.0	90.0	0.0 [−7.5, +7.5]
All (micro)	300	82.7	83.0	+0.3 [−3.0, +4.0]

Tool calls: 100 % well-formed for both. 0 request errors. Truncations at the max-token limit: 14 (NVFP4) vs 19 (BF16). With MTP enabled the NVFP4 model scored GSM8K 96.3 % (n=80), HumanEval 80.0 % (n=40), tools 90 % (n=20); MTP is lossless in principle at temperature 0, small differences come from batch-dependent numerics.

Original v1 performance (DGX Spark GB10, SGLang, same flags for all rows)

Prompt 1,051 tokens, exactly 256 output tokens (ignore_eos), greedy, unique prompts (no prefix-cache hits), --mem-fraction-static 0.45, CUDA graphs on. No MTP rows: 16 requests per level; MTP rows: 8 requests per level. Aggregate output tokens/s (per-request decode tok/s p50 in parentheses):

Concurrency	BF16	NVFP4 W4A4	BF16 + MTP	NVFP4 W4A4 + MTP	NVFP4 W4A16 (Marlin)
1	5.75 (5.8)	9.39 (9.6)	7.51 (7.6)	12.21 (12.6)	10.98 (11.4)
2	11.44	17.61	14.77	24.79	–
4	21.86	33.48	26.81	42.51	–
8	39.93 (5.5)	64.79 (9.3)	45.70 (7.0)	64.43 (12.2)	58.17 (8.8)

TTFT p50 at c=1: 0.75 s BF16 → 0.58 s NVFP4. 0 errors / OOMs at any level. MTP mean accepted length (3 draft steps): 2.93 NVFP4 vs 2.95 BF16 on the eval tasks; on the synthetic perf prompts ≈2.0, which is why MTP gives no extra aggregate throughput at c=8 there. KV-cache capacity at the same memory budget: 407 k tokens (NVFP4) vs 308 k (BF16).

Kernel evidence (native FP4): a torch-profiler trace of NVFP4 serving shows the quantized linears executing cutlass3x_sm120_bstensorop_…gemm_block_scaled_ue4m3xe2m1_ue4m3xe2m1… (block-scaled FP4×FP4 tensor-core GEMM, dispatched via the FlashInfer cuDNN backend) with activations quantized by quantize_with_block_size — i.e. true W4A4, not dequantize-to-BF16 (evaluation/kernel_summary.txt). The W4A16 column uses the same checkpoint with Marlin weight-only kernels; it is faster for single-stream decode on GB10 but slower under concurrency.

Hardware & software requirements

NVIDIA Blackwell GPU with FP4 tensor cores. Tested only on DGX Spark (GB10, SM121). Weights need ≈9 GB plus KV/recurrent-state memory.
Serving: SGLang image lmsysorg/sglang:nightly-dev-cu13-20261029-c10e0e17 (sha256:78142f9f4b7fb3ee27f59065615dc7bb263f643ed256c9299c3861086ade4d47) with the files in runtime/overlay (SGLang PR #47281 @ 336c54ac1e721197806d94b6929f567446a59693 + our patches) mounted over the image. As of 2026-10-30 we found no engine supporting sensenova_loop + NVFP4 out of the box (checked: SGLang main, LightLLM main — model supported but no NVFP4, vLLM 0.27.1 — no sensenova_loop).
Plain Transformers (trust_remote_code=True) loads the original BF16 model but cannot execute this packed NVFP4 checkpoint or MTP; use the SGLang runtime above.

Usage

Docker and the NVIDIA container runtime are required. Use one of the following paths.

From the GitHub companion (weights are downloaded once to an ignored local directory):

git clone https://github.com/gitcommit90/sensenova-loop-9b-nvfp4-dgx-spark.git
cd sensenova-loop-9b-nvfp4-dgx-spark
python3 -m venv .venv-download
.venv-download/bin/pip install 'huggingface_hub>=1,<2'
.venv-download/bin/hf download gitcommit90/SenseNova-Loop-9B-Preview-NVFP4 --local-dir models/SenseNova-Loop-9B-Preview-NVFP4
# Verify the downloaded release before serving:
(cd models/SenseNova-Loop-9B-Preview-NVFP4 && sha256sum -c SHA256SUMS)
MHC_MODE=exact bash runtime-v2/serve_sglang.sh models/SenseNova-Loop-9B-Preview-NVFP4

From a complete Hugging Face snapshot containing model.safetensors:

# from the snapshot root
MHC_MODE=exact bash runtime-v2/serve_sglang.sh . # conservative optimized W4A4 + MTP, 127.0.0.1:30080
curl -s http://127.0.0.1:30080/v1/models
python3 runtime/probe.py http://127.0.0.1:30080/v1 sensenova-loop-9b-nvfp4 [image.jpg]   # text / thinking / tool / image smoke test
# options: --no-mtp ; W4A16: SGLANG_DISABLE_SILU_FP4_QUANT_FUSION=1 runtime/serve_sglang.sh . --fp4-gemm-backend marlin

OpenAI-compatible API; thinking is toggled with chat_template_kwargs: {"enable_thinking": false|true}; tools use the qwen3_coder parser (the model's XML <tool_call><function=…> format).

Profile selection and rollback

MHC_MODE=fast bash runtime-v2/serve_sglang.sh /path/to/model  # fastest; see numerical/quality caveats
MHC_MODE=exact bash runtime-v2/serve_sglang.sh /path/to/model # conservative optimized
bash runtime/serve_sglang.sh /path/to/model                 # frozen v1 rollback

The launcher replaces only its named model container. No extra quantization step is required. Benchmark reproduction is documented in runtime-v2/bench/README.md.

Limitations

Upstream is a Preview model; all upstream limitations and intended-use terms apply.
The serving path depends on an unmerged SGLang PR plus our patches, pinned to one nightly image; it was tested only on GB10. Pipeline/tensor parallelism and other GPUs were not tested.
Evaluation is small (300 items, thinking off, contexts ≤ ~3 K tokens). Long context (> 32 K), video input, thinking-mode quality at scale and non-English quality were not evaluated. Only greedy decoding was benchmarked.
The frozen v1 mHC path uses many small FP32 kernels. Runtime v2 fuses those operations; the remaining profile is more GEMM/output-head bound.
Runtime v2 fast is not bitwise equivalent to v1 and did not pass every original first-run quality tripwire. Exact is the conservative option.
The draft vocabulary shortlist was built from training corpora; no broad sampling-equivalence or full-context quality study was performed.
In this SGLang build, --fp4-gemm-backend marlin crashes unless SGLANG_DISABLE_SILU_FP4_QUANT_FUSION=1 is set (upstream bug: the fused SiLU+FP4-quant path is enabled regardless of backend).
Quantization may change behaviour on inputs unlike the calibration mixture.

Files

model.safetensors, config.json, hf_quant_config.json — NVFP4 checkpoint (+ BF16 MTP head, FP32 mHC).
Upstream (unchanged): tokenizer*.json, chat_template.jinja, *processor_config.json, generation_config.json, modeling_sensenova_loop.py.
runtime-v2/ — optimized profiles, Triton source, draft token map, final results and differential tests.
runtime/ — original tested launcher, SGLang overlay and patches, probe script, compatibility notes.
quantization/ — quantization/verification scripts, stats, structural verification report, environment lock, calibration manifest.
evaluation/ — quality gates, eval summaries, paired comparison, perf JSONs, kernel trace summary, reference checks.
SHA256SUMS — checksums of every file.

Licence

Apache-2.0, inherited from the base model (LICENSE, verbatim from OpenSenseNova/SenseNova-Loop @ a521b0af; the upstream repository has no NOTICE file). SGLang files in runtime/overlay are Apache-2.0 (sgl-project).

Citation

Please cite the original model:

@misc{sensenova2026loop,
  title        = {{SenseNova-Loop-9B-Preview}},
  author       = {{SenseNova}},
  year         = {2026},
  howpublished = {Hugging Face model repository},
  url          = {https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview}
}

Downloads last month: 1

Safetensors

Model size

6B params

Tensor type

F32

BF16

Model tree for gitcommit90/SenseNova-Loop-9B-Preview-NVFP4

Base model

sensenova/SenseNova-Loop-9B-Preview

Quantized

(7)

this model