Instructions to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="rodrigoramosrs/veriloop-coder-e2-nvfp4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("rodrigoramosrs/veriloop-coder-e2-nvfp4")
model = AutoModelForCausalLM.from_pretrained("rodrigoramosrs/veriloop-coder-e2-nvfp4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rodrigoramosrs/veriloop-coder-e2-nvfp4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rodrigoramosrs/veriloop-coder-e2-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/rodrigoramosrs/veriloop-coder-e2-nvfp4

SGLang

How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rodrigoramosrs/veriloop-coder-e2-nvfp4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rodrigoramosrs/veriloop-coder-e2-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rodrigoramosrs/veriloop-coder-e2-nvfp4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rodrigoramosrs/veriloop-coder-e2-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with Docker Model Runner:
```
docker model run hf.co/rodrigoramosrs/veriloop-coder-e2-nvfp4
```

VeriLoop Coder E2 · NVFP4

Coding-Optimized Quantized Model (NVIDIA NVFP4)

Original Model ↗ · GitHub · Apache-2.0

Overview

This repository contains an NVFP4 quantization of VeriLoop E2, an open 27B post-trained model built on Qwen3.8-27B for code, mathematics, and physics. Its core reasoning discipline is VeriLoop-Governed Recurrence (VGR): candidate states are recursively proposed, externally checked, and retained only when the protected evidence state improves without regression.

Quantized by Rodrigo Ramos.

Quantization Approach

Produced with NVIDIA Model Optimizer using the canonical NVFP4_W4A4_WEIGHT_LOCAL_HESSIAN_CFG recipe: static per-block 4-bit weights (group size 16) + dynamic 4-bit activations, FP8 attention, local-Hessian calibration with MSE and fp8 scale sweep.

The linear_attn (Gated Delta Net) blocks and the self-attention projections ship in BF16 on purpose — matching validated official NVFP4 releases for this architecture family (which exclude linear_attn* per layer) and the widely-deployed MLP-only NVFP4 pattern. Only the MLP blocks (gate/up/down) are NVFP4. Embeddings, lm_head and small projections stay BF16 per the recipe.

Calibration ran on 512 samples × 512 tokens (262144 tokens) of competitive-programming code from nvidia/Nemotron-Competitive-Programming-v1, keeping quantization faithful on the distributions that matter most for coding tasks.

The result retains the original model's strong software-engineering capabilities at roughly half of the BF16 footprint, ready for NVFP4-capable inference stacks on Blackwell GPUs.

Files

File	Size	Notes
`model-00001-of-00008.safetensors` … `model-00008-of-00008.safetensors`	~29.2 GB total	NVFP4 MLP weights (packed U8 + FP8 block scales) + BF16 attention/GDN/embeddings/head
`model.safetensors.index.json`	—	Shard index (1427 tensors)
`hf_quant_config.json`	—	`quant_algo: NVFP4`, group size 16
`config.json`, `tokenizer.*`, `merges.txt`, `vocab.json`	—	Config + tokenizer from the base model
`scripts/quantize_veriloop.py`	—	Full reproduction script (multi-GPU PTQ)
`scripts/validate_nvfp4.py`	—	Structural shard/index validator

Usage

This checkpoint targets NVFP4-capable runtimes (Blackwell GPUs). Plain transformers cannot load modelopt NVFP4 checkpoints directly.

Compatibility note: per the NVIDIA recipe, the linear_attn blocks, the self-attention projections, a small set of projections (linear_attn.in_proj_a/b, conv1d), plus the embeddings and lm_head, intentionally ship in BF16. The exclude_modules list in hf_quant_config.json (mirrored in config.json, including fused names such as in_proj_ba/in_proj_qkvz that some loaders synthesize) declares exactly which modules those are — inference loaders must honor it and serve those modules unquantized. If a loader errors with "weight is torch.bfloat16 but the quant config declares NVFP4", it is ignoring the exclusion list, not a checkpoint defect.

vLLM

pip install 'vllm>=0.17.0'

vllm serve rodrigoramosrs/veriloop-coder-e2-nvfp4 \
    --tensor-parallel-size 1 \
    --max-model-len 131072 \
    --kv-cache-dtype fp8_e4m3 \
    --gpu-memory-utilization 0.92

SGLang

pip install 'sglang[all]'

python -m sglang.launch_server \
    --model-path rodrigoramosrs/veriloop-coder-e2-nvfp4 \
    --trust-remote-code \
    --mem-fraction-static 0.88

TensorRT-LLM

Build the engine from this checkpoint with trtllm-build using an NVFP4 + FP8-KV configuration, then serve with trtllm-serve. See the TensorRT-LLM documentation for the exact flags for your GPU.

Reproducing

pip install "nvidia-modelopt[all]" torch transformers accelerate datasets safetensors

# needs ~55 GB VRAM across CUDA GPUs for the 27B BF16 source model
python scripts/quantize_veriloop.py \
    --model ./model-bf16 \
    --output ./model-nvfp4 \
    --calib-size 512 \
    --calib-seq-len 512

python scripts/validate_nvfp4.py ./model-nvfp4

On Windows the Triton JIT needs a C compiler: run from a Visual Studio Native Tools prompt (vcvars64) with CC pointing at cl.exe. See the header of scripts/quantize_veriloop.py for multi-GPU layout tuning (--layers-split, --gpu-order).

Runtime notes

Verified working: FreeToken (NVFP4, Blackwell) — end-to-end generation confirmed.
Known limitation (SGLang ≤ 0.5.20): serving this checkpoint yields degenerate output (constant-token repetition). Bisection points at SGLang's Gated-Delta-Net path for the dense-27B head layout (48 value heads / 16 key heads, ratio 3): the same SGLang build serves Llama-3.1-8B-NVFP4 and Qwen3.5-9B (ratio 2, BF16 and NVFP4-MLP) correctly, across PP/TP layouts, KV dtypes, attention/sampling/GEMM backends and with the SiLU fusion disabled. Affects any dense-27B Qwen3.5 regardless of which blocks are quantized. vLLM / TensorRT-LLM untested with this checkpoint.

Acknowledgements

Libo Wang and the Intelligent Robotics Laboratory, Tsinghua SIGS for developing the original VeriLoop E2 model.
NVIDIA for Model Optimizer and the NVFP4 quantization tooling.
The original model repository: tsinghua-sigs-robot-lab/VeriLoop-E2

License

Apache-2.0. The weights are quantized from the original Apache-2.0 licensed model. See the original repository for full licensing details and third-party notices.

Downloads last month: 585

Safetensors

Model size

18B params

Tensor type

BF16

Model tree for rodrigoramosrs/veriloop-coder-e2-nvfp4

Base model

Qwen/Qwen3.8-27B

Finetuned

tsinghua-sigs-robot-lab/VeriLoop-E2

Quantized

(4)

this model