Instructions to use Accio-Lab/occamy-1.0-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Accio-Lab/occamy-1.0-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Accio-Lab/occamy-1.0-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Accio-Lab/occamy-1.0-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("Accio-Lab/occamy-1.0-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Accio-Lab/occamy-1.0-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Accio-Lab/occamy-1.0-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Accio-Lab/occamy-1.0-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Accio-Lab/occamy-1.0-NVFP4

SGLang

How to use Accio-Lab/occamy-1.0-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Accio-Lab/occamy-1.0-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Accio-Lab/occamy-1.0-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Accio-Lab/occamy-1.0-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Accio-Lab/occamy-1.0-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Accio-Lab/occamy-1.0-NVFP4 with Docker Model Runner:
```
docker model run hf.co/Accio-Lab/occamy-1.0-NVFP4
```

Occamy 1.0 NVFP4

Mixed-precision NVFP4 weights of Accio-Lab/occamy-1.0.

The routed MoE experts use NVFP4 weights and activations through NVIDIA Model Optimizer 0.44.0 NVFP4_EXPERTS_ONLY_CFG. Attention, routers, shared experts, embeddings, output head, and vision weights retain BF16. The original Occamy tokenizer and chat template are preserved. This checkpoint does not include a donor MTP head.

Calibration

128 records from the train split of CNN/DailyMail, configuration 3.0.0, truncated to 256 tokens. Sixteen separate test records were held out. The exact dataset revision and selection are in data-provenance.json; the quantization configuration is in quantization-recipe.json. No training or fine-tuning was performed.

For expert weights whose observers were not activated, ModelOpt used weight-derived or fused-tensor scaling during export. This limited calibration set does not establish coverage of every expert on every workload. The runtime checks below test the exported checkpoint.

Evaluation

Check	BF16	NVFP4
Text requests ending normally	8/8	8/8
Strict JSON fixtures	3/4	3/4
Tool-call JSON fixtures	2/2	2/2
Structured tool calls through the API	2/2	2/2
Local add-tool execution and result follow-up	True	True
Generated code with independent tests	2/2	2/2
Held-out token NLL	2.252837	2.262056
Synthetic red-image recognition	pass	pass

Full scalar results are in VALIDATION.json. These are small regression checks, not general benchmark results. The add tool was executed locally and its result returned to the model; the weather case validates the structured call only. Code fences were removed for code execution; JSON checks require raw JSON. No broad vision, audio, video, long-context, or throughput claim is made. The validation context length was 2048.

Tested serving configuration

SGLang 0.5.13.post1, FlashInfer 0.6.12, PyTorch 2.11.0+cu130. Native NVFP4 execution requires compatible hardware and kernels.

python -m sglang.launch_server \
  --model-path Accio-Lab/occamy-1.0-NVFP4 \
  --quantization modelopt_fp4 --dtype bfloat16 \
  --moe-runner-backend flashinfer_cutlass \
  --attention-backend triton --mamba-backend triton \
  --mm-attention-backend sdpa \
  --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
  --context-length 2048 --max-total-tokens 4096 \
  --max-running-requests 1 --max-mamba-cache-size 4 \
  --disable-cuda-graph --disable-radix-cache --disable-overlap-schedule

The tests used greedy decoding and chat_template_kwargs={"enable_thinking": false}.

vLLM compatibility

On a single H200, this checkpoint completed all 9 requests and passed all 9 semantic checks, covering text, code, strict JSON, tool calls and result follow-up, and images. The BF16 reference also passed 9/9. There were no request errors or truncated responses. This used the Marlin W4A16 path, not native FP4 compute. MTP was disabled. This is a small compatibility check, not a broad quality or speed benchmark.

Attribution and license

Original model: Accio-Lab, based on Qwen3.6-35B-A3B. Original Apache-2.0 license is included.

See Technical details for reproduction settings.

Downloads last month: -

Safetensors

Model size

19B params

Tensor type

BF16

Model tree for Accio-Lab/occamy-1.0-NVFP4

Base model

Qwen/Qwen3.6-35B-A3B

Finetuned

Accio-Lab/occamy-1.0

Quantized

(10)

this model