Instructions to use Accio-Lab/occamy-1.0-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Accio-Lab/occamy-1.0-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Accio-Lab/occamy-1.0-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Accio-Lab/occamy-1.0-FP8")
model = AutoModelForMultimodalLM.from_pretrained("Accio-Lab/occamy-1.0-FP8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Accio-Lab/occamy-1.0-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Accio-Lab/occamy-1.0-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Accio-Lab/occamy-1.0-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Accio-Lab/occamy-1.0-FP8

SGLang

How to use Accio-Lab/occamy-1.0-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Accio-Lab/occamy-1.0-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Accio-Lab/occamy-1.0-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Accio-Lab/occamy-1.0-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Accio-Lab/occamy-1.0-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Accio-Lab/occamy-1.0-FP8 with Docker Model Runner:
```
docker model run hf.co/Accio-Lab/occamy-1.0-FP8
```

Occamy 1.0 FP8

Blockwise FP8 conversion of Accio-Lab/occamy-1.0, source revision c1ce84770260c4137712cf22115574c4d06b993a. All weights come from Occamy; no donor weights or MTP head are included.

In the paired evaluation below, FP8 increased token NLL by 0.3378%. Sampled mathematics accuracy was 62/64 versus 63/64 for BF16; science, code, JSON, multi-turn, long-input and tool-roundtrip scores matched the baseline. See the complete comparison below and the validation notes for per-case details.

Format and integrity

FP8 E4M3FN weights, 128 × 128 blocks, FP32 inverse scales, dynamic activation quantization.
30,970 weight tensors converted; 696 tensors preserved exactly, including vision weights. Source tokenizer, normalizer, processor and chat template are byte-identical.
14 safetensors shards, 36,613,923,840 bytes total. File size is not a runtime VRAM requirement.
Conversion: Transformers 5.8.1 Fp8Quantize, PyTorch 2.11.0+cu130. No calibration data or training was used.
Tensor selection follows the format metadata of Qwen/Qwen3.6-35B-A3B-FP8, revision 95a723d08a9490559dae23d0cff1d9466213d989. None of its weights are included.

CONVERSION.json, STRUCTURE-VALIDATION.json and SHA256SUMS record provenance, checksums and shape/type/preserved-weight checks. Use the included Hugging Face tokenizer; it retains the original NFC normalizer.

Expanded paired evaluation

SGLang 0.5.13.post1, source commit 85fd90072d1a9f2432842b03588f63b745e524e4, PyTorch 2.11.0+cu130, FlashInfer 0.6.12. Identical prompts, greedy decoding, seed 42, thinking disabled, four concurrent requests, 16K configured context. Dataset revisions and fixed sample indices are in evaluation/provenance.json; fixtures and raw scored responses are included.

Check	BF16	FP8
GSM8K test sample	63/64	62/64
ARC-Challenge test sample, answer accuracy	59/64	59/64
Python tasks with independent functional tests	12/12	12/12
Strict JSON fixtures	20/20	20/20
Multi-turn memory fixtures	8/8	8/8
Long-input retrieval fixtures	6/6	6/6
Synthetic color/OCR/counting images	11/12	10/12
Native add-tool call, local execution and result follow-up	6/6	6/6
Request errors / truncated primary responses	0 / 0	0 / 0
WikiText test-sample token NLL, 4,080 tokens	2.206196	2.213649

FP8 NLL increased 0.3378%; the perplexity ratio is 1.00748. All scored log probabilities were finite, with matched token counts. No OOM or CUDA errors were observed.

These are sampled regression tests, not full benchmark scores. Per-case results, repeat checks, scoring details and coverage are in VALIDATION-NOTES.md and evaluation/. This release does not include MTP or claim a measured throughput speedup.

Tested serving command

SGLANG_JIT_DEEPGEMM_PRECOMPILE=0 python -m sglang.launch_server \
  --model-path Accio-Lab/occamy-1.0-FP8 --served-model-name occamy \
  --host 127.0.0.1 --port 18400 --dtype bfloat16 \
  --context-length 16384 --max-running-requests 4 \
  --max-total-tokens 65536 --max-mamba-cache-size 16 \
  --mem-fraction-static 0.60 --attention-backend fa3 \
  --mm-attention-backend fa3 --mamba-backend triton \
  --disable-cuda-graph --random-seed 42 \
  --tool-call-parser qwen3_coder --enable-metrics

FP8 is detected from config.json; --dtype bfloat16 applies to preserved weights. Memory settings depend on hardware. The original qwen tool parser does not parse this checkpoint's XML tool format correctly; the tested parser is qwen3_coder.

With the repository downloaded and the server ready:

python evaluation/evaluate.py http://127.0.0.1:18400 sglang-fp8
python evaluation/evaluate_nll.py http://127.0.0.1:18400 sglang-fp8

For the matching baseline, replace --model-path with Accio-Lab/occamy-1.0, add --revision c1ce84770260c4137712cf22115574c4d06b993a, retain all other settings and run with result name sglang-bf16. Evaluation clients require Python 3.12 and Requests. Code tests strip Markdown fences and run in a separate process with a restricted AST/builtins policy and CPU/memory/time limits. Exact prompts, output budgets and expected results are in evaluation/cases.json.

Attribution and license

Original model: Accio-Lab/Occamy-1.0, Apache-2.0; original license included in LICENSE. Format metadata reference: Qwen/Qwen3.6-35B-A3B-FP8. Evaluation datasets retain their own licenses and attribution; see evaluation/DATA-LICENSES.md. This is a quantized Occamy derivative, not an Unsloth Dynamic export or an MTP release.

Downloads last month: -

Safetensors

Model size

35B params

Tensor type

BF16

F8_E4M3

Model tree for Accio-Lab/occamy-1.0-FP8

Base model

Qwen/Qwen3.6-35B-A3B

Finetuned

Accio-Lab/occamy-1.0

Quantized

(9)

this model