Instructions to use jdagostino/Splice-QFN-W4A16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jdagostino/Splice-QFN-W4A16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="jdagostino/Splice-QFN-W4A16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("jdagostino/Splice-QFN-W4A16")
model = AutoModelForMultimodalLM.from_pretrained("jdagostino/Splice-QFN-W4A16", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jdagostino/Splice-QFN-W4A16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jdagostino/Splice-QFN-W4A16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jdagostino/Splice-QFN-W4A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/jdagostino/Splice-QFN-W4A16

SGLang

How to use jdagostino/Splice-QFN-W4A16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jdagostino/Splice-QFN-W4A16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jdagostino/Splice-QFN-W4A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jdagostino/Splice-QFN-W4A16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jdagostino/Splice-QFN-W4A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use jdagostino/Splice-QFN-W4A16 with Docker Model Runner:
```
docker model run hf.co/jdagostino/Splice-QFN-W4A16
```

Splice-QFN

tl;dr: This is a merge of the Swift 1.5 and OrcaRouter-Uncensored finetunes of Qwen3.8-Flash-Next; it does a fairly decent job at combining the shorter chain-of-thought from Swift and the reduced refusal from the OrcaRouter finetune. My own home inference server has four AMD Radeon Pro V620 GPUs, the W4A16 quant here runs great in the leapdragon vLLM fork on that machine. Your own mileage may, of course, vary.

for lack of a better name, I'm calling this thing Splice-QFN.

I had gork do the merge and it wrote a long and poorly-written system card here that I deleted. Short version is that it went through every tensor and did this:

output = swift + 1.0 * (orcarouter - qwen3.8-flash-next)

The n-gram table is not included here and didn't change in the merge so download it elsewhere. If you're following that leapdragon franken-vllm you can grab it form the same place it suggests: primitive-ai/Qwen3.8-Flash-Next-PLE-quant, put it in the folder ples_int4/.

Quantization

routed experts are 4-bit and shared experts and other important stuff is bf16

Benchmark

I had gork spend a few hours to benchmark this against OrcaRouter's ablit and the stock Qwen weights, I'll let it explain:

Compared against the OrcaRouter W4A16 quant and the stock Qwen3.8-Flash-Next W4A16 quant on the same server. One sample, temperature 0, thinking left at the chat template default (xhigh). Do not pass reasoning_effort=low for this comparison: that flag writes "keep your thinking brief" into the prompt and makes the base model look short too. Generation cap 24,576 tokens. Run on 2026-10-04.

The 12 items were fixed before any of these outputs:

AIME 2024: I-8, I-15, II-8, II-15 (Maxwell-Jia/AIME_2024)

GPQA-Diamond rows 17, 53, 89, 125 in dataset order (fingertap/GPQA-Diamond)

HumanEval 32, 68, 85, 129, graded by executing the official tests

Correct Median thinking tokens Mean thinking tokens Hit the 24,576 cap

Splice 11 / 12 7,228 8,068 1

OrcaRouter W4A16 9 / 12 16,010 12,748 3

Stock Qwen W4A16 8 / 12 11,474 12,025 4

Paired over all 12 items, Splice's median thinking-token ratio is 0.66 versus OrcaRouter and 0.70 versus the stock model. Those ratios count a capped trace as 24,576, which hides how long the capped runs would have gone. On items both sides actually finished, the median ratio is about 0.48 versus OrcaRouter and about 0.70 versus the stock model.

Every miss is an unfinished trace. A capped run was still inside the thinking block and had not emitted a final answer. No finished answer from any model was wrong.

Item Splice OrcaRouter Stock

AIME 2024-I-8 correct, 6,873 correct, 14,206 correct, 10,241

AIME 2024-I-15 correct, 12,397 correct, 17,813 correct, 16,553

AIME 2024-II-8 correct, 7,584 correct, 21,883 correct, 12,707

AIME 2024-II-15 correct, 15,436 capped capped

GPQA row 17 correct, 459 correct, 616 correct, 461

GPQA row 53 correct, 502 correct, 609 correct, 897

GPQA row 89 correct, 8,271 correct, 18,653 capped

GPQA row 125 correct, 18,402 capped capped

HumanEval/129 correct, 1,994 correct, 4,830 correct, 4,779

HumanEval/68 correct, 135 correct, 442 correct, 186

HumanEval/85 correct, 181 correct, 201 correct, 168

HumanEval/32 capped capped capped

HumanEval/32 never produced code from any of the three models, so it says nothing. GPQA rows 17 and 53 are a few hundred tokens for everyone, so they do not show a chain-of-thought difference. The gap is the AIME problems and the harder GPQA items: Splice finished them, including the three the others did not finish inside the cap, and used substantially fewer thinking tokens where all three finished.

Twelve questions, one sample each. This is not Swift's published suite (GPQA-Diamond full, AIME 2026, LiveCodeBench, Terminal-Bench), and it is not a refusal benchmark. It is enough to see that the short traces survived the graft and that the finished answers were at least as good as the other two W4A16 checkpoints.

	Correct	Median thinking tokens	Mean thinking tokens	Hit the 24,576 cap
Splice	11 / 12	7,228	8,068	1
OrcaRouter W4A16	9 / 12	16,010	12,748	3
Stock Qwen W4A16	8 / 12	11,474	12,025	4

Item	Splice	OrcaRouter	Stock
AIME 2024-I-8	correct, 6,873	correct, 14,206	correct, 10,241
AIME 2024-I-15	correct, 12,397	correct, 17,813	correct, 16,553
AIME 2024-II-8	correct, 7,584	correct, 21,883	correct, 12,707
AIME 2024-II-15	correct, 15,436	capped	capped
GPQA row 17	correct, 459	correct, 616	correct, 461
GPQA row 53	correct, 502	correct, 609	correct, 897
GPQA row 89	correct, 8,271	correct, 18,653	capped
GPQA row 125	correct, 18,402	capped	capped
HumanEval/129	correct, 1,994	correct, 4,830	correct, 4,779
HumanEval/68	correct, 135	correct, 442	correct, 186
HumanEval/85	correct, 181	correct, 201	correct, 168
HumanEval/32	capped	capped	capped

anecdotally it seems to be at least as good as the OrcaRouter finetune alone, which I was using for a couple weeks before I cooked this thing up.

License

my attorney, who is also a large language model, has instructed me to state the following:

The Swift contribution in these weights is under the Swift Open License v1.0, included here as LICENSE. That license's redistributor notice covers a merge and a quant: keep that license, keep the attribution, and also comply with the Qwen Community License 1.0 for the base model. The Swift commercial threshold (gross revenue of one million USD, above which their enterprise license is required) still applies to the Swift part of this checkpoint. Nothing here grants use of the Swift, UkisAI, Qwen, or OrcaRouter names as if those projects shipped Splice.

Downloads last month: 68

Safetensors

Model size

127B params

Tensor type

I32

I64

BF16

Model tree for jdagostino/Splice-QFN-W4A16

Qwen/Qwen3.8-Flash-Next

orcarouter/Qwen3.8-Flash-Next-Uncensored

ukisai/Swift1.5-Qwen3.8-Flash-Next

Merge model

this model