Instructions to use Entity-27th/Magnum-Opus-35B-A3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Entity-27th/Magnum-Opus-35B-A3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Entity-27th/Magnum-Opus-35B-A3B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Entity-27th/Magnum-Opus-35B-A3B")
model = AutoModelForMultimodalLM.from_pretrained("Entity-27th/Magnum-Opus-35B-A3B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Entity-27th/Magnum-Opus-35B-A3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Entity-27th/Magnum-Opus-35B-A3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Entity-27th/Magnum-Opus-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Entity-27th/Magnum-Opus-35B-A3B

SGLang

How to use Entity-27th/Magnum-Opus-35B-A3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Entity-27th/Magnum-Opus-35B-A3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Entity-27th/Magnum-Opus-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Entity-27th/Magnum-Opus-35B-A3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Entity-27th/Magnum-Opus-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Studio

How to use Entity-27th/Magnum-Opus-35B-A3B with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Entity-27th/Magnum-Opus-35B-A3B to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Entity-27th/Magnum-Opus-35B-A3B to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Entity-27th/Magnum-Opus-35B-A3B to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="Entity-27th/Magnum-Opus-35B-A3B",
    max_seq_length=2048,
)

Docker Model Runner
How to use Entity-27th/Magnum-Opus-35B-A3B with Docker Model Runner:
```
docker model run hf.co/Entity-27th/Magnum-Opus-35B-A3B
```

details about the model

by Roman1111111 - opened Mar 1

Discussion

Roman1111111

Mar 1

could you pls provide details including what dataset did you use to fine tune

Entity-27th

Owner Mar 1

@Roman1111111 I used nohurry/Opus-4.6-Reasoning-3000x-filtered dataset. Updated the model card

Roman1111111

Mar 1

do you plan on testing it on benchmarks to see how the opus 4.6 dataset align on it?

Entity-27th

Owner Mar 1

•

edited Mar 1

@Roman1111111 I indeed wanted to, but currently I am busy so I might not be able to evaluate the model myself. I am not really expecting much from this model because the dataset is small, and there is a possibility I messed with wrong hyperparameters, etc(Do note that I am no expert, merely a student). But feel free to evaluate or use the model for any purpose you see fit.

Roman1111111

Mar 1

oh, no worries, i will try to test it, im not the expert either))

Entity-27th

Owner Mar 1

@Roman1111111 Thank you! I am also curious about the model's performance for I was quite interested with the original Qwen3.5's intelligence. I do forward to see how much of the model's performance was boosted with the distillation

Roman1111111

Mar 1

thank you too for the model itself

Entity-27th

Owner Mar 1

@Roman1111111 No worries. Don't forget to share your evaluation results if possible :)

Roman1111111

Mar 1

Of course, just waiting for the download to finish

CryptoAIM

Mar 1

you are everywhere roman

Roman1111111

Mar 2

u too❤️

you are everywhere roman

CryptoAIM

Mar 2

❤️ roman sniffing new data for the gemini collection

Roman1111111

Mar 3

hello, currently i tested just for to confirm eveything works on gsm8k benchmark - and it showed outstanding results with reduce of tokens use by 40-50% and better scores - by 17% than the base in this becnh, here are scores - original qwen3.5 35b a3b-77%/71.5%/35.15min, this model - 94%/84.5%/21.65min

here is the qwen 3.5 original model - 77%/71.5% - (base) romanyemelyanov@WIN-3KDMBNSD145:~/lm-evaluation-harness$ lm_eval --model local-chat-completions --tasks gsm8k --model_args model=qwen/qwen3.5-35b-a3b,base_url=http://172.29.128.1:1234/v1/chat/completions,num_concurrent=4 --ap
ply_chat_template --num_fewshot 5 --gen_kwargs max_tokens=5000 --output_path ./gsm8k_final_results --limit 200
2026-03-02:17:09:16 WARNING [config.evaluate_config:281] --limit SHOULD ONLY BE USED FOR TESTING. REAL METRICS SHOULD NOT BE COMPUTED USING LIMIT.
2026-03-02:17:09:16 INFO [config.evaluate_config:301] Using default fewshot_as_multiturn=True.
2026-03-02:17:09:19 INFO [_cli.run:376] Selected Tasks: ['gsm8k']
2026-03-02:17:09:19 INFO [evaluator:211] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-03-02:17:09:19 WARNING [evaluator:223] generation_kwargs: {'max_tokens': 5000} specified through cli, these settings will update set parameters in yaml tasks. Ensure 'do_sample=True' for non-greedy decoding!
2026-03-02:17:09:19 INFO [evaluator:236] Initializing local-chat-completions model, with arguments: {'model': 'qwen/qwen3.5-35b-a3b', 'base_url': 'http://172.29.128.1:1234/v1/chat/completions', 'num_concurrent': 4}
2026-03-02:17:09:19 INFO [models.api_models:172] Using max length 2048 - 1
2026-03-02:17:09:19 INFO [models.api_models:193] Using tokenizer None
2026-03-02:17:09:24 INFO [tasks:700] Selected tasks:
2026-03-02:17:09:24 INFO [tasks:691] Task: gsm8k (gsm8k/gsm8k.yaml)
2026-03-02:17:09:24 INFO [evaluator:314] gsm8k: Using gen_kwargs: {'until': ['Question:', '', '<|im_end|>'], 'do_sample': False, 'temperature': 0.0, 'max_tokens': 5000}
2026-03-02:17:09:24 WARNING [evaluator:333] Overwriting default num_fewshot of gsm8k from 5 to 5
2026-03-02:17:09:24 WARNING [evaluator:490] Chat template formatting change affects loglikelihood and multiple-choice tasks. See docs/chat-template-readme.md for details.
2026-03-02:17:09:24 INFO [api.task:311] Building contexts for gsm8k on rank 0...
100%|████████████████████████████████████████████████████████████████████████████████| 200/200 [00:00<00:00, 506.09it/s]
2026-03-02:17:09:24 INFO [evaluator:584] Running generate_until requests
2026-03-02:17:09:24 INFO [models.api_models:733] Tokenized requests are disabled. Context + generation length is not checked.
Requesting API: 50%|████████████████████████████████▌ | 100/200 [17:41<21:31, 12.92s/it]Requesting API: 70%|█████████████████████████████████████████████▊ | 141/200 [25:03<11:42, 11.91s/it]Requesting API: 100%|█████████████████████████████████████████████████████████████████| 200/200 [35:11<00:00, 10.56s/it]
2026-03-02:17:44:41 INFO [loggers.evaluation_tracker:247] Saving results aggregated
local-chat-completions ({'model': 'qwen/qwen3.5-35b-a3b', 'base_url': 'http://172.29.128.1:1234/v1/chat/completions', 'num_concurrent': 4}), gen_kwargs: ({'max_tokens': 5000}), limit: 200.0, num_fewshot: 5, batch_size: 1

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.770	±	0.0298
		strict-match	5	exact_match	↑	0.715	±	0.0320
(base) romanyemelyanov@WIN-3KDMBNSD145:~/lm-evaluation-harness$

and here is your model - (base) romanyemelyanov@WIN-3KDMBNSD145:~/lm-evaluation-harness$ lm_eval --model local-chat-completions --tasks gsm8k --model_args model=qwen3.5-opus-4.6,base_url=http://172.29.128.1:1234/v1/chat/completions,num_concurrent=4 --apply_chat_template --num_fewshot 5 --gen_kwargs max_tokens=5000 --output_path ./gsm8k_final_results --limit 200
2026-03-02:16:46:05 WARNING [config.evaluate_config:281] --limit SHOULD ONLY BE USED FOR TESTING. REAL METRICS SHOULD NOT BE COMPUTED USING LIMIT.
2026-03-02:16:46:05 INFO [config.evaluate_config:301] Using default fewshot_as_multiturn=True.
2026-03-02:16:46:08 INFO [_cli.run:376] Selected Tasks: ['gsm8k']
2026-03-02:16:46:08 INFO [evaluator:211] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-03-02:16:46:08 WARNING [evaluator:223] generation_kwargs: {'max_tokens': 5000} specified through cli, these settings will update set parameters in yaml tasks. Ensure 'do_sample=True' for non-greedy decoding!
2026-03-02:16:46:08 INFO [evaluator:236] Initializing local-chat-completions model, with arguments: {'model': 'qwen3.5-opus-4.6', 'base_url': 'http://172.29.128.1:1234/v1/chat/completions', 'num_concurrent': 4}
2026-03-02:16:46:08 INFO [models.api_models:172] Using max length 2048 - 1
2026-03-02:16:46:08 INFO [models.api_models:193] Using tokenizer None
2026-03-02:16:46:12 INFO [tasks:700] Selected tasks:
2026-03-02:16:46:12 INFO [tasks:691] Task: gsm8k (gsm8k/gsm8k.yaml)
2026-03-02:16:46:12 INFO [evaluator:314] gsm8k: Using gen_kwargs: {'until': ['Question:', '', '<|im_end|>'], 'do_sample': False, 'temperature': 0.0, 'max_tokens': 5000}
2026-03-02:16:46:12 WARNING [evaluator:333] Overwriting default num_fewshot of gsm8k from 5 to 5
2026-03-02:16:46:12 WARNING [evaluator:490] Chat template formatting change affects loglikelihood and multiple-choice tasks. See docs/chat-template-readme.md for details.
2026-03-02:16:46:12 INFO [api.task:311] Building contexts for gsm8k on rank 0...
100%|████████████████████████████████████████████████████████████████████████████████| 200/200 [00:00<00:00, 524.23it/s]
2026-03-02:16:46:12 INFO [evaluator:584] Running generate_until requests
2026-03-02:16:46:12 INFO [models.api_models:733] Tokenized requests are disabled. Context + generation length is not checked.
Requesting API: 100%|█████████████████████████████████████████████████████████████████| 200/200 [21:41<00:00, 6.51s/it]
2026-03-02:17:07:58 INFO [loggers.evaluation_tracker:247] Saving results aggregated
local-chat-completions ({'model': 'qwen3.5-opus-4.6', 'base_url': 'http://172.29.128.1:1234/v1/chat/completions', 'num_concurrent': 4}), gen_kwargs: ({'max_tokens': 5000}), limit: 200.0, num_fewshot: 5, batch_size: 1

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.940	±	0.0168
		strict-match	5	exact_match	↑	0.845	±	0.0257
(base) romanyemelyanov@WIN-3KDMBNSD145:~/lm-evaluation-harness$

Roman1111111

Mar 3

now waiting for all becnhamrks scores, then to compare more reliably

Roman1111111

Mar 3

i was running q4_k_m precision

Roman1111111

Mar 3

on both

Entity-27th

Owner Mar 3

@Roman1111111 Fascinating! Thank you for sharing your results on GSM8K. This is way better than I initially anticipated for this distillation variant. Training smaller models on frontier LLM reasoning dataset is indeed useful on boosting model reasoning performance. I am especially intrigued on the fact that it used substantially less tokens. Again, thank you for sharing this impressive result!

Roman1111111

Mar 3

your welcome, wait for the upcoming --tasks mmlu,hellaswag,arc_challenge,truthfulqa_mc2,winogrande,gsm8k,bbh_cot_fewshot,triviaqa,piqa,sciq,aime25,mmlu_pro results, and you actually made impressive model fine tuning

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment