Instructions to use iamrahulreddy/Quintus with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use iamrahulreddy/Quintus with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="iamrahulreddy/Quintus")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("iamrahulreddy/Quintus")
model = AutoModelForCausalLM.from_pretrained("iamrahulreddy/Quintus")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use iamrahulreddy/Quintus with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "iamrahulreddy/Quintus"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "iamrahulreddy/Quintus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/iamrahulreddy/Quintus

SGLang

How to use iamrahulreddy/Quintus with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "iamrahulreddy/Quintus" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "iamrahulreddy/Quintus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "iamrahulreddy/Quintus" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "iamrahulreddy/Quintus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use iamrahulreddy/Quintus with Docker Model Runner:
```
docker model run hf.co/iamrahulreddy/Quintus
```

Quintus / docs /benchmarks.md

iamrahulreddy

docs: add full benchmark matrix

9bf21aa verified 21 days ago

preview code

Raw

History Blame Contribute Delete

4.87 kB

Benchmarks

The release scoreboard compares Qwen3-1.7B-Base, Qwen3-1.7B-Instruct, and Quintus-1.7B. Evaluations use a mixture of EvalPlus and lm-evaluation-harness style benchmarks, with greedy or deterministic settings where applicable.

For the detailed benchmark-control rules, see Evaluation Methodology.

Final Scoreboard

Benchmark	Qwen3-1.7B-Base	Qwen3-1.7B-Instruct	Quintus-1.7B
HumanEval pass@1	67.1%	70.7%	67.7%
MBPP pass@1	67.2%	58.2%	64.8%
GSM8K, 10-shot flexible	69.98%	69.75%	74.30%
ARC-Challenge acc_norm	55.72%	52.99%	58.36%
WinoGrande, 5-shot	65.67%	61.01%	66.38%
PIQA acc_norm	75.63%	72.09%	75.57%

Full Checkpoint Matrix

The compact scoreboard above is the headline comparison. The full matrix below records the broader evaluation suite across four checkpoints:

Base: Qwen/Qwen3-1.7B-Base
Instruct: Qwen/Qwen3-1.7B-Instruct
Pre-SFT: online KD checkpoint before targeted SFT
Quintus SFT: final public Quintus checkpoint

$\Delta$ vs Instruct is computed as Quintus SFT minus Qwen/Qwen3-1.7B-Instruct, in percentage points. GSM8K strict and flexible scores are listed separately because parser behavior and EOS handling can change the measured result.

Area	Benchmark	Base	Instruct	Pre-SFT	Quintus SFT	$\Delta$ vs Instruct
Coding	HumanEval pass@1	67.1%	70.7%	68.3%	67.7%	-3.0 pp
Coding	HumanEval+	60.4%	64.0%	62.8%	60.4%	-3.6 pp
Coding	MBPP pass@1	67.2%	58.2%	63.0%	64.8%	+6.6 pp
Coding	MBPP+	58.2%	50.0%	54.5%	56.3%	+6.3 pp
Math	GSM8K flexible	70.0%	69.8%	74.4%	74.3%	+4.5 pp
Math	GSM8K strict	69.6%	69.8%	74.1%	60.9%	-8.9 pp
Reasoning/commonsense	WinoGrande, 5-shot	65.7%	61.0%	66.0%	66.4%	+5.4 pp
Reasoning/commonsense	ARC-Challenge acc	51.5%	49.5%	51.9%	54.8%	+5.3 pp
Reasoning/commonsense	ARC-Challenge acc_norm	55.7%	53.0%	55.6%	58.4%	+5.4 pp
Reasoning/commonsense	BoolQ	79.0%	77.5%	77.3%	71.6%	-5.9 pp
Reasoning/commonsense	PIQA acc	75.6%	72.9%	75.8%	75.2%	+2.3 pp
Reasoning/commonsense	PIQA acc_norm	75.6%	72.1%	75.7%	75.6%	+3.5 pp

Interpretation

The strongest result is the reasoning crossover: Quintus beats both the base and the official 1.7B instruct model on GSM8K, ARC-Challenge, and WinoGrande, despite remaining at the same parameter scale.

The coding picture is mixed but useful:

HumanEval remains slightly below Qwen3-1.7B-Instruct.
MBPP is substantially above Qwen3-1.7B-Instruct, though still below the base model.

This suggests the model gained useful instruction-following and reasoning behavior without fully matching larger or more heavily aligned code-specialized models.

What The Benchmarks Support

These results support four claims:

Online KD transferred reasoning capability into a compact student.
The final model did not merely memorize assistant formatting; it improved several reasoning and commonsense metrics.
SFT helped expose the distilled capability in an assistant setting.
The model still has capacity limits typical of the 1.7B scale, especially on code execution reliability and long multi-step algorithm generation.

Evaluation Caveats

Benchmark comparisons are sensitive to prompt format. Raw completion, chat-template generation, and log-likelihood multiple-choice scoring can produce different rankings. For fair interpretation:

Compare raw models against raw models when measuring base reasoning.
Compare chat-wrapped models against chat-wrapped models when measuring format alignment.
Treat open-ended qualitative prompts as alignment tests, not as a replacement for standardized benchmarks.

Important implementation caveats:

GSM8K extraction can differ between strict #### parsing and flexible number extraction.
Multiple-choice log-likelihood tasks can be distorted by chat templates.
acc_norm is preferred when answer-option length bias can change the ranking.
Metric extraction scripts must reject stderr and alias fields when looking for the actual score.
Runtime versions should be recorded with benchmark outputs because harness behavior can change across releases.

Earlier Development Signals

Before the final Qwen3 8B -> 1.7B run, earlier experiments showed that sparse offline top-k KD could not consistently outperform strong baselines. Those runs were useful because they identified the bottleneck: sparse cached teacher logits were not dense enough to transfer deeper reasoning pathways.

The final move to online full-vocabulary KD is the key methodological change behind the stronger final results.