Instructions to use ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized")
model = AutoModelForCausalLM.from_pretrained("ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized

SGLang

How to use ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized with Docker Model Runner:
```
docker model run hf.co/ewin-reg/MiniCPM5-2B-RotSVDMix-Quantized
```

MiniCPM5-2B-RotSVDMix-Quantized

Commit History

Sep 24

11:56
Update README.md

ewin-reg

Verified signature
b962434
11:51
Update README.md

ewin-reg

Verified signature
d638c22

Sep 17

16:36
docs: sync README_v14 with intra-runtime benchmark scorecard

ewin-reg

Verified signature
6905f23
16:36
data: update benchmark scorecard with intra-runtime and cross-runtime diagnostics

ewin-reg

Verified signature
ede032c
16:36
docs: update empirical benchmark with strict intra-runtime baseline isolation

ewin-reg

Verified signature
88cf2e0
16:20
docs: sync README_v14 with full GGUF benchmark metrics

ewin-reg

Verified signature
a151327
16:19
data: upload verified Colab GPU benchmark results

ewin-reg

Verified signature
7694285
16:19
docs: update empirical GGUF concordance metrics and plain English unslop explanation

ewin-reg

Verified signature
284c073
15:58
Overhaul README: Unslop, plain English explanation, copy-paste quickstart, and clear empirical benchmarks

ewin-reg

Verified signature
2f3a34b
15:51
Publish empirical Colab Tesla T4 GGUF benchmark scorecard

ewin-reg

Verified signature
e7cc2eb
15:51
Update comparative benchmark table with empirical Colab Tesla T4 GGUF measurements

ewin-reg

Verified signature
2e79e44
15:42
Upload MiniCPM5_2B_GGUF_Colab_Benchmark.ipynb with huggingface_hub

ewin-reg

Verified signature
f95d875
14:24
Refine GGUF comparison table: replace extrapolated PPL with verified repository metrics

ewin-reg

Verified signature
d854401
14:20
Add comparative analysis table and takeaways against GGUF Q4_K_M and Q6_K

ewin-reg

Verified signature
577594f
14:17
Overhaul README with real scraped analytics, official badges, and scraper configs

ewin-reg

Verified signature
9926f0a
14:14
Overhaul README with real analytics, hosting app scraper configs (vLLM/TGI/Ollama), and zero assumptions

ewin-reg

Verified signature
0202ebc
14:11
Overhaul README for SEO, GEO, and AEO with structured FAQ and clear benchmarks

ewin-reg

Verified signature
a2f4510
14:09
Clean README: remove all broken inline dollar signs for clean markdown rendering

ewin-reg

Verified signature
3a04127
14:07
Rewrite README.md with clean, simple LaTeX formulations

ewin-reg

Verified signature
f79a7a8
14:02
Overhaul README.md for v14 ExTernD low-rank ternary release

ewin-reg

Verified signature
4e31044
13:58
Update real_benchmark_results.json for v14 ExTernD release

ewin-reg

Verified signature
07cf6e0
13:58
Update scorecard.json for v14 ExTernD release

ewin-reg

Verified signature
a967b33
13:58
Release v14: ExTernD Low-Rank Ternary Stacking (PPL 20.92, 1.985 GB, 98.7% Cosine)

ewin-reg

Verified signature
0c85ff1
07:16
Sync real_benchmark_results.json with v13 metrics

ewin-reg

Verified signature
b2ab41f
07:16
Update README.md for Rot-SVD-Mix v13 Tapered Hierarchy release

ewin-reg

Verified signature
64a36ec
07:14
Publish v13 scorecard

ewin-reg

Verified signature
d46eb11
07:14
Publish Rot-SVD-Mix v13 Tapered Hierarchy (PPL 20.96, +2.42%, Cos 98.64%, Top1 96.0%)

ewin-reg

Verified signature
ad29add
06:39
Publish v16 scorecard

ewin-reg

Verified signature
8e7e277
06:38
Publish Rot-SVD-Mix v16 Champion (PPL 20.88, +2.02%, Cos 98.57%, Top1 92.0%)

ewin-reg

Verified signature
4aef437
02:15
Publish v11 scorecard

ewin-reg

Verified signature
c40796b
02:15
Publish Rot-SVD-Mix v11 G16-Hybrid (PPL 21.08, +3.04%)

ewin-reg

Verified signature
a21ca76
00:41
Publish v8 scorecard

ewin-reg

Verified signature
0d31aed
00:41
Publish Rot-SVD-Mix v8 (PPL 21.31, +4.15%)

ewin-reg

Verified signature
4008121

Sep 16

23:51
Publish v6 scorecard

ewin-reg

Verified signature
2883a7d
23:51
Publish Rot-SVD-Mix v6 with LDLQ sequential compensation

ewin-reg

Verified signature
9e3bfcc
23:34
Update README.md with verified v5 empirical benchmarks (< 2.00 GB decimal)

ewin-reg

Verified signature
849d534
23:33
Publish Stacked Quant v5 scorecard

ewin-reg

Verified signature
f83d4e5
23:33
Publish Stacked Quant v5 empirical benchmarks

ewin-reg

Verified signature
e85c76c
23:33
Publish Stacked Quant v5 SafeTensors weights (< 2.00 GB Decimal)

ewin-reg

Verified signature
9c3abfc
23:20
Update model card with simple clean LaTeX formatting

ewin-reg

Verified signature
4197f01
23:17
Update model card with simple clean LaTeX formatting

ewin-reg

Verified signature
7f91ca1
17:50
Update model card with Stacked Quantization v4 precision empirical benchmarks

ewin-reg

Verified signature
040caba
17:45
Publish Stacked Quant v4.3 scorecard

ewin-reg

Verified signature
b57db86
17:45
Publish Stacked Quant v4.3 empirical benchmarks

ewin-reg

Verified signature
722a323
17:45
Publish Stacked Quant v4.3 SafeTensors weights

ewin-reg

Verified signature
4251c0a
17:34
Publish Stacked Quant v4.2 scorecard

ewin-reg

Verified signature
76f21e4
17:34
Publish Stacked Quant v4.2 empirical benchmarks

ewin-reg

Verified signature
5ada25b
17:34
Publish Stacked Quant v4.2 SafeTensors weights

ewin-reg

Verified signature
1113ead
17:19
Publish Stacked Quant v4 scorecard

ewin-reg

Verified signature
cf0a0e4
17:19
Publish Stacked Quant v4 empirical benchmarks

ewin-reg

Verified signature
e388d43

Commit History

Sep 24

Update README.md

Update README.md

Sep 17

docs: sync README_v14 with intra-runtime benchmark scorecard

data: update benchmark scorecard with intra-runtime and cross-runtime diagnostics

docs: update empirical benchmark with strict intra-runtime baseline isolation

docs: sync README_v14 with full GGUF benchmark metrics

data: upload verified Colab GPU benchmark results

docs: update empirical GGUF concordance metrics and plain English unslop explanation

Overhaul README: Unslop, plain English explanation, copy-paste quickstart, and clear empirical benchmarks

Publish empirical Colab Tesla T4 GGUF benchmark scorecard

Update comparative benchmark table with empirical Colab Tesla T4 GGUF measurements

Upload MiniCPM5_2B_GGUF_Colab_Benchmark.ipynb with huggingface_hub

Refine GGUF comparison table: replace extrapolated PPL with verified repository metrics

Add comparative analysis table and takeaways against GGUF Q4_K_M and Q6_K

Overhaul README with real scraped analytics, official badges, and scraper configs

Overhaul README with real analytics, hosting app scraper configs (vLLM/TGI/Ollama), and zero assumptions

Overhaul README for SEO, GEO, and AEO with structured FAQ and clear benchmarks

Clean README: remove all broken inline dollar signs for clean markdown rendering

Rewrite README.md with clean, simple LaTeX formulations

Overhaul README.md for v14 ExTernD low-rank ternary release

Update real_benchmark_results.json for v14 ExTernD release

Update scorecard.json for v14 ExTernD release

Release v14: ExTernD Low-Rank Ternary Stacking (PPL 20.92, 1.985 GB, 98.7% Cosine)

Sync real_benchmark_results.json with v13 metrics

Update README.md for Rot-SVD-Mix v13 Tapered Hierarchy release

Publish v13 scorecard

Publish Rot-SVD-Mix v13 Tapered Hierarchy (PPL 20.96, +2.42%, Cos 98.64%, Top1 96.0%)

Publish v16 scorecard

Publish Rot-SVD-Mix v16 Champion (PPL 20.88, +2.02%, Cos 98.57%, Top1 92.0%)

Publish v11 scorecard

Publish Rot-SVD-Mix v11 G16-Hybrid (PPL 21.08, +3.04%)

Publish v8 scorecard

Publish Rot-SVD-Mix v8 (PPL 21.31, +4.15%)

Sep 16

Publish v6 scorecard

Publish Rot-SVD-Mix v6 with LDLQ sequential compensation

Update README.md with verified v5 empirical benchmarks (< 2.00 GB decimal)

Publish Stacked Quant v5 scorecard

Publish Stacked Quant v5 empirical benchmarks

Publish Stacked Quant v5 SafeTensors weights (< 2.00 GB Decimal)

Update model card with simple clean LaTeX formatting

Update model card with simple clean LaTeX formatting

Update model card with Stacked Quantization v4 precision empirical benchmarks

Publish Stacked Quant v4.3 scorecard

Publish Stacked Quant v4.3 empirical benchmarks

Publish Stacked Quant v4.3 SafeTensors weights

Publish Stacked Quant v4.2 scorecard

Publish Stacked Quant v4.2 empirical benchmarks

Publish Stacked Quant v4.2 SafeTensors weights

Publish Stacked Quant v4 scorecard

Publish Stacked Quant v4 empirical benchmarks